Roughly once a year, my co-founder, Can Esenbel, takes over writing duties. He has a knack for startling insight and thoughtful perspective, weaving many strands together, and has done so again in this letter, which I hope you enjoy.
Faisal Sheikh
--
In the context of the disturbing invasion of Ukraine by Russia in late February, I want to use this piece to build on Faisal’s two excellent previous quarterly letters [1] and look more at the art and science of forecasting. With the invasion came the usual flood of opinion and predictions. Many are articulate, well informed and coherent. Most, if not all, are wrong in varying degrees. This is not to be judgmental but to point out a simple truth – prediction in complex systems is inordinately hard.
There is an apocryphal story of some analysis conducted in 1967 at the height of the Vietnam war. Wanting to gauge how close the US was to victory, the generals ordered an analysis using the fancy new computer at the Pentagon. They loaded as much all the data they had into the computer – number of soldiers, planes, tanks, resources, economic strength etc. and left it to run over the weekend.
On Monday morning the answer was waiting: ‘You won in 1965’ [2]
Even with better tools not much has changed, look at the forecast below from Metaculus [3] (a crowd forecasting website) on whether Russia would invade Ukraine. It tended toward the correct answer eventually but for most of the period considered the likelihood low.
This is fair enough. In December forecasts were based on what they knew in December. As the situation developed and previously unknown (and unknowable) information became available they updated their forecast.
Financial markets did not do a better job of pricing in the risk, Russian credit default swaps (insurance against bond default) went from sanguine to paying out territory in under 2 months [4]:
The question then becomes, how useful are all these predictions to capital allocators?
A few years ago, there was a spat between Nate Silver of FiveThirtyEight and Nassim Taleb. Silver was the darling of the election forecasting industry, lauded for his highly accurate predictions in the 2008 and 2012 US elections. Taleb is tormentor in chief of all those he disagrees with (and too often quoted by me despite my aversion to his nastiness).
Taleb took issue with Silver’s election model during the 2016 US presidential election on the basis that Silver’s winning probabilities oscillated wildly as the model updated based on the latest polls [5].
Taleb believed that if the model oscillated to such a degree it was accounting for only the current uncertainty and ignoring the future uncertainty of the system (the unknown unknowns). This, in his view, made Silver’s model a poor forecasting tool. The probabilities should, for a two-horse race, oscillate closer to 50-50 rather than the wider ranges that Silver’s model showed. On the other side, Silver considered the final model output the only one that matters.
Isaac Faber, in an excellent article on this topic, explains the uncertainty question as follows:
“What is not clear is that there is a factor hidden from the FiveThirtyEight reader. Predictions have two types of uncertainty; aleatory and epistemic. Aleatory uncertainty is concerned with the fundamental system (probability of rolling a six on a standard die). Epistemic uncertainty is concerned with the uncertainty of the system (how many sides does a die have? And what is the probability of rolling a six?). With the latter, you have to guess the game and the outcome; like an election!” [6]
Geopolitics and investment are brimming with epistemic uncertainty. It is reasonable to have predicted whether Russia would invade (a binary event) but what then? The overwhelming consensus was that the Russian military monster would overwhelm the Ukrainians and be in Kyiv in 3 days. (To check my bias, I spent some time trawling through articles on Ukraine written in January and February. To date I haven’t found any that suggested the Russian military effort would be the shambles we have seen so far.)
Even a well-informed insider struggled here:
“However, if Ukrainian troops operate in large formations, they will become easy targets for Russian airstrikes. Indeed, any Russian offensive would likely begin with a devastating air and missile campaign designed to wipe out Ukrainian armored units, fighter jets, and key elements of the country’s military infrastructure.
“In such circumstances, it is likely that the Russian military will be able to advance deep into Ukrainian territory. However, holding this territory will be a different matter entirely.” [7]
Or this in Foreign Policy by an ex-US army soldier and professor at the war college:
“While the Ukrainian army appears, on paper, to have rough parity with the Russian forces surrounding the country and occupying the Crimean Peninsula and the Donbas region in eastern Ukraine, its combat readiness, training, and air force and missile assets cannot compare with those arrayed against it. Kyiv will likely fall within days, if not hours, of a full-on Russian attack, with horrific human suffering among innocent Ukrainian civilians; casualties could number in the tens of thousands.” [8]
These views are entirely reasonable, they are well informed and thoughtful. As an outsider it makes sense to accept and absorb this view.
I’ve picked one event, but for everything that is happening around us there is opinion, there is analysis, there is forecasting. We are the first cohort of humanity that is taking in an almost constant flow of information and our brains are not designed to handle it.
This evolution of the environment is shaping our minds and beliefs. This has always been so to a large extent but what we have in the present day is this effect on steroids. The writer David Perrell puts our situation evocatively:
“The structure of our social media feeds place us in a Never-Ending Now. Like hamsters running on a wheel, we live in an endless cycle of ephemeral content consumption — a merry-go-round that spins faster and faster but barely goes anywhere.” [9]
Attention and focus are limited. To be successful investors we should place our attention and focus where it will be usefully employed.
I hope the above shows you a little of the difficulties of knowing what will happen in environments with high uncertainty and that we are not immune from these effects. If we accept this, then it is illogical to ignore it and carry on as before. But this is what most of us do, industry expectations and our conditioning keep us in this trap. I remember the reaction to Faisal’s Q3 2021 letter; many latched on to the example he chose to demonstrate his main point and overlooked the underlying message.
There is power in saying ‘I don’t know’ constructively, particularly in an industry that favours knowing and tidy narrative. In doing so we open some space to work with the reality of uncertainty rather than remaining in the prison of compelling noise. To do this is to carve out a competitive edge in a world where many edges are unstable or degraded.
- CKE Apr '22
Notes
تعليقات