Arguments about the slippery concepts of change, probability and forecasting have been going on since Parmenides vs Heraclitus, and no doubt long before. A modern version of this debate can be staged—somewhat artificially, but usefully—as Taleb versus Tetlock.
Nassim Nicholas Taleb’s Black Swan theory warns that some of the events that matter most are sudden, radical and effectively unpredictable. Taleb cautions that tomorrow’s biggest risks cannot necessarily be anticipated from yesterday’s patterns—and that believing otherwise produces dangerously fragile models of the world. Taleb raises a further objection: success on bounded forecasting questions is not necessarily the same as useful prediction of consequential real-world events. If superforecasters possessed an economically valuable predictive edge, he asks in effect, wouldn’t they already be super-rich?
On the other side, Philip Tetlock, co-author of the book Superforecasting, argues that much change is forecastable, and to reject that is to discard too much useful information. “History does sometimes jump,” Tetlock and co-author Dan Gardner write. “But it also crawls, and slow, incremental change can be profoundly important.” Importantly, Tetlock and Gardner have shown that some human forecasters can be consistently better than others over prolonged periods of time and across many different sorts of useful questions (even if not immediately profitable ones).
A reading tip: You can get a good handle on this debate directly from Superforecasting, which engages seriously with Taleb’s objections, without also having to read The Black Swan. But if you do want to read some Taleb, Fooled by Randomness is my favourite book by him, and it details one of the craziest cases of forecasting hubris: the collapse of Long-Term Capital Management (LTCM) in the late 1990s. Alternatively, if you’re new to questions of probability and forecasting, or want to read around these topics, it might help to start with Spiegelhalter’s The Art of Uncertainty, which I think is a bit more fun and approachable than either of the others.
A pragmatic compromise: models that work when they break
One place I've seen the tension between different views of change play out is in quantitative finance. This field has contributed plenty to our understanding of risk and forecasting, mainly because traders are tired of losing money to the Tetlock-Taleb extremes and have tried to use mathematics to find a safer middle ground.
At a basic level, quant trading offers a way to behave as though both Taleb and Tetlock can be correct: it is accepted that there are periods when the future resembles the past and models can be profitable, but it is equally acknowledged that there are distinct periods of regime change when risk management is critical to survival.
But the pragmatism runs deeper than that, in the form of a specific type of forecasting model called a factor model, used in factor-based trading, which can be designed to extract quite detailed information from both continuity and change.
Among trend-following, market-making, arbitrage and many other quant strategies offered to investors, factor-based trading is one that takes a little longer to explain. However, the concept is powerful enough to be worth the effort – not least because factor models are used across many other disciplines, from engineering to econometrics, which also have to deal with uncertainty.
Factor models and what makes them special
Imagine you have an exchange with 100 different stocks being traded on it. Each stock has its own price history, so you have 100 time series of data — that's a lot of data containing a lot of variability. But you observe that some stock returns tend to move together rather than being completely random or independent: those in the same industry often rise and fall in tandem, as may those of companies in the same country or with similar characteristics.
You might find, for illustration, that much of the variation across all these 100 returns can be explained by just ten or so common factors. Each return, at any moment, reflects some weighting ("loading") of each of these ten factors, plus a residual element that appears to be idiosyncratic because the factor model cannot explain it. That residual is a critical element.
While the world remains reasonably stable, markets and returns may be volatile, but this volatility remains explainable by the factors, while the residuals may continue to behave much as they have in the past. In some statistical-arbitrage strategies, traders specifically look for residuals that have historically been mean-reverting. Prices that deviate from the factor model then become trading opportunities, because the deviation is expected to be fleeting.
The residual on its own is no magic bullet. On the contrary, assumptions of mean-reversion and convergence are what brought down LTCM (alongside excessive leverage), because these assumptions held true for four years before breaking catastrophically in the fifth.
However, the residual can also be used diagnostically. If the world begins to behave differently, the model can start failing in detectable ways. Residuals may become larger, more persistent or more correlated across multiple series. Factor loadings may shift, or the relationships among the factors may change. None of this identifies the cause automatically: it might reflect bad data, an inadequate model, an idiosyncratic shock or a genuine structural break. But it gives the investigator somewhere precise to look.
Exploring new applications for factor models
A key element in using factor models is time. More history can make estimates more reliable when relationships are stable—but it can also anchor the model to a world that no longer exists. The practical challenge is to gather enough evidence to distinguish a structural change from temporary volatility without waiting until the change is obvious to everyone.
Time can be painfully expensive in trading, where factor-led positions (particularly within statistical arbitrage strategies) may be entered and exited within hours or days. But other professions concerned with forecasting and change-spotting don't have to be so hasty: business newsrooms, for example, or management consultancies or VC fund managers, where analyses of change can develop over weeks, months or even years. They do not have unlimited time, but nor must they attach an immediate leveraged position to every signal.
AI makes factor modelling more accessible to such users than it ever has been in the past. It can help a small team find data suppliers, write API connectors, standardise data, document transformations and maintain a proprietary data library assembled from public and private sources. The statistical calculations can then be performed reproducibly by conventional software rather than by the LLM itself.
Some early experiments
My preliminary experiments suggest that a journalist who asks frontier AI a complex, future-looking question stands a chance of getting a better answer if they also supply the LLM with a pack of hard, source-attributed data series. For example, if I ask Anthropic's Fable 5 whether energy is becoming a limiting factor in UK data centre growth, I can get a slightly more concrete, more auditable answer if I also give the LLM a pack of data on grid loads, auctions and energy prices (fetched from reliable sources via API requests beforehand), rather than solely relying on the LLM's live internet search.
If I return three months later, the accumulating series give the investigation continuity. They do not guarantee a better answer, as far as I've been able to tell: the new observations may be uninformative, noisy or misleading. But they make it possible to determine whether the evidence has changed, which series are responsible and whether the model’s conclusion has changed with them.
These are early tests and do not yet provide evidence for my broader thesis about factor models. Moving from a curated data pack to a statistical factor model is also a big methodological step. It requires decisions about which outcomes to model, how to transform and align the series, whether factors should be specified in advance or inferred statistically, and how to distinguish structural change from bad data or model misspecification.
Nevertheless, the direction of travel is clear. The goal is to create a sufficiently broad, clean and persistent data library to support a factor model that can establish normal relationships and monitor departures from them that become candidates for further, human-led investigation.
This would not resolve the argument between Tetlock and Taleb, or predict a Black Swan before it occurred. Its more modest purpose would be to extract information from both continuity and model failure. A specialist reporter, unlike a leveraged trader, could then take the time to investigate provisional signals – especially those which connect with stories they're already working on. If they can identify the emerging change before it has acquired an obvious name or narrative, that could still constitute a significant scoop.
