In an unprecedented “forecasting tournament,” five teams will compete to see who can most accurately predict future political and economic developments. One of the five is Tetlock’s “Good Judgment” Team, which will measure individual differences in thinking styles among 2,400 volunteers (e.g., fox versus hedgehog) and then assign volunteers to experimental conditions designed to encourage alternative problem-solving approaches to forecasting problems. The volunteers will then make individual forecasts which statisticians will aggregate in various ways in pursuit of optimal combinations of perspectives. It’s hoped that combining superior styles of thinking with the famous “wisdom of crowds” will significantly boost forecast accuracy beyond the untutored control groups of forecasters who are left to fend for themselves.
Other teams will use different methods, including prediction markets and Bayesian networks, but all the results will be directly comparable, and so, with a little luck, we will learn more about which methods work better and under what conditions. This sort of research holds out the promise of improving our ability to peer into the future.
But only to some extent, unfortunately. Natural science has discovered in the past half-century that the dream of ever-growing predictive mastery of a deterministic universe may well be just that, a dream. There increasingly appear to be fundamental limits to what we can ever hope to predict. Take the earthquake in Japan. Once upon a time, scientists were confident that as their understanding of geology advanced, so would their ability to predict such disasters. No longer. As with so many natural phenomena, earthquakes are the product of what scientists call “complex systems,” or systems which are more than the sum of their parts. Complex systems are often stable not because there is nothing going on within them but because they contain many dynamic forces pushing against each other in just the right combination to keep everything in place. The stability produced by these interlocking forces can often withstand shocks but even a tiny change in some internal conditional at just the right spot and just the right moment can throw off the internal forces just enough to destabilize the system—and the ground beneath our feet that has been so stable for so long suddenly buckles and heaves in the violent spasm we call an earthquake. Barring new insights that shatter existing paradigms, it will forever be impossible to make time-and-place predictions in such complex systems. The best we can hope to do is get a sense of the probabilities involved. And even that is a tall order.
Human systems like economies are complex systems, with all that entails. And bear in mind that human systems are not made of sand, rock, snowflakes, and the other stuff that behaves so unpredictably in natural systems. They’re made of people: self-aware beings who see, think, talk, and attempt to predict each other’s behavior—and who are continually adapting to each other’s efforts to predict each other’s behavior, adding layer after layer of new calculations and new complexity. All this adds new barriers to accurate prediction.
When governments the world over were surprised by this year’s events in the Middle East, accusing fingers were pointed at intelligence agencies. Why hadn’t they seen it coming? “We are not clairvoyant,” James R. Clapper Jr, director of national intelligence, told a hearing of the House intelligence committee. Analysts were well aware that forces capable of generating unrest were present in Tunisia, Egypt, and elsewhere. They said so often. But those forces had been present for years, even decades. “Specific triggers for how and when instability would lead to the collapse of various regimes cannot always be known or predicted,” Clapper said.