The art and science of predicting
At the moment in which one seeks to predict with greater accuracy the short and medium term Brazil, the predictions specialist Dan Gardner reminds us that the path to predict scenarios is always to test – with focus, humility and an open mind.
October-December | 2016To be intellectually humble, and yet curious, and to maintain the field of vision and interests always open. These are, according to Dan Gardner, essential qualities for anyone who wants to be a superforecaster – a person who is able to do great and accurate predictions.
The Canadian journalist has teamed his experience as a writer to the consistent researches of Philip Tetlock on the subject and, together, they launched the best-seller “Superforecasting – The Art and Science of Prediction”. Tetlock is a professor of Administration at Wharton School and of Psychology at the Faculty of Arts and Sciences of the University of Pennsylvania, in addition to being a researcher, for decades, about predictions.
Although the theory of “art and science” of predictions described in the book has called the attention especially of politicians and national agencies of information, the authors argue that it can also be useful for those who make decisions in the business world – such as executives, entrepreneurs and investors who are today seeking to anticipate scenarios that will impact Brazil and the world in 2017 and beyond. Read below the interview given by Gardner to Mundo Corporativo.
How can business managers apply the principles of the theory described in your book?
The first step is to recognize that we are all quite a bit “forecasters” – in the sense of the one who knows how to make predictions. All too often, people assume that the prediction is a formal analysis, using models that only who is in the “Predictions Department” understands. This is not true. Each important decision we make in our lives, whether business or personal, is based on predictions. If an executive says “I’ll hire you because I think that he will be great for this job”, he made a prediction about how the candidate would perform the functions, and decided on the basis of the prediction he made. Should we introduce a new product or service? Should we enter a new market? Any important decision in the business world is based on the perception of how the future will unfold. In other words, it is based on predictions.
Why do people have difficulty in seeing this? And what are the consequences?
The failure to recognize that we make predictions all the time has serious consequences. People who do not understand that their predictions are the basis of the major decisions they make will not think too much about how to make predictions nor will take measures to improve. And so, inevitably, they do not evolve. After the awareness, the next step is to practice. Making predictions is a skill, and, like any skill, you need to practice it: do it several times, analyze your results and review what you are doing in the light of this feedback.
What can these feedbacks teach?
Phil Tetlock has invested much of his career in techniques that allow people to make real world predictions and obtain clear and precise feedback. Think about how you learn to ride a bike: you may learn some things you need through formal training, but, to obtain good performance, you need to ride, fall, rise, try again, improve. This is our approach about learning the art of making predictions. In our book, we provide some formal education – practical advice on how to do it – but, to get really good at predicting, it is necessary to learn from the experience.
How did you develop the theory? What did you find in your researches that helped design the foundations of superforecasting?
Tetlock is an eminent American psychologist and has been studying political and economic prediction since the 1980s. One of his first discoveries was how difficult it is to properly test the accuracy of many types of predictions. This is what led him to develop methods to circumvent the barriers. Then he showed that a large part of the experts’ predictions is bad. He even found an inverse correlation between fame and accuracy – so that the more famous the expert, less accurate his predictions were.
Why does this happen?
The essential problem is the belief that an expert is always right. Daily, we see very confident specialists on television saying what will happen, and we listen and take them seriously, because everyone – television producers, experts and the public – think they have a real forecasting ability. However, in almost all cases, their predictive models were not tested. Imagine if doctors prescribed medicines that were never scientifically tested for their safety and efficacy, but in which everyone trusted, because, after all, they are doctors. It is ridiculous. It is unscientific and primitive. And, however, it is how we treat the predictions given by “experts”. Tetlock was determined to scientifically tackle the problem, the same way medicine science does. After decades of work, he formulated a great research initiative, funded by the Office Director of National Intelligence – a government agency of the United States which oversees all North American intelligence agencies -, to apply his own work using real problems of global forecasting, as, for example, “How China’s economy will be in the fourth quarter?”. This research has produced a wealth of discoveries about what builds more accurate predictions, which is the basis of the book.
“What happened with Brazil is an opportunity to learn – or remember – that the reality is complex, changes and surprises are inevitable, comfortable assumptions are dangerous, and our ability to predict with accuracy is limited and dependent on hard work.” Dan Gardner
Which practices can executives use to improve the predictions in their businesses?
Testing is essential. You cannot improve what you don’t measure, so that leaders must implement methods for testing the predictions accuracy. Ideally, these methods should also provide the “training field” that people can use to make predictions, learn and improve. When it comes to people making predictions, the most important rule is that all rules are only guidelines. Reality is too complex for simple instructions. Having recognized this, we urge people to consult beforehand the ‘outside vision’. Look for different perspectives, people’s conclusions and different methods – and think about how they can be synthesized. Don’t think in terms of “this is going to happen or not”; think in terms of probability, from 1% to 99%, in shades so thin as it is possible to manage. And to avoid excessive information, “break” a problem in parts and seek only the information necessary for each of these parts. Finally, we recommend updating your prediction in light of new information, but also always in parts – to avoid an overreaction. And, above all, keep the intellectual humility. In this complex context, with our imperfect human minds, mistakes are inevitable. Think constantly about your thinking, questioning, doubting. Everyone really good at predictions – like [the investor] George Soros – are deeply introspective. It is not that they are perfect. Everyone makes mistakes. However, their metacognition (thinking about thinking) leads them to detect and fix more errors than their competitors, and this is everything in the game.
And how can the general approaches be adapted to deal with the changing circumstances such as the ones companies have faced in Brazil in recent years?
What happened with Brazil is an opportunity to learn – or remember – one of the most important lessons: reality is infinitely complex, changes and surprises are inevitable, comfortable assumptions are dangerous, and our ability to predict with accuracy is limited and dependent on hard and relentless work. Tetlock’s research proved that those who share this intellectually humble perspective might not be fortunetellers and gurus who predict everything, but are better predictors than most other people, which, in the business world, makes all the difference.
Is there a case of an entrepreneur who has improved his ability as a forecaster that you could share?
In addition to George Soros – whose success in the 60s up to the 80s was surprising -, I like to tell the story of John Maynard Keynes. Everyone knows Keynes as a legendary economist. However, Keynes was also a fantastic investor. In Great Britain, between the First and Second World Wars, Keynes invested money for himself, as well as for his family and friends in several institutions. He made several fortunes, which is impressive when one knows that Great Britain at that time was a terrible place to invest. Then, how did he do this? In many ways, Keynes was a perfect example of what we discussed in the book. He was intellectually humble and exceptionally willing to admit errors, learn and change his thinking. And there is no doubt that this is the big reason why he was a successful investor.
Do you think that political and economic crises can create real opportunities for companies that accept the challenge to face them? Or isn’t the belief that crises bring opportunities so real?
A basic feature of human psychology is the status quo bias – when something big and important happens, we expect more of the same, we expect repetition, the maintenance of that condition. Therefore, if there is a shocking terrorist attack, we expect more terrorist attacks. If there is a boom in the economy, we expect that the economy will continue to expand. If the stock market crashes, we think that it will remain in a spiral down. It’s a safe bet that many people in Brazil, taken by the status quo bias, assume that things will only get worse. It’s a classic mistake. The line will bend and rotate. It is always so. Those who recognize this error are in a position to make use of the excessive pessimism of those around them. As [the investor] Warren Buffett says: “The key is to be afraid when others are greedy and to be greedy when others are afraid”.
What is the best advice for people who want to be good in predicting future trends in the market?
I always recommend a basic understanding of psychology. Understand how others perceive, think and decide is a valuable tool for any business. Understand how yourself perceive, think and decide is essential. However, beyond that, I think it is more important to be intellectually and widely curious, than studying only one field in particular. The Tetlock’s research has shown that the best predictions are routinely fruit of intellectual curiosity. This is not a coincidence. Intellectual curiosity leads people to seek different ways of seeing reality – this is the heart of the accurate prediction. Do you want to hire a person like that? In job interviews, ask candidates to discuss three books that they have read recently that really encouraged them and led them to think. People who do not know how to respond to it are not the ones you want to hire. However, the person who responds to this question with enthusiasm can be a great predictor. Hire them.