Chris Johns: Is our faith in scientific forecast models misplaced?

We live in the age of the model. Epidemiological variants dominate public policy in ways reminiscent of the heyday of economic models a decade or two ago. All-encompassing models of the economy are no longer very fashionable. Pragmatic and theoretical developments led to their demise.

First and foremost, they never really worked. Their forecasts are rubbish and they couldn’t answer questions like, “what happens if the oil price goes up or down a lot?” As well as rarely working in practice, economists realised there were deep issues of principle. The methodology was fundamentally flawed. It’s tempting for an economist to wonder if epidemiologists are repeating some of the mistakes made by economists years ago.

There are differences between the two disciplines. Medical and epidemiological advice is at the heart of policy. For very good reasons, I’ve never seen a government minister stand up and say, “policy is being driven entirely by the advice of the economists”. We have to hope that the scientists are on top of their stuff.

It's slightly disconcerting to see published epidemiological models – ones used to inform policy – disparaged for their "15,000 lines of buggy code, written for a flu pandemic". It's downright worrying to see an eminent scientist speculate about "epidemiological dark matter" as the reason why Germany has had a relatively good crisis. Dark matter is the way physicists fudge their models to account for the existence of the universe. If the epidemiologists have to adopt those kinds of modelling tricks we might have grounds for concern.

More than 160 scientists funded by Zuckerberg call for Facebook action on Trump

---

China lauds own coronavirus response as global death toll tops 400,000

---

Clubs play practice matches ahead of Premier League return

---

Iran records surge in Covid-19, raising fears of a second wave

---

Misplaced confidence?

With so much policy emphasis being placed on "he science", we need to be sure that this confidence is not misplaced. Models, economic or medical, should be able to explain the past as well as the future. An emerging trend, another slight concern, is that there appears to be much uncertainty about why some countries are doing so much better, or worse, than others. Japan, for instance, looks like it has managed the crisis well. But nobody seems to know why. There are plenty of theories about Germany's success: dark matter is just one of many competing explanations. Britain stands accused of "locking down too late" but that's a conclusion driven by modelling and may not stand up to scrutiny once all the data is in.

Economists still use models, but ones usually written on the back of a fag packet. That’s because small, easy to understand, models are easy to construct and use and generally do just as good a job as the big, complex variants. And that can often be not very good at all. Last Friday was a case in point.

The most important piece of economic data in the world, at least from a financial market point of view, is the monthly US jobs report. Economists, using models of one kind or another, had forecast a fall in American employment during May of about 7.5 million. The only difference between the hundreds of published forecasts was just how many millions of jobs were expected to be lost. Unemployment was forecast to rise to 20 per cent of the workforce. The data showed an increase of 2.5 million jobs and an unemployment rate of 13.5 per cent. That’s probably worth marking as the biggest economic forecasting error in history. Again, we have to hope that epidemiological modelling is centuries ahead of its economic counterpart.

Humility needed

If much more humility and honesty is needed when it comes to forecasting, that doesn’t mean we can’t construct outlines of what the future might look like. At the very least, planning requires some crystal-ball gazing. But rather than basing everything on a particular vision of the future, there should be much conditionality: different outcomes will require different responses. One encouraging aspect of Irish Covid-19 policymaking has been a willingness to embrace this kind of response. Much more of this is required, particularly honesty about what we know and what we don’t.

Economists have noticed how modern economies are practically weightless: GDP is comprised mostly of intangible services. It’s natural to speculate that the future also looks increasing contactless. We’ve become used to cashless payments but the avoidance of contact is inevitably becoming more widespread. Airports are turning their attention to how the whole check-in/security/boarding experience can be made with minimal human interaction. That’s an example of how trends in evidence pre Covid-19 will be accelerated in its aftermath. There will be lots more of this.

That’s not to say that everything will be changed permanently. An acquaintance of mine runs an off-licence. He’s long been used to making most of his annual profit during three days over Christmas. He tells me that the last three months have been like those three days. It’s natural for him to ask if any or all our new habits will prove temporary. It has become obvious to all of us just how expensive were some of our pre-crisis habits. Like working from home, I suspect the pendulum will swing back, but not to where it was before.

At least one thing has proved impervious to the virus: Brexit. We are seven months away from the hardest Brexit imaginable. A deal is still possible by the autumn but the odds are against it. The UK government is calling it the "Australian model". They don't mention that Australia doesn't have a trade deal with the EU. Models come in various guises.