There are, as Benjamin Disraeli famously remarked, lies, damned lies, and statistics, all confusing to the innocent or uninitiated. Some time ago in The New Scientist, Robert Matthews highlighted in this context - and I mean statistics - the dangers of placing too much trust in your daily weather forecast. He takes predictions of rain as an example.
Matthews assumes, first of all, that weather forecasters can predict rain with, say, 80 per cent accuracy. This is a conservative but not unreasonable estimate, which would lead the man in the street to expect that eight times out of 10 when rain is forecast, rain will actually fall. He then notes the comparative rarity in these islands, strange as it may seem, of rain falling at all. He gives it what he calls a "base rate" of about 0.1; this is taken to indicate that there is only a one-in-10 chance of rain falling in any particular hour, and therefore a nine-in-10 chance of rain not falling. To put it another way, out of 100 one-hour
shopping trips, climatological statistics suggest that it will rain on 10 of them, and remain dry on the other 90.
Based on this, and on the assumption of the 80 per cent accuracy of forecasts, it is possible to analyse the wetness of the 100 shopping trips as follows:
Rain Falls No Rain Falls Total
Forecast of Rain - 8 18 26
Forecast of No Rain - 2 72 74
Total - 10 90 100
We see from the bottom line, for example, that statistically it will rain on 10 of the trips and stay dry on the other 90. In the case of the column headed "rain falls", when it does rain the forecast will have predicted rain correctly 80 per cent of the time, so of the 10 "rainy" trips eight will have been forecast correctly, and the forecast will have predicted "no rain" on the remaining two trips. Similarly, of the 90 trips where no rain was experienced, the forecast will have predicted "no rain" on 72, while for the remaining 18 trips, the forecast will have been wrong.
What the table tells us then is that rain will have been forecast to occur on 26 of the 100 shopping trips, but of these forecasts only eight will have proved to be correct, a success rate of a mere 39 per cent. By contrast, of the 74 forecasts of "no rain", 72 will be correct, an enviable accuracy of 97 per cent. Both scores are far removed "right eight times out of 10" - or is there a flaw somewhere in the argument?