"It's an unusual day if nothing unusual happens." This aphorism encapsulates a characteristic pattern of events called the Improbability Principle. Popularised by statistician Sir David Hand, emeritus professor at Imperial College London, it codifies the paradoxical idea that extremely improbable events happen frequently.
We all experience coincidences on an everyday basis: you are thinking of a long-lost friend just as she phones you; on a holiday in Rome you meet your next-door neighbour; you join a bridge club to find there is already a member with the same name as your own. It seems that highly improbable events are commonplace.
Prof Hand appeals to five underlying strands that together make up the improbability principle. First, the Law of Inevitability states an obvious idea: something must happen. In a lottery draw, it is highly unlikely that you will pick the winning numbers, but quite likely that some lucky person has bought a winning ticket. After all, many tickets are sold and some particular combination of numbers must win.
Next, the Law of Truly Large Numbers says that, given enough chances or choices, something outlandish is bound to occur. With regular lotteries throughout the globe, the same six numbers in two consecutive draws are to be expected somewhere, sooner or later. Indeed, just such a coincidence occurred in September 2009 in the Bulgarian lottery.
What about Ireland? With six numbers to choose from 47, there are about 11 million possible combinations. But after 4,000 draws the chances are better than even that the same six numbers will have come up twice. This is very surprising; the same counterintuitive result occurs when we consider the chances that two people in a group share the same birthday. A simple calculation shows that with just 23 people the chances of a common birthday are better than evens.
Then the Law of Selection shows that consideration of events that have already happened gives a distorted impression: by focusing on the occurrence of the same set of winning numbers in consecutive weeks, we tend to ignore the reality of the many thousands of cases in which no such coincidence has occurred.
What Hand calls the Law of the Probability Lever shows how minor changes can have a major impact on probability. Changing the assumptions about the underlying probability distribution can greatly affect the likelihood of extreme events. Recognising that events thought to be independent of each other are actually connected can change a “remarkable coincidence” to something to be expected. Finally, the Law of Near Enough deals with treating similar events as if they are identical. If you do not find an exact match, you look for a near match and disregard the discrepancy.
Prof Hand weaves these five strands into a general principle that gives him the title of his book The Improbability Principle, which bears the subtitle "Why Incredibly Unlikely Things Keep Happening". He applies this principle to gambling, extreme weather, medical tests and motor accidents and more widely to explain why surprising things happen regularly and how statistical analysis gives us insight into a wide range of real-world events.
Prof Hand will present the second Gossett Lecture, The dangers of not seeing what isn't there, at 6pm this evening at the Royal Irish Academy (tickets required).
Peter Lynch is emeritus professor at the school of mathematics and statistics, University College Dublin. He blogs at thatsmaths.com