Streaks in random sequences

Why does a genuinely random sequence look far streakier than people expect it to?

Ask a room of people to write down twenty coin flips from imagination, then flip a real coin twenty times, and hand both lists to a third party. They can usually pick out the fake — and the giveaway is that the fake is too tidy. It alternates too often and never contains a long run.

So our sense of what randomness looks like is systematically wrong, and wrong in a specific direction: we expect it to be smoother than it is. Where does that expectation come from, and what is the true shape it is failing to match?

Start by pinning down what "random" actually asserts. For a fair coin it says two things: each flip has probability one half, and each flip is independent of every other. Notice what the second condition does not say. It does not say the sequence self-corrects. It does not say heads and tails take turns. Independence means the coin has no memory — and a process with no memory has no mechanism for avoiding repetition.

Now compute rather than intuit. After any flip, what is the chance the next one matches it? One half — there is no other option, since the coin is fair and indifferent. So runs are built by repeated coin-flips of their own: a run continues with probability one half at each step, which makes run length geometric, with an average of two. Repetition is not a rare event in a random sequence; it is the modal event.

Then ask what happens over a longer stretch. Twenty flips contain seventeen overlapping windows of four consecutive positions, each with a one-in-sixteen chance of being all-same. Those windows overlap, so we cannot simply multiply — but the enumeration is easy enough to do exactly, and over all 2^20 sequences the probability that twenty fair flips contain a run of four or more is 0.77. A run of five or more turns up in 46 percent of them. The expected longest run is about 4.7. Would you have guessed that? Most people put the typical longest run at three.

So the arithmetic is unambiguous. The interesting question is what our intuition is computing instead, since it is clearly computing something systematic rather than merely being noisy.

Here is the best-supported answer, from Tversky and Kahneman's work on representativeness: we judge whether a sample is random by asking whether it resembles the process that produced it — and we apply that test locally, to every short stretch. A fair coin is fifty-fifty, so a random-looking stretch should be about fifty-fifty and should not favour either face for long. That test is correct about the whole sequence and wrong about its parts. The law of large numbers governs the long run; there is no corresponding law forcing every window of six to behave. Believing there is one has a name — the belief in the law of small numbers — and it predicts the exact error we see. When people generate sequences by hand, the alternation rate comes out around 0.6 rather than 0.5: they switch faces more often than a coin does, because switching is what "random" feels like.

Now the consequence that costs real money. If a genuinely random process produces clusters, then observing a cluster is weak evidence of anything. Yet a cluster demands explanation, because it does not look random to us. That is the clustering illusion, and it has a distinguished history: the German V-1 strikes on London in 1944 looked pointedly clustered, prompting theories about targeting and about which districts held spies, until R. D. Clarke divided the map into squares in 1946 and found the counts fit a Poisson distribution — exactly what scattered independent impacts produce.

One honesty check, because the most famous application of this idea has since been complicated. Gilovich, Vallone and Tversky argued in 1985 that the basketball "hot hand" was an illusion of this kind. In 2018 Miller and Sanjurjo showed their statistical method carried a subtle selection bias — conditioning on a streak having just occurred makes the next shot look worse than chance even in truly random data — and correcting for it, modest real hot-hand effects do appear in the data. The perceptual bias we have been describing is solid; the specific verdict on basketball is not.

The first misconception to clear is the gambler's fallacy, which is the same error wearing the opposite coat. Having seen five heads, people expect a tail as a correction; having seen a streak in sports, they expect it to continue. Both come from the assumption that short stretches must look representative — one enforces balance, the other reads the streak as a hidden cause. Independence rules out the first and is agnostic about the second.

The second is that none of this proves streaks are meaningless. It shows that the existence of a streak is not evidence, because the null hypothesis produces streaks abundantly. To claim a real effect you need more streaks, or longer ones, than chance supplies — which is a comparison against a computed baseline, not a judgement made by eye.

The third is practical, and it is why this shows up in engineering: because true randomness clusters, systems that look random to users often are not. Shuffle algorithms in music players have been deliberately de-randomised, spreading artists out, because listeners reported genuine shuffles as broken when the same artist came up twice in a row.

• The runs test, and how to check a sequence for too many or too few streaks quantitatively.
• Poisson clumping, and why disease-cluster investigations so often find nothing.
• How Miller and Sanjurjo's correction changed the wider literature that had adopted the original hot-hand method.

Every term the collection defines is gathered in the glossary.