Inspection paradox

Why is the bus you board almost always more crowded than the average bus on the route?

The timetable says a bus every ten minutes, and the operator's own figures say the average bus carries twenty-two people. Yet the bus you actually get on is packed, and the wait beforehand felt like more than five minutes. It is tempting to conclude that the operator is lying, or that you are unlucky, or that you only remember the bad days.

But suppose the operator's number is exactly right, and suppose your memory is perfectly accurate. Can both be true at once? They can — and the reason has nothing to do with buses. It is that the operator and you are not sampling the same thing.

Ask what an average bus means to the operator. She lines up every departure of the day and takes the mean passenger count; each bus counts once, whether it left empty or wedged.

Now ask what it means to you. You are one passenger. If we polled every passenger on the route and asked "how crowded was your bus?", each bus enters that poll once per passenger it carried — a bus with sixty people contributes sixty answers, one with four contributes four. So the passengers' average is weighted by crowding, the very quantity being measured.

Do you see that the bias is not an error but a definition? Nobody miscounted. Two different populations were averaged: buses, and passenger-experiences of buses. Wherever the thing you sample by is also the thing you are measuring, the sample is size-biased, and the biased mean is always the larger of the two — strictly larger unless every bus carried exactly the same load.

Now push one step further back and ask why the loads differ so much in the first place. Buses do not arrive on a perfect ten-minute grid; the gaps wander. Consider one hour with gaps of 4, 3, 21, 5, 22 and 5 minutes. Six buses in sixty minutes — the timetable's ten-minute average, honoured exactly. If you arrive at a moment you did not choose, which gap do you land in? Not one of six equally; you land in a gap in proportion to its length. Forty-three of those sixty minutes lie inside the two long gaps. So over seventy per cent of arrivals fall into a gap more than twice the advertised headway.

That gives the crowding its cause. The bus that ends a twenty-two-minute gap has been sweeping up arrivals for twenty-two minutes; the one ending a three-minute gap has almost nobody. The long gap produces the crowded bus, and the long gap is also the one you are most likely to be standing in.

Both effects are real and they compound. The general result: if headways have mean m and standard deviation s, the gap containing a randomly chosen moment has mean length m + s²/m, and your expected wait is half of that. For our hour, the gap you land in averages about 16.7 minutes rather than 10, and your expected wait is about 8.3 minutes rather than 5. Only if the buses were perfectly regular — s = 0 — would the naive answer be right.

The most useful thing to notice is how many familiar complaints are this one fact wearing different clothes.

Class sizes: a university can truthfully advertise an average class of thirty while its students truthfully report an average of ninety, because students are sampled in proportion to class size — the same m + s²/m, with the same equality-only-if-identical condition. Hospital stays: a ward audited on one afternoon over-represents long-stay patients, which is where the phenomenon got the name inspection paradox — an inspector arriving at a random moment inspects the long intervals.

And your friends: Feld's 1991 observation that, on average, your friends have more friends than you do. A popular person appears on many friend lists and a solitary one on few, so sampling "a friend of someone" is sampling people in proportion to their number of friends. The excess is exactly s²/m in degrees. Nothing about self-esteem; the same weighting, again.

Two honest qualifications. First, the arithmetic above assumes your arrival time is independent of the timetable — if you consult a schedule and time your walk, you deliberately break the sampling and much of the effect goes with it. Second, the bunching feedback is a genuine causal process on top of the sampling bias, not the same thing said twice: it explains why s is large, while the inspection paradox explains what a large s does to you. Operators fight the first with holding controls at timing points; nothing fights the second, because it is not a fault.

• Why an exponential-headway route makes your expected wait equal to the full mean headway, not half of it.
• How holding buses at timing points trades a longer scheduled trip for a smaller variance, and when riders come out ahead.

Every term the collection defines is gathered in the glossary.