The noisy neighbour

Why does every tenant using no more than their fair share still degrade the service for all of them?

The usual story has a villain. One tenant runs a runaway job, saturates the cluster, and everyone else's requests slow to a crawl. Fix: cap that tenant, restore order.

But now suppose there is no villain. A hundred tenants share a system sized for a hundred, and each one uses exactly its entitled one-hundredth. By the accounting nobody is doing anything wrong — and the service is still bad for all of them. That version is more interesting, because it means the problem is not misbehaviour. Where could the degradation possibly be coming from?

Look hard at what "one-hundredth of capacity" means. It is an average, taken over some window. Nobody consumes at a constant rate; demand arrives as a process with peaks. A tenant averaging one unit may peak at twenty, and that is completely normal.

If those peaks were scattered randomly in time, the arrangement would work — the peaks would rarely coincide, and the shared pool would absorb them. That is statistical multiplexing, and it is the entire economic basis of selling shared capacity: you sell more peak than you own, because the peaks are assumed independent.

So ask the obvious question: are they? Tenants share working hours, month-ends, and the same holiday shopping season. Their schedulers fire on the top of the hour, because that is what people write. They react to the same upstream events. Correlation is the norm, not the exception, and correlation is exactly what statistical multiplexing assumed away. Nothing has to be abusive for every peak to land in the same minute.

Now the deeper mechanism, and this is the one that answers the question as posed. Suppose demand were perfectly smooth and everyone used exactly their share. Total utilisation is then 100 percent — and a queueing system run at 100 percent utilisation does not run at full efficiency, it collapses. Waiting time does not rise in proportion to load; in the simplest model of a single queue, it scales roughly as one over the unused fraction. At half utilisation the queue is negligible. At 90 percent, waiting is about ten times the service time. At 95 percent, twenty. As utilisation approaches one, the queue grows without bound.

Sit with the consequence. "Fair share" defined as capacity divided by tenants is, by construction, the operating point where the system is at its worst. The pain is not caused by anyone exceeding their share; it is caused by everyone reaching it. This is why shared systems must run with headroom, and why the honest entitlement is not capacity over N but something meaningfully below it.

There is a third mechanism worth having, because it explains cases the first two do not. The resource you meter is usually not the resource that is contended. You count requests; the contention is in a connection pool, a lock, a page cache, memory bandwidth, or a shared disk. A tenant well inside its request quota can issue one query that evicts everybody's cached working set, or hold a lock while it does something slow. And a single expensive request occupying a worker blocks whatever queued behind it, regardless of whose it is — head-of-line blocking, which converts one tenant's slow work into everyone's latency.

Finally, the feedback that turns a bad minute into an outage. When responses slow, clients time out and retry. Retries are new load, arriving precisely when there is least room for it, and the system can settle into a state where it is doing mostly retried work and completing almost nothing — a metastable failure that persists after the original trigger is gone.

What helps follows from the mechanisms rather than from stricter accounting. Limit concurrency per tenant, not only request rate, since concurrency is what maps to contended resources. Queue per tenant and serve those queues fairly, so one backlog cannot occupy the head of the line. Shed load at admission rather than admitting everything and degrading. Keep headroom deliberately. And bound how many tenants share fate at all — cell-based deployments, or shuffle sharding, which gives each tenant a random subset of workers so any two overlap only partially.

Three refinements.

First, quotas and isolation are not the same thing, and confusing them is the most common design error here. A quota is accounting: it tells you afterwards who used what, and it caps a total. Isolation is mechanical: a separate queue, a concurrency limit, a separate thread pool, a separate cell. Only isolation bounds the interference, which is why a system can be perfectly fair on the invoice and terrible to use.

Second, "fair" needs a definition before it can be engineered. Equal shares of what — requests, CPU seconds, bytes, concurrent connections? Tenants differ enormously in shape, and a rule fair in one currency is unfair in another. Weighted fair queueing exists precisely because the useful notion is usually proportional shares of the contended resource, not equal shares of a proxy for it.

Third, a misconception: that the answer is more capacity. Adding capacity moves the operating point down the curve and buys real relief — and then demand grows into it, because the entitlement was defined as a share of capacity. Without headroom policy and isolation, a larger system reaches the same bad point with more tenants on it.

• Shuffle sharding, and how random subsets of workers bound the blast radius of any one tenant.
• Metastable failure, and why retry policy matters more than capacity during an incident.
• Priority and load shedding: choosing what to drop before the queue chooses for you.

Every term the collection defines is gathered in the glossary.