Write amplification

Why does saving a few kilobytes to a solid-state drive sometimes force it to rewrite far more than that?

You save four kilobytes. The drive's own counters, tracking how much the host asked it to store and how much it actually programmed into flash, disagree — sometimes by a factor of two, sometimes by ten. Nothing was lost or duplicated for safety, and the file is the size you think it is.

So where did the extra writing come from? The tempting answer is overhead — metadata, checksums, alignment. That is wrong in an interesting way, because such overhead is roughly constant while this ratio moves: it depends on how full the drive is and how you write. What mechanism produces a cost that grows with occupancy?

Start with what flash physically permits, because the whole phenomenon is downstream of one asymmetry. Cells are programmed in units of a page — a few kilobytes, though the figure varies by generation and I am recalling rather than deriving it. But they cannot be individually returned to the blank state: erasing works only on a much larger unit, a block of many pages, typically measured in megabytes.

So what happens when the host overwrites four kilobytes that already exist in flash? The drive cannot clear that page in place, because clearing means erasing everything around it, so it does not try. It writes the new data to a fresh page elsewhere, updates an internal map from logical address to physical location, and marks the old page invalid. Fast, cheap, and nothing amplified yet.

The bill arrives later. Do that enough times and fresh pages run out, while the drive is littered with blocks part live and part invalid. To recover space it must erase a block, and to erase a block it must first move every still-valid page elsewhere. That copying is the extra writing: nobody asked for it, it carries no new information, and it consumes flash exactly as real writes do.

Now put a number on it. Suppose the block chosen for reclamation has a fraction u of its B pages still valid. The drive copies u times B pages out, then erases, yielding B free pages — but u times B are immediately consumed by the copies just made, so the net gain is one minus u, times B. Every page of host data therefore costs one page for itself plus u over one minus u pages of copying: a total of one over one minus u. At u of a half, amplification is two; at three quarters, four; at nine tenths, ten. That is not a slope but a cliff, which is why nearly all the damage happens in the last stretch of a full drive.

Two things set u. The first is spare capacity: every drive keeps some flash the host cannot address, and that reserve is what keeps blocks from being nearly all live at reclamation time. The second is the access pattern. Write a large file sequentially and its pages land in the same blocks and die together, so deleting it invalidates whole blocks at once, u falls near zero, and amplification approaches one. Scatter small writes randomly and every block ends up a mixture, so u stays high. Amplification is not a property of the drive but of a workload meeting an erase granularity.

A third factor concerns deletion, because the drive cannot see your filesystem. Deleting a file unlinks a name and marks blocks free — bookkeeping invisible below. Unless the filesystem tells the drive those addresses are now meaningless, the drive believes every page it ever wrote is still live, u climbs toward one, and it dutifully relocates data nobody wants. That is what the discard command sends.

How would we know this account is wrong? It makes two predictions, cheap to check against the drive's own host-write and physical-write counters. Leave part of the drive unallocated, and measured amplification should fall with no change to the workload. Convert a random small-write workload into an append-only sequential log and it should fall toward one, with no change to the hardware. Both are standard practice because both work. The refuting observation would be a workload whose amplification is unmoved by over-provisioning and unmoved by making writes sequential — which would say the extra traffic comes from something else, such as metadata, error-correction encoding, or the filesystem above.

Three refinements.

The first is why this matters. Flash cells tolerate a finite number of program and erase cycles, and that budget shrinks as manufacturers store more bits per cell — I will not quote figures, since published endurance ratings vary widely by generation and vendor. Amplification spends the budget faster than the host's write volume suggests, and consumes write bandwidth too, which is why a nearly full drive gets slower as well as older.

The second is the distinction from the garbage collection that pauses a running program. The mechanisms rhyme — both reclaim space held by dead objects by relocating live ones — but a runtime's collector spends latency, a drive's a finite physical endurance budget.

The third is that the real subject is not flash. The durable fact is a granularity mismatch: whenever the unit you can clear is much larger than the unit you write, the difference is paid in relocation — by log-structured filesystems in cleaning, by merge trees in compaction, by shingled disks when overlapping tracks are rewritten. Flash illustrates a constraint that will outlive it.

• How log-structured merge trees trade write amplification against read amplification.

Every term the collection defines is gathered in the glossary.