Backward compatibility
A Socratic walk-through of backward compatibility — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why is the hardest part of changing a widely used interface the promise you made about the old one?
Changing a function inside your own program is a small act: edit it, fix the callers, run the tests. Changing the same function once it is published — as a library, an HTTP endpoint, a file format — can take years and consume more effort than writing the replacement.
The code is identical in both cases. What became expensive is not the change but something attached to it: a promise. It is worth working out precisely what that promise covers, since almost everyone finds it covers more than they agreed to.
Reasoning it through
REASONING #Begin with what publishing an interface actually does. Before publication, you own both sides: the definition and every use of it. After publication, you own one side, and an unknown number of people own the other. The edit that used to be atomic is now distributed across parties who cannot be edited on your schedule, or at all.
So the first cost is coordination, and it is not really about compatibility yet. Notice, though, that this cost does not scale with how big the change is. Renaming one field and rewriting the whole protocol both require every caller to act. That is why interfaces get frozen rather than carefully evolved: the fixed cost of asking everyone to move dominates the variable cost of the change itself.
Now the harder part. What exactly did you promise? The intuitive answer is: whatever the documentation says. Test that. Suppose your endpoint documents an unordered list, and it happens to come back sorted because of how the database returns rows. Callers write code that assumes sorted order — not maliciously, they simply observed it working. You change an index; the order changes; their software breaks.
Were they wrong? Formally, yes. Practically, it does not matter — their software is broken and yours did it. This is Hyrum's Law: with enough users, every observable behaviour of your system will be depended on by somebody, regardless of what you promised. Timing, ordering, error message wording, whether a field is ever null, how long a call takes, the exact bytes of an identifier: all of it becomes contract by observation.
That is the step that makes this hard rather than merely tedious. The promise is not a document you wrote; it is the set of behaviours anyone can see. You did not choose its boundaries and you cannot enumerate them. Which explains an otherwise strange piece of engineering advice: deliberately randomise the parts you have not promised — shuffle unordered results, vary an iteration order, add jitter — so that no one can come to rely on an accident. The only defence against implicit contracts is preventing them from forming.
Given all that, what can actually be done? Two families of answer, and they trade against each other.
Versioning keeps the old promise alive beside the new one. Semantic versioning encodes the claim — a major bump means "I have broken something" — but communicating the break does not reduce the work. Someone still maintains both, and versions accumulate faster than they retire.
Parallel change, sometimes called expand and contract, is the sequence that actually retires things. Add the new form beside the old; write both; move callers over while both work; only then remove the old. The compatibility window is the middle phase, and its length is set by how quickly your slowest consumer can move — which for an internal service might be a week and for a public format is measured in years. Everything about the migration follows from that number, and it is a property of your users rather than of your code.
There is an honest limit worth stating. Some promises cannot be kept: a security flaw in the design, a legal requirement, a guarantee that was always wrong. Then the question is not how to avoid breaking compatibility but how to break it deliberately — with a long notice, a loud signal, detectable failure rather than silent misbehaviour, and ideally a mechanical migration. A break that fails loudly on day one is far kinder than one that quietly changes an answer.
The analogy
THE ANALOGY #Think of a published railway timetable rather than a private schedule. Rearranging your own day is free. Once a timetable is printed, people have arranged childcare, jobs, and connecting journeys around the 07:42 — and the promise you made is not the printed page but the pattern people planned against, including the fact that it has run three minutes late for a decade and everyone allows for it.
a railway can eventually reach every passenger through the same channel it published in, whereas an API's dependants are often unknown and unreachable — so the deprecation notice reaches the ones who were already paying attention, and the breakage lands on the ones who were not.
Clarifying the model
THE MODEL #Three refinements.
First, distinguish compatibility's directions. Backward compatible usually means new code accepts old inputs or old callers; forward compatible means old code tolerates new inputs it does not understand — typically by ignoring unknown fields. Formats designed to skip unknown fields buy their future evolution cheaply, which is why the choice made at version one determines how expensive version five is.
Second, additive changes are not automatically safe. An optional field is usually fine; a new enum value breaks consumers that switch exhaustively, and a relaxed validation rule breaks anyone who relied on rejection. The test is not "did I add or remove" but "can any existing correct caller now observe something different".
Third, scope of blast radius should govern how much of this discipline you apply. Two services owned by one team can be changed together and should not pay the cost of a public API. The discipline is priced by how many parties can say no, so applying public-API ceremony inside a single deployable is a common and expensive mistake — as is discovering too late that an internal endpoint acquired outside users.
A picture of it
THE PICTURE #How to readRead left to right as four releases of the same interface. The stretch that matters is the middle, where both forms are written and maintained at once — that overlap is the whole cost of compatibility, and its length is set by the slowest consumer rather than by the difficulty of the change. Note the third item under v1: the ordering was never promised but was observed, so it must be handled in the same migration as the field itself. Only at v4, after usage has actually been measured rather than assumed, does removal become a decision instead of a gamble.
What became clearer
WHAT CLEARED #Publishing an interface converts an edit into a negotiation with people you cannot schedule. The promise that makes it hard is broader than the one you wrote down, because users bind to whatever they can observe — ordering, timing, error text, nullability — so the real contract is discovered rather than declared. That is why the practical work is not clever versioning but three unglamorous things: preventing accidental promises from forming, running both forms side by side long enough for the slowest consumer to move, and measuring actual usage before removal. And when a promise genuinely must be broken, the goal shifts from avoiding the break to making it loud, early, and mechanically fixable.
Where to go next
ONWARD #- How wire formats that skip unknown fields buy forward compatibility, and what they give up to get it.
- Why long term support branches are the usual outcome when the slowest consumer cannot be moved at all.
Key terms
TERMS #| Term | What it means |
|---|---|
| Hyrum's Law | the observation that with enough users, every observable behaviour of a system will be depended upon by somebody, whatever the documented contract says. |
| Parallel change (expand and contract) | adding the new form alongside the old, migrating callers, then removing the old form. |
| Semantic versioning | a version numbering scheme in which a major increment signals an incompatible change. |
| Forward compatibility | the ability of existing consumers to tolerate data or calls produced by a newer version, usually by ignoring what they do not recognise. |
Every term the collection defines is gathered in the glossary.