Exactly once is a property of the pipeline

Why can no single component promise that a message is handled exactly once?

Every broker's documentation offers a ladder of guarantees: at most once, at least once, and — on the top rung, sometimes with an asterisk — exactly once. The asterisk is doing an enormous amount of work.

The wish is entirely reasonable. Nobody wants a customer charged twice, and nobody wants an order silently dropped. What is worth examining is the belief that this is a setting: that some component, correctly configured, can be made to deliver each message once and only once. The reasoning below is not that exactly-once is unachievable — it is achieved in production every day — but that it is never a property any one component can have.

Start with the smallest possible interaction. A sender transmits a message and waits for an acknowledgement. The acknowledgement does not arrive. What has happened?

Three possibilities, and the sender cannot distinguish them: the message was lost in transit; the message arrived and was processed but the acknowledgement was lost; the message arrived and is still being processed. In the first case the sender must resend or the message is gone. In the second, resending duplicates it. The sender has to choose without knowing which world it is in.

That is the entire problem, and it does not go away with better engineering. It is the two generals problem, and it is provably unsolvable with a finite exchange of messages over a channel that may lose them — no protocol lets two parties reach certainty about each other's state. Adding a third acknowledgement just moves the doubt one hop along.

So the choice is forced. Resend on doubt, and you get at least once with duplicates. Do not resend, and you get at most once with losses. There is no third option available at the level of delivery, which is why "exactly-once delivery" is the phrase to be suspicious of.

But now notice a shift that changes everything. What the business actually requires is not that the message arrive once — it is that the customer be charged once. That is a statement about effects, not about deliveries. And effects can be made to survive duplicate delivery, if the receiver is built for it. Given a stable identifier on the message, the receiver can record which identifiers it has already applied and ignore repeats. Delivery stays at-least-once; the effect happens once.

Which raises the question that decides whether this actually works. When the receiver applies the effect and records the identifier, are those one operation or two? If two, the whole scheme has a gap: crash between them and the effect is applied with no record, so the redelivery applies it again. The dedup record and the side effect must commit atomically, in the same transaction, or the guarantee is only a narrowing of the window.

Follow that constraint and it explains what the strong systems actually do. Kafka's exactly-once semantics work because a consume-transform-produce cycle can commit the produced records and the consumed offset in one transaction within Kafka — the effect and the bookkeeping share a commit. The moment the effect leaves that boundary, sending an email, calling a payment API, writing to a store that is not in the transaction, the guarantee stops applying, because there is no longer a single atomic unit containing both. This is why the transactional outbox pattern exists: it drags the outbound message back inside the database transaction that made the change, so the two cannot disagree.

Notice the shape of that argument. It is the end-to-end argument, stated by Saltzer, Reed and Clark in 1984: a function that requires knowledge held only at the endpoints cannot be completely provided by the communication system beneath them. Lower layers can make it cheaper — fewer duplicates, fewer retries — but only the endpoint knows what "the same operation" means. Two payment requests with the same amount might be one duplicate or two genuine purchases, and no broker can tell.

So the honest formulation is this. At-least-once delivery, plus an identifier chosen by the sender, plus a receiver whose deduplication commits atomically with its effect, produces exactly-once processing. Every one of those three parts lives in a different place. Take any one away and the property vanishes, which is precisely why no single component can promise it.

Three refinements.

First, "exactly-once delivery" and "exactly-once processing" are different claims, and the marketing usually asserts the first while the engineering delivers the second. A vendor claim is worth reading closely for the boundary it applies within: almost always, it holds inside that vendor's own system and lapses at the edge.

Second, idempotence is a property of the operation, not a switch. set balance to 100 is naturally idempotent; add 10 to balance is not, and can only be made so by attaching an identifier and remembering it. Rewriting an operation into its idempotent form is often cheaper than building a deduplication store, and worth trying first.

Third, deduplication state cannot be kept forever, so real systems keep a window — a few hours, a few days. That is a genuine limit rather than an implementation detail: a duplicate arriving after the window will be applied again. Choosing the window is choosing how long a delayed retry may lurk before it becomes a correctness problem.

• The transactional outbox, and why it exists precisely at the boundary where atomicity is lost.
• Idempotency keys in public HTTP APIs, and how long providers retain them.
• What Kafka's exactly-once semantics do and do not cover once an effect leaves the cluster.

Every term the collection defines is gathered in the glossary.