Service discovery

How do programs find each other when the addresses they live at change every few minutes?

For most of computing's history, one program found another by being told where it lived. An address in a configuration file, a hostname, a port. It worked because the answer was stable: the machine was in a rack, and it stayed there.

Now a service runs as a dozen containers that are created and destroyed continuously — scaled up at lunchtime, replaced on every deployment, rescheduled when a host is drained. Each new one gets a fresh address. The list of places your dependency lives is different from what it was ninety seconds ago. So how does a caller find it, and why can that not simply be another configuration file?

Begin by asking what the caller actually wants, because the wording matters more than it seems. It does not want an address. It wants a healthy instance of the thing called payments. The address is an implementation detail it has been forced to care about, and the whole discipline is about removing that.

So we need a level of indirection: a stable name resolved, at call time, to a currently valid location. Immediately three problems appear, and it is worth naming them separately because different systems solve them with different mechanisms.

How does the list get populated? Something must record that a new instance exists. Either the instance announces itself when it starts — self-registration — or a platform that already knows it created the instance registers it on the instance's behalf. The second is what Kubernetes does, and it is generally better, because a process that crashes cannot be relied upon to announce its own death.

How does a dead entry leave the list? This is the harder half, and the one that causes outages. A crashed instance cannot deregister itself, so the registry cannot wait to be told. It must actively distrust its own contents: registrations are leases that expire unless renewed, and entries are backed by health checks that remove an instance that stops answering. Notice that this makes the registry's data continuously provisional. There is no moment at which it is exactly right, only a bounded window in which it is stale.

How does the caller get the current answer without asking on every call? It caches — and now the staleness moves into the client. If a caller caches for sixty seconds, it may spend up to sixty seconds sending requests to an address that no longer exists. Shorten the cache and you get correctness at the cost of load on the registry and latency on every call. This is the trade at the heart of the subject, and it has no clean solution, only a choice of window.

What mechanisms implement this? DNS is the oldest: the name is a hostname, the list is a set of A records, the cache window is the TTL. Everything already speaks it — but TTLs are advisory and widely ignored, and resolver layers, connection pools and some runtimes have historically cached lookups far longer than the record permitted, which is exactly the failure mode above. Purpose-built registries — Consul, etcd, ZooKeeper, Eureka — add health checking, leases, and a watch mechanism so clients are pushed changes rather than polling. Kubernetes combines both: a Service gives a stable name, EndpointSlices hold the live instance list, and CoreDNS answers the name.

There is one more axis, and it explains why architectures that look different are solving the same problem. In client-side discovery, the caller fetches the list and chooses an instance itself; it gets full control of load balancing, at the cost of that logic living in every client and every language. In server-side discovery, the caller sends to a fixed load balancer or sidecar proxy that knows the list; the client stays simple and the intelligence sits in one place. Service meshes are essentially the second done thoroughly — a local proxy beside every instance, fed the current membership by a control plane.

And what happens when the registry itself is unavailable? It is now a dependency of everything, so this deserves an answer rather than an assumption. Good designs degrade rather than fail: clients keep serving from their last known list, since a stale list beats no list. That is a deliberate choice of availability over freshness — and it means a registry outage can hide until instances start changing.

A few things to get straight.

Service discovery is not load balancing, though they are almost always deployed together. Discovery answers where are the instances; balancing answers which one should this request go to. Keeping them separate clarifies why a mesh sidecar and a DNS record are not competitors — the sidecar does both, DNS does only the first and does it coarsely, since a client that takes the first returned record ignores the balance the record set was meant to imply.

The registry's data is never authoritative about the present, only about the recent past. Every design in this space is a choice of how stale you will be and what you do when you are wrong. Which is why a caller must handle connecting to a departed instance gracefully — a fast retry against a different address is part of the discovery mechanism, not a workaround for it. If a stale entry causes a user-visible error, discovery has been implemented halfway.

Finally, health checks decide the quality of the whole thing. A check that merely confirms the process is listening will keep an instance in rotation while it fails every real request; a check that exercises its critical dependency may pull the entire fleet out of rotation the moment that dependency wobbles. Neither extreme is safe, and choosing between them is a judgement rather than a best practice.

• Why DNS TTLs are so often ignored in practice, and what that breaks when instances churn quickly.

Every term the collection defines is gathered in the glossary.