Isolates, containers, and startup cost
A Socratic walk-through of isolates, containers, and startup cost — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why does making a sandbox weaker sometimes make the system safer to run at scale?
If you want two customers' code kept apart, the obvious instinct is to put the strongest possible wall between them. Separate virtual machines. Separate kernels. Perhaps separate hardware. Nobody was ever fired for over-isolating.
Yet the platforms that run untrusted code at the largest scale — hundreds of thousands of tenants on one server — have moved in the opposite direction, to a boundary that is weaker than a container, sometimes weaker than an operating-system process. That looks like a straightforward regression in safety. What could make it a considered choice rather than a corner cut?
Reasoning it through
REASONING #Let us be precise about what a boundary costs, because "overhead" hides the important part. A full virtual machine carries its own kernel, its own boot sequence, its own page tables and its own memory image; starting one takes seconds, and its resident cost is measured in hundreds of megabytes. A container drops the separate kernel — it is a normal process fenced off with namespaces and cgroups — so it starts in a fraction of a second and costs tens of megabytes. A microVM such as Firecracker deliberately strips a VM's device model down to almost nothing to get back toward that range; its designers report boot times on the order of a hundred and twenty-five milliseconds. And a V8 isolate is smaller again: not a machine, not a process, but a separate heap and execution context inside one process, with a start-up measured in single-digit milliseconds and a footprint of a few megabytes.
Now hold those numbers next to the workload. Suppose a request arrives, runs for two milliseconds, and finishes. What fraction of the total is the boundary? With a virtual machine, essentially all of it — so you must keep the machine warm between requests, which means paying for it while it sits idle. That is the hidden consequence people miss: an expensive boundary does not merely add latency, it forces you to keep things running, and keeping things running is what limits how many tenants fit on a box.
So here is the reframing. Ask not "how strong is this wall?" but "how many walls can I afford, and how long must each stand?" A cheap boundary lets you give every tenant their own, start it on demand, and destroy it the instant the request ends. An expensive one pushes you toward sharing, pooling, and long-lived instances.
And now the counter-intuitive part comes into view. Suppose ten thousand tenants must share a hundred virtual machines, because you can only afford a hundred. Every VM now holds a hundred tenants, and inside it they are separated by something much weaker than the VM boundary — process permissions, or nothing at all. The strong wall bought you a strong boundary between groups, and left the boundary within a group thin. Whereas the cheap boundary, applied per tenant, means no two tenants share anything above the process's memory allocator. So a weaker wall used everywhere can beat a stronger wall used sparingly. That is the actual argument, and it is about the achievable arrangement, not about the wall in isolation.
Honesty is required here, because the argument has a real limit. An isolate is a language-runtime boundary, enforced by the correctness of a JIT compiler and a memory manager, with no hardware assistance behind it — a bug in that compiler is a full escape. And because tenants share a process and an address space, side-channel attacks that infer another tenant's data from timing are a genuine concern; deployed mitigations include coarsening the clock the guest can observe, precisely because a precise timer is what such an attack needs. A hardware-virtualised boundary needs none of that. The trade is real.
Which is why the mature answer is not one model but a stack of them: isolates for enormous numbers of small, short, single-language workloads; containers where a tenant needs a filesystem and arbitrary binaries; microVMs where untrusted code needs a hardware boundary but the platform cannot afford full VM start-up. Each is chosen for a ratio of work done to boundary paid for.
The analogy
THE ANALOGY #Think of a library's reading rooms. A private locked room per reader is the strongest arrangement, and precisely because it is expensive the library only has a few — so most readers end up sharing one, seeing each other's papers across the table. Numbered desks with low partitions are a much weaker barrier, but the library can afford one per reader, so no two readers ever share a surface. Which reader is more exposed depends less on the height of the partition than on how many people are behind it with you.
a low partition still requires a human being to physically stand up and look over it, whereas a software boundary can be defeated by a single latent bug in the runtime that everyone shares at once — so the failure of a weak software wall is total and simultaneous in a way the reading room's never is.
Clarifying the model
THE MODEL #Three refinements.
First, the misconception that this is about performance. It is not really latency that drives the choice — it is density, the number of independent tenants one machine can hold. Startup time matters chiefly because a slow start forces you to keep instances warm, and warm instances occupy memory, and memory is what runs out first.
Second, weaker is not automatically better; it is better only when the cheapness is actually spent on more boundaries. Running many tenants in one isolate-based process while also sharing that process between them without per-tenant isolates would take the weakness and none of the benefit. The argument depends entirely on the cheap boundary being used per tenant, on demand.
Third, these boundaries constrain what can run, not just how safely. An isolate typically means one language runtime, no arbitrary system calls, no local filesystem, and strict execution limits. That is not incidental — it is part of why the boundary is cheap. You are choosing a smaller universe for the code, not only a wall around it.
A picture of it
THE PICTURE #How to readmove rightwards for a stronger separation and upwards for a costlier start. The isolate sits low and left — so weak that a runtime bug escapes it, but so cheap that every tenant can have their own and throw it away after one request. The full virtual machine sits high and right, which is why it must be kept warm and therefore shared. The microVM is the interesting point: deliberately placed low on a strong-boundary column, it exists to break the diagonal that the other three fall along.
What became clearer
WHAT CLEARED #The question was never how strong one wall is. It is how many walls the system can afford to erect, and how many tenants end up behind each one. A cheap boundary can be spent per tenant and per request, so nobody shares; an expensive one forces pooling, and that sharing may expose tenants to each other more than the weaker wall would. The cost of the reasoning is real: an isolate's protection rests on a runtime's correctness rather than on hardware, and a shared address space makes timing side channels an active concern rather than a footnote.
Where to go next
ONWARD #- How microVMs cut boot time so far without giving up hardware virtualisation.
- What timing side channels actually require, and why clock coarsening blunts them.
- Why a cheap boundary's restrictions are what make it cheap.
Key terms
TERMS #| Term | What it means |
|---|---|
| Isolate | a separate heap and execution context inside a single language-runtime process, used to keep tenants apart without a process or machine each. |
| Container | a normal operating-system process confined by namespaces and control groups, sharing the host kernel. |
| MicroVM | a hardware-virtualised guest with a minimal device model, designed for fast boot and low memory overhead. |
| Density | how many independent tenants one physical machine can hold at once. |
| Side channel | an attack that infers data from observable effects such as timing, rather than reading it directly. |
Every term the collection defines is gathered in the glossary.