Prompt chaining
A Socratic walk-through of prompt chaining — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why does splitting one hard instruction into four easy ones produce a better result than the single instruction?
One instruction: read these twelve support tickets, work out the recurring themes, decide which are urgent, and draft a one-page summary for the executive team in their house style.
Four instructions, run in sequence: extract the complaint from each ticket; cluster the complaints into themes; rank the themes by urgency; write the summary from the ranked themes. Same model, same underlying work, and the four-step version reliably comes out better.
That is odd if you think of the model as a worker who understands the job. Nobody splits a task into four emails to make a colleague perform better. So what is it about a single complex instruction that degrades the result?
Reasoning it through
REASONING #Start by counting what the single instruction actually asks for. It contains at least four objectives, and each carries its own constraints: extraction should be faithful, clustering should be neither too fine nor too coarse, ranking needs a criterion for urgency, drafting needs length, audience and tone. That is easily a dozen simultaneous requirements, all of which must be satisfied by one pass of generation.
Now, what does a model do when requirements compete? It produces one stream of text, left to right, and every token is a compromise across everything it is currently trying to satisfy. Instruction-following benchmarks show the pattern plainly: compliance degrades as the number of simultaneous constraints in a single instruction rises, and the constraints that get dropped are usually the ones mentioned earliest or phrased most briefly. The model is not ignoring you. It is trading one requirement against another with no way to tell you it did so.
There is a second thing happening, and it is easy to miss. In the single-instruction version, where does the intermediate work live? The themes have to be identified somewhere before they can be ranked — but nothing in the output is required to contain them. They exist, if at all, implicitly, and any error in them is invisible. Split the task and the themes become an actual artefact: text you can read, count, and object to.
That artefact does more than aid inspection. Ask what the fourth step receives in each version. In the single instruction, the drafting happens while the model is still holding twelve raw tickets in context, along with every other objective. In the chain, the drafting step receives a short ranked list and one instruction about tone — a far simpler problem, with the irrelevant material gone. Each step in the chain works on a cleaner input than the monolith ever gets.
And that unlocks the practical payoff. Once the steps are separate, you can put things between them: a check that clustering produced between three and seven themes, a deterministic sort instead of asking a model to rank, a retry of just the clustering step when it comes out wrong. In the single-instruction version there is nowhere to stand — the only remedy for a bad output is to run the whole thing again and hope.
Now the honest counterweight, because chaining is sold too hard. Splitting is not free, and there are three real costs.
The first is arithmetic. If each step succeeds nine times in ten, four steps in sequence succeed a little over sixty-five per cent of the time — and unlike the monolith, a failure early in the chain is not merely a bad output, it is a bad input that every later step will treat as fact. Compounding cuts both ways: it is why gates between steps matter, and why a chain without them can be worse than the single call it replaced.
The second is lost context. Information you discard at step one cannot be recovered at step four. If the extraction step drops the fact that three of the complaints came from the same customer, no downstream step can weigh it, whereas a monolithic pass at least had it in view. Decomposition is a commitment about what matters, made before you know what will matter.
The third is cost and latency: four sequential calls instead of one, each re-sending its own context.
So the real question is never "should I chain?" but "which seams are worth cutting?" Split where the objectives genuinely differ, where an intermediate artefact is worth inspecting, and where a cheap check can be applied. Do not split a task into steps that must all hold the same context to be done at all.
The analogy
THE ANALOGY #Think of a kitchen preparing one dish. A single cook doing everything at once — chopping while the pan heats while the sauce reduces while plating is meant to be happening — makes exactly the errors you would predict: something burns while attention is elsewhere. Split the work into stations, each with one job and a clean handoff, and quality rises even though the skill in the room is unchanged.
a kitchen's stations run in parallel and the cooks can shout to each other, whereas a prompt chain is strictly sequential and each step sees only what the previous one wrote down — so a chain has no equivalent of the cook who notices the fish is off and warns the plating station.
Clarifying the model
THE MODEL #Three refinements.
The first corrects the most common conflation. Prompt chaining is not chain-of-thought. Chain-of-thought asks for reasoning inside a single generation, where the intermediate steps are text the model produces for itself; chaining uses several separate calls, where the intermediate result leaves the model entirely and can be validated, transformed, or replaced by ordinary code before the next call. The second gives you control points that the first does not.
The second is about what improves. The model is not more capable in a chain. Each individual step is doing the same kind of work it always did — the gain comes from every step facing fewer competing objectives and a cleaner input, and from the errors becoming visible in between. This is why the technique tends to help most on tasks that were failing through requirement-dropping, and least on tasks that were failing because the model simply does not know something.
The third is where the effort goes. Once the steps are separate, the design work moves to the seams: what step two receives, in what shape, and what happens when it is malformed. A chain with vague handoffs reproduces the ambiguity it was meant to remove.
A picture of it
THE PICTURE #How to readFollow the solid path downward: data in, four sub-processes that each do one job, finished page out. The diamond is the reason chaining pays — a check that can only exist because there is a seam to put it in, with a "no" edge that re-runs one step rather than the whole task. The dashed path is the cost side: whatever the first step failed to carry forward is gone by the time the drafting step needs it.
What became clearer
WHAT CLEARED #A single hard instruction fails not because the model cannot do any one part of it, but because every token has to satisfy all the parts at once, and something quietly loses. Chaining converts implicit trade-offs into explicit artefacts — inspectable, checkable, fixable one at a time. The price is more things that can go wrong, which is why the gates between the steps are the technique, not a refinement of it.
Where to go next
ONWARD #- Where to place validation between steps, and what to do when a step keeps failing its gate.
- Routing and parallel branches: chains that are not straight lines.
- When to replace a model step with deterministic code.
Key terms
TERMS #| Term | What it means |
|---|---|
| Prompt chaining | decomposing a task into several sequential model calls, where each call's output becomes the next call's input. |
| Chain-of-thought | eliciting intermediate reasoning inside a single generation; distinct from chaining, which spans separate calls. |
| Gate | a check between chain steps that validates an intermediate result before it is allowed to propagate. |
| Error propagation | the way a mistake made early in a chain is treated as established fact by every later step. |
Every term the collection defines is gathered in the glossary.