Chain-of-thought
A Socratic walk-through of chain-of-thought — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why does asking a model to work step by step improve arithmetic it could already do?
A model gets a multi-step word problem wrong. You add five words — "work through it step by step" — and it gets the same problem right. Nothing was taught. No new fact entered. The arithmetic involved was arithmetic it demonstrably knows, because it does each individual step correctly once the steps exist.
So what did the five words buy? Something happened that was not the acquisition of knowledge, and finding out what it was tells you more about these systems than the trick itself.
Reasoning it through
REASONING #Begin with something easy to overlook: how much computation happens between reading a question and emitting the next token. In a transformer, the answer is a fixed amount. The input passes through a fixed number of layers, each doing a fixed amount of work, and a token comes out. Whether the question is "what is 2 plus 2" or a four-step inventory problem, the machinery available before that first output token is identical.
Now ask the uncomfortable question. If a problem genuinely requires more sequential work than fits in that fixed depth, what can the model do? It cannot think longer before answering — there is no "longer". It answers with whatever the fixed pass produced, which for a hard problem is closer to a guess shaped by familiarity than a computation.
Unless — and here is the hinge — it can write something down. Each token it emits is appended to the context and read back as input for the next pass. So a token of working is not just a report of thinking; it is a place to put a partial result so that the next fixed-depth pass can start from there rather than from the beginning. Twenty tokens of working is twenty more passes through the whole network, each one able to build on the last.
Notice what that reframes. "Step by step" is not a request for a certain style of answer. It is a request for more serial computation, obtained the only way the architecture allows: by turning the output stream into scratch paper. The visible reasoning is the mechanism, not a description of one. This is not merely a nice story either — there is theoretical work (Merrill and Sabharwal, and Feng and colleagues, among others) showing that intermediate tokens genuinely expand what a fixed-depth transformer can compute, which is a stronger claim than "it helps in practice".
There is a second effect worth separating out. Writing the steps also constrains the answer. Once "12 times 3 is 36" sits in the context, the final number has to be consistent with it — the model is now continuing a document that contains its own commitments. A wrong answer would be a less likely continuation than it was before. So the improvement has two sources: more computation, and a written trail that the rest of the generation must stay coherent with.
Then the finding that gives all this its weight. Wei and colleagues reported in 2022 that chain-of-thought prompting does not help small models much, and can hurt — the benefit appears at larger scale. If the mechanism were just "a good format for answers", you would expect a small model to benefit too. That it does not suggests the extra passes only pay off when each pass is competent enough for its output to be worth building on.
And a caveat I will not soften. The written steps are not a readout of an internal process. Turpin and colleagues showed in 2023 that a model's stated reasoning can be systematically unfaithful: bias the prompt so the model reaches a particular answer, and it will produce a fluent, plausible chain of steps that never mentions the actual cause. The chain helped it compute; that does not make it an honest account of what determined the answer.
The analogy
THE ANALOGY #Think of doing long multiplication in your head versus on paper. The sum is not harder on paper; you are not smarter with a pencil. What paper gives you is a place to park the partial products so that holding them is no longer a job your working memory has to do while also doing the next step. Take the paper away and the same person, with the same arithmetic, starts failing at four digits.
your scratch paper is private and can be messy or wrong without affecting what you say, whereas a model's working is its output stream — the steps and the answer are the same act of writing, which is exactly why a confident wrong step drags the final answer along with it.
Clarifying the model
THE MODEL #Three refinements.
First, the correction to the natural reading: nothing about "step by step" tells the model to be careful. There is no diligence dial. The phrase changes the shape of the text being generated, and the change in shape is what supplies the computation. It is a structural intervention wearing the costume of an instruction.
Second, this explains why the trick fails on problems that were never compute-bound. If the model does not know a fact, more passes over its own writing will not produce it — it will produce fluent working toward a wrong figure. Chain-of-thought converts problems that are hard because they are long into problems the model can handle. It does nothing for problems that are hard because the knowledge is absent, and the confident working it generates in that case is actively misleading.
Third, on cost, since it is the practical consequence. Every step token is generated and then re-read; a long chain is genuinely more expensive and slower than a bare answer. That is not a defect to be optimised away but the price of the extra computation, and it is the direct ancestor of models that are trained to spend a budget of hidden thinking tokens before replying.
A picture of it
THE PICTURE #How to readStart at the input at the top and follow the arrows down into the store, which holds the question plus every word written so far. The loop is the whole point: each line of working is appended to that store and read back, so the network makes another full pass with the partial result already in hand. The dashed branch is the honest hazard — when the missing ingredient is a fact rather than a step, the same loop still runs and produces confident working toward the wrong number.
What became clearer
WHAT CLEARED #Asking for the steps does not make a model more careful; it gives a fixed-depth machine a way to spend more computation on a hard problem, by writing partial results where its next pass can read them. The visible reasoning is the scratch paper, not a window into the mind — which is why it reliably improves long arithmetic and just as reliably fails to flag the moment it is confabulating.
Where to go next
ONWARD #- How trained reasoning models internalise this, spending a budget of thinking tokens the user never sees.
- Why sampling several chains and taking the majority answer beats one chain, and what that says about where the errors are.
- What it would take to make stated reasoning faithful, and why that is a distinct research problem from making it effective.
Key terms
TERMS #| Term | What it means |
|---|---|
| Chain-of-thought prompting | eliciting intermediate reasoning steps in the output before the final answer. |
| Zero-shot chain-of-thought | getting the same effect from an instruction such as "let us think step by step", with no worked examples. |
| Serial computation | work that must happen in order, which a fixed-depth network can only extend by emitting and re-reading tokens. |
| Faithfulness | whether stated reasoning reflects the process that actually produced the answer. |
Every term the collection defines is gathered in the glossary.