Documentation that points at its evidence

Why must generated documentation carry a pointer back to the line it came from?

A tool reads a codebase and produces a hundred pages describing what it does. The prose is fluent and mostly right. One sentence says premiums are recalculated whenever the sum insured changes.

Is that true? You cannot tell from the document. And notice what follows: because you cannot check any individual sentence cheaply, you are forced into a single global judgement about the whole document — trust it or don't. A hundred pages of mostly-right prose is therefore worth less than it looks, because the reader has no way to spend a little effort on the sentence that matters to them. Attaching a file and line to each claim looks like tidiness. What is it actually buying?

Start with the unit of failure. Without anchors, the document has one credibility rating for all hundred pages. With anchors, each sentence has its own, and a reader who cares about one rule can verify that rule in thirty seconds. The document stops being a single artefact you accept or reject and becomes a hundred separate claims you can audit independently. That change of granularity is the main effect, and everything else follows from it.

Consider what it does to review. Reading a hundred pages against two million lines of source is not a task anyone completes. Sampling forty anchored claims, checking each against the cited line, and finding three wrong is a task that takes an afternoon and yields an error rate — an actual number, with an actual confidence interval, attached to the document. Anchors convert an impossible review into a statistical one, which is the difference between a document being reviewed and being nodded at.

Now the effect over time, which may be larger. Code changes; the description does not. Unanchored prose decays silently, and nothing in the document signals which parts have gone stale. An anchored claim can be checked mechanically: has the cited file changed since the commit the document was generated from? Has the cited symbol moved or disappeared? None of that proves the claim is now wrong — a file can change without invalidating it — but it identifies the claims that need re-examination, and it does so without anyone reading anything. Staleness goes from invisible to enumerable, which is the same move as recording what you could not determine.

There is a third effect on the producer rather than the reader. When a wrong claim is anchored, you can see what the extractor was looking at when it went wrong, and the correction is usually to the rule that generated it rather than to the sentence. Fixing the rule fixes every instance. Without the anchor you are editing prose and the tool will regenerate the same error on the next run.

Here is where an honest caveat belongs. An anchor makes a claim checkable; it does not make it true. The cited line may exist, be current, and not support what the sentence says — particularly when the sentence is a summary spanning several places and the anchor points at only one of them. Multi-source claims are where anchoring is weakest, and where a document should either cite all its sources or say plainly that it is generalising.

What should an anchor consist of, given that line numbers are the least stable thing in a repository? A bare line number is broken by the next formatting pass. A commit identifier plus a path plus a symbol name plus a line degrades gracefully: the commit makes the reference exact and permanent, the path and symbol survive edits within the file, and the line number is a convenience that can be re-derived. The principle is that the reference should still locate the evidence after the code has moved, because it will.

And there is a category of claim that cannot be anchored at all — the ones about intent, about why a thing is done. Those are inferences over evidence rather than readings of it, and the correct treatment is not to fabricate an anchor but to mark them as a different kind of statement. A document that mixes verifiable readings with unmarked inferences invites its readers to trust both equally.

Three refinements.

First, anchoring is not the same as quoting. Including the cited lines inline is tempting and makes the document self-contained, but a copy is a snapshot that cannot go stale visibly — it will simply disagree with the current code while looking authoritative. A reference that resolves against the live repository stays honest; an embedded excerpt needs its own freshness marker.

Second, the anchor's audience includes machines. If every claim carries a structured reference, a build can re-resolve all of them and fail when a proportion no longer resolve. That turns documentation freshness into something continuous integration can enforce, rather than a periodic manual campaign that never quite happens.

Third, coverage cuts the other way and is worth reporting too. Anchors tell you which claims have evidence; they say nothing about which parts of the code have no claims at all. A document can be perfectly anchored and describe a third of the system, and only a coverage figure computed from the source side will show it.

• How to anchor a claim that summarises behaviour spread across many files.
• What proportion of unresolvable anchors should fail a build, and how that threshold is chosen.
• How to measure the coverage of a generated document from the source side rather than the document side.

Every term the collection defines is gathered in the glossary.