Autonomy as a dial
A Socratic walk-through of autonomy as a dial — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why is 'how much may it decide for itself' a setting rather than a yes-or-no question?
"Should the agent be allowed to act on its own?" sounds like a question with two answers. Say no and you have a suggestion engine that saves nobody any time. Say yes and you have handed a probabilistic system an unbounded licence.
Both answers are unsatisfying, and when both answers to a yes-or-no question are wrong it is usually the question that is malformed. So what is the hidden variable? What is actually being traded, such that "more" and "less" are meaningful rather than "on" and "off"?
Reasoning it through
REASONING #Try the obvious first framing: the dial measures trust in the agent. It has a certain reliability, so we allow it a proportional amount of rope. Test that against a case. An agent that drafts a reply and an agent that sends one are the same model with the same reliability — yet almost nobody is comfortable granting them the same latitude. So reliability alone does not determine the setting. Something about the action is doing work that the model's competence cannot account for.
What property of the action? Consider the two examples again. A draft can be discarded at no cost. A sent message cannot be recalled, and the damage is proportional to the audience. So the first axis is reversibility: what it costs to undo, and whether undoing is possible at all. Deleting a temporary file, cancelling an unsubmitted order, restoring from a snapshot — these are cheap. Publishing, paying, emailing, deleting the backup itself, changing a production credential are not.
Now hold reversibility fixed and vary something else. A reversible action taken once a week and the same reversible action taken four thousand times an hour are not the same risk, because review capacity is finite and errors compound before anyone looks. So the second axis is frequency, or more precisely the total exposure the action accumulates before a human next sees it.
Those two together explain most of what practitioners actually build. Where the action is cheap to undo and happens constantly, gate nothing and review afterwards by logging — this is where autonomy pays for itself. Where the action is hard to undo but rare, ask first, because a human interruption costs little at that rate. Where it is cheap and rare, let it run. And where an action is hard to undo and high-volume, an approval gate does not work: nobody can review four thousand irreversible decisions, so the honest response is to withhold that capability, cap it, or redesign it into something reversible — staging a change rather than applying it, for instance.
Which surfaces the failure mode worth naming. Add too many approval prompts and people stop reading them. This is the same dynamic as alarm fatigue in clinical monitoring, where the documented consequence of frequent alerts is that genuine ones get dismissed, and there is a corresponding literature on automation bias: humans supervising a mostly-correct automated system become worse at catching its errors, not better. So an approval gate is not free safety. Each prompt spends a scarce resource — attention — and spending it on the routine leaves none for the consequential. A dial with too many notches at the low end is quietly less safe than a coarser one.
Notice, too, that the dial is not a single setting for the agent. It is a setting per capability. The same agent might read anything, write to a scratch directory freely, open a pull request without asking, and never touch production without a named human. That is one agent at four positions, which is only coherent if the property being set belongs to the action rather than to the agent.
Two further inputs are worth folding in, honestly labelled as harder to operationalise. One is the agent's own confidence, which is useful as an escalation trigger but unreliable as a guarantee, since a model's stated certainty is imperfectly calibrated. The other is blast radius: how many things a single mistaken action can touch. Both are inputs to where the notch sits; neither replaces reversibility and frequency.
The analogy
THE ANALOGY #Think of signing authority in an organisation. A new employee is not given either total spending freedom or none. They are given a limit — spend up to this much on your own, above it get a manager, above that get finance, and here are things nobody at your level signs regardless of amount. The limit rises with demonstrated judgement, and the whole scheme exists so that scrutiny is spent where the consequences justify it.
an employee who is unsure will notice their own uncertainty and ask, escalating on judgement the policy never anticipated, whereas an agent applies exactly the thresholds you wrote and will proceed confidently through a case you failed to imagine — so the boundaries have to be drawn as enforced limits, not as guidance.
Clarifying the model
THE MODEL #Three refinements.
First, the gate must be enforced somewhere the agent cannot route around. An instruction in a prompt saying "always ask before deleting" is a preference, not a control; the control lives in the tool layer that refuses to execute without an approval token.
Second, the point of the dial is not to keep a human in the loop everywhere, it is to place the human where their judgement is decisive. Reviewing before an irreversible act is decisive. Reviewing every reversible step is theatre that consumes the attention the first review needed.
Third, undo is a design lever rather than a fixed fact. If you want more autonomy, do not argue that the agent is trustworthy — make its actions cheaper to reverse. Staging, dry runs, soft deletes, generated pull requests instead of direct commits, and reversible provisioning all move an action leftward on the reversibility axis, and the notch can then move with it. That is the most productive thing this framing tells you: the dial is not the only adjustable part.
A picture of it
THE PICTURE #How to readPlace each capability, not each agent. Left of centre is where autonomy is nearly free because mistakes cost a retry. The lower right is where a human approval gate belongs, and it works there only because those actions are rare enough for someone to actually read the prompt. The upper right has no safe setting — an approval gate collapses under that volume — so the answer is to make the action reversible or withhold it.
What became clearer
WHAT CLEARED #Autonomy is not a property of the agent at all; it is a property of each action it may take, set by how expensive the mistake is to undo and how often the action recurs. That reframing is what makes it a dial — and it makes the most useful safety work not "supervise more" but "make the consequential actions reversible", because that is what lets the dial move without the risk moving with it.
Where to go next
ONWARD #- Calibration: whether a model's stated confidence can ever be a trustworthy escalation trigger.
- Designing approval prompts that resist rubber-stamping, and how to measure whether they are being read.
- Capability-based tool permissions as the enforcement layer beneath the policy.
Key terms
TERMS #| Term | What it means |
|---|---|
| Graduated autonomy | granting an automated system different degrees of independent action per capability rather than one blanket setting. |
| Approval gate | a control that blocks an action until a human authorises it, enforced in the tool layer rather than by instruction. |
| Reversibility | how cheaply, and whether at all, an action's effects can be undone. |
| Blast radius | the number of systems, records, or people a single erroneous action can affect. |
| Automation bias | the tendency of human supervisors to accept an automated system's output and miss its errors. |
Every term the collection defines is gathered in the glossary.