Newton's method
A Socratic walk-through of Newton's method — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why does the same repeated correction converge instantly from one starting guess and fly apart from another?
Ask a calculator for the square root of two and it will find it in three or four passes of a rule you could do on paper: guess, divide, average, repeat. The digits do not creep in one at a time — they roughly double each pass. Yet the very same rule, pointed at a slightly different equation from a slightly different starting point, can wander off to nowhere, or bounce between two values forever, and never get closer at all.
That is odd, because nothing about the rule changed. So the interesting question is not "how does Newton's method work" but this: if the procedure is fixed, what is it about the starting point that decides between instant success and permanent failure?
Reasoning it through
REASONING #Begin with what the step actually does. You want the place where a curve crosses zero. You do not know it, but you do know where you are standing, and you know the slope there. So pretend for a moment that the curve is its tangent line, slide down that line to where it hits zero, and stand there instead. In symbols, the next guess is the current one minus the height divided by the slope: x - f(x)/f'(x).
Now here is the move worth dwelling on. The rule is not a formula for the answer; it is a feedback loop. Its output becomes its own input, forever. So stop asking what one step gives and ask what the loop does to an error. Suppose you are off by a small amount e. Expand the function about the true root: the linear part is exactly what the tangent cancels, so what survives is the curvature term. Work it through and the next error is about (f''/2f') times e squared.
Squared. Do you see what that buys? If your error is 0.01, the next is roughly 0.0001, then 0.00000001. The loop's gain on the error is not a constant fraction — it is proportional to the error itself, so the closer you are, the harder it pulls. That is why the digits double, and it is why "instant" is not an exaggeration for a good start.
But read the same expression the other way. The gain is (f''/2f') * e. Convergence needs that quantity to be smaller than one. So the loop only contracts while you are already inside a neighbourhood set by the ratio of curvature to slope. Outside it, the same squaring that raced you in now races you out, and each step throws you further than the last.
And what happens where the slope itself is near zero? The step divides by f'(x). A guess that lands near a flat spot on the curve produces a tangent that is nearly horizontal, and a nearly horizontal line meets zero a very long way away. One step can hurl you into a different region of the problem entirely. There is a classic pocket example: x^3 - 2x + 2 started at zero sends you to one, and from one straight back to zero, forever — a stable two-cycle with no root in sight.
So the honest answer to the opening question is that the starting point is not a detail. Every root has a basin of attraction: the set of starts that eventually reach it. Ask what the boundary between two such basins looks like and the subject stops being arithmetic. Cayley posed exactly this in 1879. For a quadratic he settled it — the plane splits cleanly along the perpendicular bisector between the two roots. For a cubic he could not, and he was right to be stuck: the boundary for z^3 - 1 is a fractal, and it has the strange property that almost every point on it borders all three basins at once. That is why two starting guesses a hair apart can end up at different answers. The rule is deterministic, and it is still, in the technical sense, chaotic.
The analogy
THE ANALOGY #Picture skiing down an unfamiliar slope in fog, allowed to look only at the ground beneath your skis. You read the gradient at your feet, assume the hill keeps that gradient, and aim for where that straight slope would reach the valley floor. Where the ground genuinely is smooth and steep, you land within metres of the bottom and the next glance corrects almost everything. Where you cross a nearly level shelf, your reading says the valley is kilometres away, and you launch across a ridge into an entirely different valley.
a skier keeps moving through the terrain and feels every bump on the way, whereas Newton's method teleports — it never evaluates anything between the two guesses, which is exactly why it can pass over the root it wanted without noticing.
Clarifying the model
THE MODEL #A few refinements tie the strands together.
The doubling of digits is not a property of the method alone; it needs a simple root, one where the curve genuinely crosses rather than touches. At a repeated root the slope vanishes along with the value, the quadratic term cancels, and convergence degrades to merely halving the error each step — the same speed as bisection, from a method with far more ways to fail.
It is also worth correcting a common reading of the failures. They are not a sign that the method is unreliable in some vague sense. Each failure is a specific, diagnosable event: a near-zero derivative, a cycle, or a start outside the basin. That is why practical solvers do not abandon Newton — they fence it, damping the step when it would overshoot or falling back to a bracketing method that cannot escape, and handing control back to Newton once the iterate is close enough for the squaring to work in their favour.
Finally, "close enough" is not a universal distance. It is set by f''/2f' at the root — by how sharply the curve bends relative to how steeply it crosses. A gently bending, steeply crossing root has a huge basin; a sharply bending, shallowly crossing one has a tiny one.
A picture of it
THE PICTURE #How to readstart at the input node at the top and follow the two gates. The right-hand answers are the failure paths: a flat slope throws the guess far away, and a guess outside the basin loops back rather than terminating. Only the branch where both gates answer favourably reaches the rounded outcome where the error squares.
What became clearer
WHAT CLEARED #Newton's method is a feedback loop whose gain depends on its own error. That single fact explains both halves of the puzzle: near a simple root the gain is tiny and shrinking, so convergence is explosive; far from one the gain exceeds one, so divergence is equally explosive. The method has no opinion about which of those it is doing — the starting point decides.
Where to go next
ONWARD #- The secant method, which replaces the exact derivative with a finite difference and trades some speed for not needing one.
- Damped and trust-region Newton, the practical machinery for keeping the step honest when it wants to overshoot.
- Newton fractals as a gateway into Julia sets and complex dynamics.
Key terms
TERMS #| Term | What it means |
|---|---|
| Quadratic convergence | error at each step proportional to the square of the previous error. |
| Basin of attraction | the set of starting points from which an iteration converges to a particular root. |
| Simple root | a root where the function crosses zero with non-zero slope. |
| Newton fractal | the picture of the basins in the complex plane, whose boundary is a Julia set. |
Every term the collection defines is gathered in the glossary.