Adaptive step size

Why does a solver that keeps changing its own step size beat one with a well-chosen fixed step?

You are integrating a differential equation forward in time. You could pick one step size and march. Or you could use a solver that recomputes its step at every single step, spends effort estimating its own error, and sometimes throws a completed step away and redoes it smaller.

The second sounds like pure overhead. It does extra arithmetic per step and occasionally wastes a whole step. Yet essentially every production solver — Dormand and Prince's method, the Fehlberg pair, the routines behind ode45 and solve_ivp — works that way, and they beat carefully tuned fixed-step runs on most problems. What does the adaptation buy that pays for all that?

Ask first how a fixed step gets chosen. You need the answer accurate everywhere, so you must pick a step small enough for the hardest part of the trajectory — the sharp turn, the near-collision, the moment a fast transient fires.

Now ask what that step does everywhere else. On a long smooth stretch, where the solution is nearly a straight line, the same tiny step is still being taken — thousands of them, each costing the same arithmetic, each contributing almost nothing. The total work is set by the worst point multiplied by the whole interval, even though the worst point may occupy a thousandth of it.

So the fixed-step solver is not paying for accuracy. It is paying for accuracy it does not need, over most of the domain. That already tells us where the win must come from: not from being cleverer at any one step, but from spending effort in proportion to how hard each region actually is.

But that raises the real difficulty. You cannot know in advance which regions are hard — if you knew the solution's behaviour, you would not be solving for it. So the solver must find out as it goes. How could it possibly judge a step it has just taken, with no exact answer to compare against?

The standard answer is to compute the same step twice at different accuracies and compare. An embedded Runge-Kutta pair does this economically: from one set of function evaluations it forms two estimates of the new point, one of order p and one of order p+1. The difference between them is not the true error, but it is a usable estimate of the local error committed on that step — the error introduced by this step alone, assuming you started it from an exact value.

Now the control law. For a method of order p, the local error on a step of size h scales roughly as h^p+1. So if the estimate came back as err and you wanted tol, the step that would have hit the target is about h times (tol/err) raised to the power 1/(p+1), multiplied by a safety factor — around 0.9 is conventional — and clamped so the step cannot leap or collapse by more than a set ratio in one go.

That single formula does both jobs, and this is the point worth pausing on. If the estimate is comfortably under tolerance, the same expression tells the solver to grow; if it is over, the step is rejected, the formula supplies a smaller one, and the step is retried. Growth and retreat are not two mechanisms but one, driven by the measurement.

Which is why the retried steps are not the waste they look like. A rejection is how the solver discovers a feature it had no way to anticipate — and having discovered it, it shrinks only there, then expands again on the other side. The total work ends up proportional to how much resolution the trajectory genuinely demanded, not to its worst moment stretched across the whole interval. On a problem with one sharp event in a long quiet run, that is a difference of orders of magnitude.

Two honest limits. First, controlling local error does not control global error — the accumulated deviation from the true trajectory. Local control is a proxy, usually a good one, but on chaotic or long-horizon problems the global error can be far larger than the tolerance suggests. Second, for stiff problems an explicit method's step is limited by stability, not accuracy: push it up and the solution blows up rather than merely drifting. The controller then thrashes at a tiny step for reasons no tolerance setting will fix, and the correct response is an implicit method, not a better controller.

The natural misreading is that adaptive solvers are more accurate. Usually they are not aiming to be: given the same tolerance, they aim for the same accuracy at far lower cost. The gain is efficiency, and the accuracy is what you asked for.

It is also worth being precise about the tolerance. It is a per-step budget on the local error estimate, not a promise about the final answer: two solvers at the same tolerance can deliver visibly different global accuracy.

Finally, notice the shape of the argument, because it is not really about differential equations. A fixed parameter must be sized for the worst case and therefore wastes effort in the average case; a measurement of local difficulty converts that fixed cost into a variable one. The same reasoning drives variable bit-rate encoding and congestion windows that grow until loss appears — and it fails the same way in all of them, when the measurement stops being a faithful signal of what is being controlled.

• Embedded Runge-Kutta pairs: how one set of evaluations yields two orders of accuracy.
• PI and PID step-size controllers, which smooth the sequence of steps rather than reacting to one estimate.
• Stiff equations, and why implicit methods change the whole calculation.
• Dense output: getting values between the steps an adaptive solver actually took.

Every term the collection defines is gathered in the glossary.