Visual grouping

Why do two items on a page read as a pair simply because they sit closer together than anything else?

Put a caption under a photograph with a generous gap, and slide a second caption up close beneath it. Nobody has to be told which text belongs to which. Move nothing but the whitespace and the ownership changes — reliably, instantly, and without any option to see it otherwise.

That is a strange amount of meaning to get out of empty space. No line was drawn, no label added, nothing about either item changed. So where did the pairing come from, if not from the marks?

Begin with what the eye actually receives: a two-dimensional pattern of light on the retina. It contains marks. It does not contain objects, groups, or belonging — those are conclusions, and the pattern is consistent with a vast number of them. Two dark patches on your retina might be one object partly hidden, two unrelated objects at different distances, a shadow and a thing, or a painted decoy. The image cannot settle it, because many arrangements of the world project to exactly the same image.

Sit with the awkwardness of that. Perception is not a readout. It is an inference from evidence that underdetermines its answer, which means the visual system cannot avoid bringing assumptions of its own — and if it must guess, the useful question is which guesses are worth having.

Here the answer is not arbitrary. Which regularities does the actual world supply? Parts of one object tend to be near each other; they tend to share colour and texture; their edges tend to continue smoothly rather than turn at random; they tend to move together. A visual system with those priors will be right far more often than not, and one without them will be right by accident.

That list is not modern. The Gestalt psychologists, principally Max Wertheimer in the 1920s, catalogued grouping as a set of principles: proximity, similarity, good continuation, closure, common fate, and later additions such as common region and connectedness. They were describing what people see, not why — and their catalogue turns out to be precisely the set of cues the priors above would recommend.

The recent work closes that gap by measuring the world. Analyses of natural photographs find that edge fragments belonging to the same contour really do co-occur at short separations and shallow turning angles, in about the proportions the grouping principles assume. So the principles look less like laws of form and more like statistics of scenes, absorbed into a visual system that treats them as defaults.

Which returns us to the caption. Proximity is the strongest of the cues and the easiest to control, so spacing overrules almost everything else you might have done — and the practical rule falls straight out: the gap between groups must exceed the gap within them, or no amount of styling will rescue the reading. Whitespace is not the absence of design. It is the instrument.

Two corrections are worth making carefully.

The first is a translation. Koffka's famous line is usually quoted as "the whole is greater than the sum of its parts", which is not what he wrote and not what he meant; his German says the whole is something other than the sum of its parts, and he objected to the "greater" version in print. The distinction matters here. Nothing is being added to the marks. What the grouping produces is a different kind of thing — a structure the parts do not individually possess — and reading it as addition invites the vague idea that perception supplies extra content, when it is really supplying an interpretation.

The second is about the status of the principles. It is easy to treat them as a rulebook the eye obeys. They were assembled by observation, they routinely conflict — proximity pulling one way while similarity or a shared enclosure pulls another — and the classical framework offered no principled account of who wins, only the umbrella idea of Prägnanz, that perception settles on the simplest, most regular organisation available. Modern treatments recast the cues as probabilistic evidence combined into a best estimate, which does say how conflicts resolve; but the combination is still only partly characterised, and how early in visual processing grouping happens remains argued.

The useful residue for anyone laying out a page: you are not decorating, you are supplying evidence to an inference engine that will draw a conclusion whether or not you intended one.

• How the cues are weighted against each other when they disagree, and what happens at the point of balance.
• Why figure and ground must be settled before grouping can even begin, and what bistable images reveal about that.

Every term the collection defines is gathered in the glossary.