THIS EXPLANATION
THE ROOM
CDA·264 Computing, Data & AI 6 MIN · 8 STATIONS

Which record is the real one

A Socratic walk-through of which record is the real one — reasoned out one step at a time, not lectured.

abcdefgh
a

The question we started with

THE QUESTION #

When four systems each hold a different address for the same person, how do you decide which is true?

Four systems, four addresses, one customer. The instinct is to find a rule that picks the winner — most recent wins, or the CRM wins, or the system with the most complete record wins — and then to be slightly annoyed that nobody wrote that rule down years ago.

But the question hides two entirely separate problems inside it, and conflating them is why master data projects go wrong. Before we can ask which address is true, we have to have established that these four records are about the same person at all. And after we answer it, we still have to ask whether "true" is even the right word for a field where two different values can both be correct.

b

Reasoning it through

REASONING #

Take the first problem: deciding that four rows refer to one entity. This is record linkage, and it is a decision under uncertainty, not a lookup. If there were a shared identifier you trusted, there would be no problem — so by construction, the interesting cases are the ones without one.

What can you compare? Names that may be misspelled, abbreviated, married, or transliterated. Dates of birth with typos. Addresses in three formats. Now think about what an agreement is worth. If two records agree on surname Zylstra, that is strong evidence. If they agree on Smith, much weaker — and that difference is not a heuristic, it is the core of the standard method.

Fellegi and Sunter formalised this in 1969, and the framing is still what most tools implement. For each comparison field you ask two questions: how often does this field agree when the pair really is a match (call it m), and how often does it agree when the pair is not a match (call it u)? The evidential weight of an agreement is the ratio of those two, and you sum the weights across fields to get a score for the pair. Rare values agree by coincidence rarely, so u is small, so the weight is large — which is exactly why Zylstra tells you more than Smith.

Then, crucially, you do not set one threshold. You set two. Above the upper one, accept the link. Below the lower one, reject it. In between is the clerical review band — pairs the arithmetic cannot decide, referred to a human. Notice what that admits: some pairs genuinely do not have a determinable answer from the data available, and a system that pretends otherwise is choosing to be silently wrong rather than visibly uncertain.

Now the second problem, which only starts once linkage has succeeded. We have four records about one person and four different addresses. Which is right?

Here is the move that surprises people: the answer is decided field by field, not record by record. The rules that choose — survivorship rules — are per attribute. Take the most recently verified address, the phone number from the system whose users actually confirm it aloud, the legal name from the system with an audited onboarding process. And follow that through: the resulting record may be a composite that exists in no source system anywhere. That is not a defect. The "golden record" is a constructed best view, not a discovered original.

And then the honest complication. Sometimes the four addresses disagree because three are stale — a data quality problem. But sometimes they disagree because the person has a home address, a billing address, and a delivery address, and each system captured the one it needed. That is not a conflict at all; it is a modelling failure, an entity with several roles being squeezed into one field. Picking a winner there does not fix anything — it destroys information and produces a golden record that is wrong for three of the four consumers.

So the first question to ask of a disagreement is not "which wins" but "should these ever have been the same field".

c

The analogy

THE ANALOGY #
THE FIGURE

Think of a biographer with four accounts of one person's life: a passport, a set of letters, a newspaper obituary, and a colleague's memoir. The first job is establishing that all four are about the same individual and not a namesake — and that is done by weighing agreements, with an unusual detail worth far more than a common one. Only then does the second job begin: constructing the account. And the biographer does not choose one source and discard the rest. They take the birth date from the passport, the state of mind from the letters, the professional record from the memoir — and the biography that results is a document none of the four sources contains.

WHERE IT BREAKS DOWN

a biographer works on a life that has ended and a set of sources that will not change, whereas master data is a moving target — new records arrive daily, previously distinct entities turn out to be one, and yesterday's confident link sometimes has to be broken again, which no finished biography ever has to do.

d

Clarifying the model

THE MODEL #

Three refinements.

First, the misconception in the question itself. "Which record is the real one" presumes one of them is. Usually none is: the sources are each accurate for their own purpose and moment, and the mastered record is manufactured from them under stated rules. Master data management does not find truth; it makes a defensible construction and records why.

Second, mastering does not mean the sources stop existing. Most implementations keep the operational systems as systems of record and maintain a cross-reference — the mapping from mastered entity to each source's local key. That is what makes a link reversible when it turns out to have been wrong, and a linkage scheme with no cross-reference has quietly made every merge permanent.

Third, false merges and false splits are not symmetric harms. Failing to link two records for one customer is an inconvenience — duplicate mail, a fragmented view. Wrongly merging two different customers exposes one person's data to another and is very hard to unwind. So the thresholds should not be set to maximise raw accuracy; they should be set knowing which error you can afford.

e

A picture of it

THE PICTURE #
Which record is the real one
Which record is the real one Read top to bottom as one entity being assembled. The first three arrows are sources offering evidence, not answers. The self-arrow is the linkage decision -- pair scoring, not lookup. Notice that one pair does not clear the threshold and goes to a human rather than being guessed; that referral is a feature. Only after linkage is settled does the second self-arrow run, choosing values attribute by attribute, and the final arrow returns both the constructed view and the key that lets the merge be traced back or undone. {"generator":"[email protected]","source":"../Socrates/.diagram-cache/_src/which-record-is-the-real-one.md","sourceIndex":1,"sourceLine":4,"sourceHash":"e7519fffacf8d4523015d4f1871101533445e14eb7b49aac5225bb68550afcd5","diagramType":"sequence","layoutVariant":"source","repairedDuplicateIds":[],"motion":"entrance-with-reduced-motion-fallback","presentation":"editorial","attempt":1,"viewBox":{"x":0,"y":0,"width":1206,"height":856},"qa":{"passed":true,"findings":[]}} Steward 01 Hub 02 Support 03 Billing 04 CRM 05 record with name, date, address record with name, date, address record with name only score each pair by field agreement weight linked, above upper threshold linked, above upper threshold Support pair falls in review band confirmed same person apply survivorship rules per field mastered view plus cross reference key
KINDSlifelineparticipantmessage

How to readRead top to bottom as one entity being assembled. The first three arrows are sources offering evidence, not answers. The self-arrow is the linkage decision — pair scoring, not lookup. Notice that one pair does not clear the threshold and goes to a human rather than being guessed; that referral is a feature. Only after linkage is settled does the second self-arrow run, choosing values attribute by attribute, and the final arrow returns both the constructed view and the key that lets the merge be traced back or undone.

f

What became clearer

WHAT CLEARED #
WHAT CLEARED

There are two questions here, not one. Linkage decides which records describe the same entity, and it is probabilistic, with a band where the honest answer is "ask a person". Survivorship then decides, field by field, which value to publish — producing a record that may match no source exactly, and that is entirely correct, because the golden record is built rather than found.

g

Where to go next

ONWARD #
  • Blocking: how candidate pairs are narrowed before scoring, since comparing every pair is quadratic.
  • Registry versus consolidation versus centralised MDM, and what each does to the system of record.
  • When a disagreement should be modelled as multiple valid roles instead of resolved to one value.
h

Key terms

TERMS #
TermWhat it means
Record linkagedeciding which records across sources refer to the same real-world entity.
Fellegi-Sunter modelthe 1969 probabilistic framework scoring field agreements by how informative each is, with two thresholds and a clerical review band between them.
Survivorship rulesper-attribute rules choosing which source's value appears in the mastered record.
Golden recordthe constructed best view of an entity, assembled from several sources rather than selected from them.
Cross-referencethe stored mapping from a mastered entity to each source system's own key, which makes a link traceable and reversible.

Every term the collection defines is gathered in the glossary.

Nearby on the shelf

4