Embedding space

How can a search find the passage about getting money back when the document only ever says refund?

A customer types "how do I get my money back". The relevant paragraph says "refunds are processed within ten business days" and contains not one of those words. A keyword index — which stores, for each word, the documents containing it — has nothing to match on. Yet the search returns the paragraph.

The tempting explanation is a synonym list: somebody wrote down that "money back" means "refund". But that cannot scale to "cancel my order and put it on my card", "I was charged twice", "reverse the payment". So ask the better question: what would you have to store about a piece of text so that similarity of meaning became something you could compute?

Notice first what a keyword index makes text into. It reduces a passage to the set of symbols it contains, and symbols are either identical or unrelated. There is no partial match between "refund" and "reimbursement" — as tokens they are as different as "refund" and "raccoon". The representation itself has thrown away the thing we now want.

So we need a representation with a notion of nearness. What kind of thing has nearness built in? Points in space. Give every passage a vector — a list of, say, a thousand numbers — and you get distance for free. The whole problem becomes: how do you assign the vectors so that distance means what you want?

The old answer was distributional. Words that appear in similar contexts tend to mean similar things — Firth's slogan from 1957, that you shall know a word by the company it keeps. If you place words so that co-occurring words land near each other, "refund" and "reimbursement" converge because they turn up amid the same neighbours: charge, payment, invoice, days. Nobody wrote a synonym rule; the geometry fell out of the statistics of usage.

That handles words. But "get my money back" is not a word, and averaging its words' vectors loses word order and negation together — "not a refund" averages to something close to "a refund". The modern answer runs the whole passage through a transformer, which reads it in context, and takes a single vector from that. Now the vector is a function of the phrase as a phrase.

And crucially, the model producing it is trained specifically for the retrieval job, not just for language. The standard method is contrastive: show it real question-and-passage pairs, and pull each question's vector toward its true passage while pushing it away from other passages in the batch. Do that over millions of pairs and the space arranges itself around the property you asked for — questions land near their answers, which is not the same relation as "means the same thing".

Then similarity is arithmetic. Take the cosine of the angle between two vectors, which asks whether they point in the same direction while ignoring magnitude. The query becomes a vector; the passage nearest it in angle is your result. No word ever had to match.

Two honest cautions. Embeddings capture topical relatedness more reliably than fine distinctions, so "refunds are processed in ten days" and "refunds are not available on sale items" sit close together while meaning opposite things — negation and specific qualifiers are a known weakness. And the geometry inherits whatever the training data contained, including its biases; the celebrated word-analogy arithmetic of the word2vec era turned out to be partly an artefact of how the analogy queries were evaluated, which is a useful reminder that the space is a statistical summary rather than a map of meaning.

Three refinements.

First, dimensions are not features you could name. It is tempting to imagine one axis for formality and another for finance, and occasionally a direction does turn out interpretable. But the space is learned as a whole and its axes carry no assigned meaning; what is meaningful is relative position, not any coordinate.

Second, the embedding model at query time must be the same one used at index time, or at least its matched pair. The vectors are only comparable within the space they were produced in — swap the model and re-index, or every distance you compute is nonsense. This is a real operational cost: upgrading an embedding model means re-embedding the entire corpus.

Third, correct a misconception the demo encourages. Semantic search does not replace keyword search, it complements it. Ask for an exact product code, a surname, a version number, and the embedding is a liability — it will happily return a passage about a similar code. Keyword matching is precise about symbols and blind to meaning; embeddings are the reverse. Production systems commonly run both and fuse the rankings, which is a frank admission that neither representation is sufficient.

• How hybrid retrieval fuses a keyword ranking with a vector ranking, and why reciprocal rank fusion is the common choice.
• Why comparing every stored vector to the query is infeasible at scale, and what approximate search gives up instead.

Every term the collection defines is gathered in the glossary.