Give a machine the word "king" and it won't see a K, an I, an N, a G. It'll see a point suspended in a map of hundreds of dimensions, with an exact coordinate. Give it "queen" and it'll land at another point, close to the first. "Throne", "crown", "monarch" will fall in the same neighborhood. "Carrot" will end up on the other side of the world. That map has a name: it's called an embedding, and it's how modern machines store the meaning of things. Its rule is beautiful: close means alike. This article is about how that map gets drawn.
A classic search engine compares letters: type "auto" and it won't find "car", because they share no characters. But an embedding map would place "auto" and "car" almost on top of each other. The difference between those two ways of searching is the difference between spelling and understanding. And it all starts by turning a word into a place.
01 · the coordinateWhat embeddings are: a word turned into a place
An embedding is a list of numbers that represents the meaning of something (a word, a sentence, an image) as a point in space. Where you read "cat", the machine stores something like [0.24, −0.81, 0.05, …]: hundreds of numbers, each one a coordinate on a different axis. A point on a paper map needs two coordinates (latitude and longitude); a point in a room needs three; a word in this universe needs hundreds (typically 384, 768, 1536 or more, depending on the model). We can't draw 768 axes, but the intuition holds: every word is a point, and its position is no accident.
That number, how many coordinates each point has, is called the embedding's dimension, and it's fixed by the model that produces it. More dimensions give more room to separate nuances; fewer dimensions take less memory and compare faster. It's the first engineering decision behind any semantic map: how much detail fits in each point.
An embedding is the meaning of a word written as a postal address on a map of hundreds of dimensions.
02 · the drawingHow the map gets drawn: by the company a word keeps
Nobody places the points by hand. Nobody decides "I'll put 'king' at coordinate 0.24". The map is learned by a model reading enormous amounts of text, guided by an old linguistic hunch that the Briton J. R. Firth summed up in 1957: a word is known by the company it keeps [1]. This is the distributional hypothesis, and it's the engine behind everything that follows.
The mechanics, without formulas, go like this. The model slides a window across millions of sentences and asks itself the same question over and over: given this word, which words usually surround it? "Dog" appears near "barks", "leash", "bone"; "cat" near "purrs", "scratches", "rooftop". Since both share many neighbors ("pet", "vet", "petting"), the model keeps nudging them toward nearby coordinates. Word by word, adjustment by adjustment, the map orders itself: what's used alike ends up close together. Geometry is a residue of usage.
03 · the measureHow close: distance is read as an angle
If meaning is position, then similarity is closeness. Two similar ideas are two neighboring points; two unrelated ideas, two distant points. But it's worth being precise about what "close" means here, because it's usually not measured in a straight line. What's measured is the angle between the two vectors running from the origin to each point: the cosine similarity. If two arrows point in almost the same direction, the angle is small and its cosine approaches 1 (they mean almost the same thing). If they're perpendicular, the cosine is 0 (unrelated). The elegant part of measuring the angle and not the length is that the machine cares about which way a meaning points, not how long its arrow is.
That number between −1 and 1 is, in the end, what makes the whole map useful: it turns "are they alike?" into a sum a computer solves in microseconds. The table below translates the cosine into intuition.
| Cosine | Angle | What it means | Example |
|---|---|---|---|
| ≈ 1 | almost 0° | Almost the same meaning | "car" and "auto" |
| ≈ 0.5 | ~60° | Related, not identical | "car" and "road" |
| ≈ 0 | 90° | Unrelated | "car" and "carrot" |
| ≈ −1 | 180° | Opposite directions | rare cases in real text |
Source: author's own elaboration on the definition of cosine similarity; the examples are illustrative. The −1 to 1 scale is that of the cosine function.
Asking "are these two ideas alike?" becomes measuring an angle. Meaning turned into arithmetic.
04 · the reachNot just words: sentences, documents and images on the same map
The first famous map was of single words (word2vec, from Tomáš Mikolov's team at Google, in 2013 [2]) and with it came the marvel that "king − man + woman" landed near "queen": relationships had become directions. But the real power arrived when the map stopped being about words and turned into one of whole sentences and documents. Models like Sentence-BERT, introduced by Nils Reimers and Iryna Gurevych in 2019, learned to place a complete sentence at a single point [3], so that "how do I cancel my subscription?" and "I want to close my account" land as neighbors even though they share hardly a single word.
And the map isn't even limited to language. Multimodal models place a photo of a cat and the phrase "a tabby cat" in the same space, close to each other: that's why today you can search for images by typing what you want to see. Different doors (text, image, sound) that open onto the same firmament of coordinates.
If meaning is closeness, then searching by meaning is searching by proximity. You turn your question into a point, and you retrieve the neighboring points: the texts that mean something similar, even if they share not one word. That's semantic search, and it's the heart of RAG (retrieval-augmented generation). Before answering, a system like Qirava turns your query into an embedding, searches the map of its documents for the nearest points and hands them to the model so it answers with grounding rather than from memory. Storing and comparing millions of those points efficiently is, in turn, the job of a vector database.
A note of honesty is in order: these maps inherit what they read. If the text training the model carries prejudices, the geometry absorbs them as directions too, and that's a constellation worth watching with a critical eye. But even there the lesson holds. An embedding is a mirror of human language, with its light and its shadows, translated into coordinates. The next time a search engine understands what you meant and not just what you typed, remember the invisible map: someone turned your words into places, and measured how far they sat from what you were looking for.
Sources
- Firth, J. R. (1957). A Synopsis of Linguistic Theory, 1930-1955. En Studies in Linguistic Analysis, pp. 1-32. Oxford: Blackwell. (Origen de la frase "You shall know a word by the company it keeps", base de la hipótesis distribucional.)
- Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv:1301.3781. (Introduce word2vec y la aritmética de analogías entre embeddings de palabras.)
- Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of EMNLP-IJCNLP 2019, pp. 3982-3992. ACL Anthology: aclanthology.org/D19-1410.