Picture a night sky where every word is a star. They are not scattered at random: the ones that speak of similar things shine together, forming constellations. "River", "lake" and "stream" cluster in a damp corner of the firmament; "fury", "wrath" and "rage" burn together in another. And if you had eyes to see it, you would notice that the distance between two stars is no whim: it measures how alike their meanings are. That sky exists. It is not a poetic metaphor I made up for a pretty opening: it is, literally, how modern machines store language. It is called the vector embedding space, and within it meaning stopped being an abstract idea and became geometry.
01 · the vectorA word that turns into a coordinate
An embedding is, in essence, a list of numbers that represents a word, a phrase or even an image. Where you read "cat", the machine stores something like [0.24, −0.81, 0.05, …]: hundreds of numbers, each one a coordinate on a different axis. Just as a point on a map needs two coordinates (latitude and longitude) and a point in space needs three, a word in this semantic universe lives in a space of hundreds of dimensions (typically 300, 768 or more). We cannot picture 300 axes, but the intuition holds: each word is a point, and its position is not arbitrary. A model learned it by reading vast amounts of text, guided by an old linguistic hunch: a word is known by the company it keeps. Words that appear in similar contexts end up close together. The geometry is a residue of usage.
The distance between two words is no whim: it measures how alike their meanings are.
02 · the closenessSimilarity is distance, and distance is measured by an angle
If meaning is position, then similarity is closeness. Two similar ideas are two neighboring points; two unrelated ideas, two distant points. But here the geometry takes an elegant turn. In these spaces we usually measure not the straight-line distance but the angle between the vectors: the so-called cosine similarity. Picture two arrows leaving the origin toward their respective stars. If they point in almost the same direction, the angle between them is small and its cosine approaches 1: they mean almost the same thing. If they are perpendicular, the cosine is 0: they are unrelated. And if they point in opposite directions, it approaches −1. The beauty of measuring the angle rather than the length is that what the machine cares about is where meaning points, not how long the vector is. Direction is the sense; magnitude, almost a detail.
03 · the arithmeticKing − man + woman ≈ queen
Here occurs the marvel that made this field famous. In 2013, a Google team led by Tomáš Mikolov presented word2vec, a method for learning these vectors from raw text [1]. And they found something astonishing: the vectors did not merely cluster similar words, they encoded relations as directions. Subtracting the vector for "man" from the vector for "king" isolates something like the essence of royalty, stripped of gender. If you add "woman" to that result, you land, very close, at the point where "queen" lives. In notation: king − man + woman ≈ queen.
Read it with the geometry of Figure 2 in mind. The step from "man" to "king" is a concrete vector (an arrow with direction and length) that represents "becoming royalty". That same shift, applied to "woman", takes you to "queen". The four words trace a parallelogram. What is remarkable is that no one programmed this: it emerged on its own from the model predicting which words accompany which. Gender, plurality, even "capital of a country", became consistent directions that can be added and subtracted. Meaning had become algebra.
No one programmed the analogy: it emerged on its own from text. Meaning became algebra.
04 · the lineageFrom word2vec to GloVe: two paths, one sky
A year later, in 2014, Jeffrey Pennington, Richard Socher and Christopher Manning, at Stanford, proposed GloVe (Global Vectors) [2]. Where word2vec learned by sliding a window through the text, predicting neighbors word by word, GloVe started from a global co-occurrence matrix: it counted how many times each pair of words appears together across the whole corpus, and adjusted the vectors so their geometry mirrored those statistics. Two distinct routes, one local and predictive, the other global and count-based, that flowed into the same firmament: spaces where closeness is kinship of meaning and relations are directions. These works founded the intuition that today underpins the large language models, whose embeddings are incomparably richer, yet descend from this same idea.
If meaning is geometry, searching by sense is searching by closeness. That is the basis of semantic search: instead of hunting for exact words, you turn your question into a vector and retrieve the neighboring points in the space: the texts that mean something similar, even if they share not a single word. And it is also the heart of RAG (retrieval-augmented generation): before answering, a system like Qirava transforms your query into a vector, travels to the firmament of its documents, finds the nearest constellations and hands them to the model so it can answer with grounding and not from memory. The geometry of meaning is not merely beautiful: it is what lets a machine understand you when you ask in your own words.
A note of honesty is worth making: these spaces inherit what they read. If the text that trains the model carries prejudices, the geometry absorbs them as directions too, and that is a constellation worth watching with a critical eye. But even there the lesson holds: the vector space is a mirror of human language, with its light and its shadows, translated into coordinates. Learning to read that map is learning to read ourselves.
Sources
- Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv:1301.3781. (Introduces word2vec and the analogy arithmetic such as "king − man + woman ≈ queen".)
- Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global Vectors for Word Representation. Proceedings of EMNLP 2014, pp. 1532-1543.