Latent Verse / Part 1
An embedding model turns text into numbers that encode meaning. Similar texts get similar numbers.
By Ian Johnson
Take a sentence and feed it to a language model. Instead of asking the model to generate more text, ask it a different question: what did you notice about that sentence? The model's answer comes back not as words, but as a list of numbers. Hundreds of them.
This list is called an embedding. Each number is a dimension. Together they place the sentence at a specific point in a high-dimensional space. The model has compressed everything it understands about the meaning of that text into this single point.
Here is a dad joke, and its embedding:
Squinting at that bar chart, there is no obvious pattern a person could read. But the model that produced it was trained on enormous amounts of text, and the positions of these numbers relative to each other encode something about what the sentence means.
To see the structure, we need to compare.
When two pieces of text are about the same thing, their embeddings land near each other in that high-dimensional space. We can measure how close they are with cosine similarity: a score from 0 (completely different directions) to 1 (pointing the same way).
Pick two jokes below and watch their embeddings. When the jokes are similar in meaning, you'll see the similarity score climb. When they're unrelated, it drops.
Notice that similarity here is about meaning, not surface-level word overlap. Two jokes about animals might use completely different words but still land near each other. The model learned associations from millions of examples. It recognizes topical similarity, structural patterns, and semantic relationships that go beyond keyword matching.
We have a collection of dad jokes, each one already embedded. To search them, we embed a query and find the jokes whose embeddings are closest. This is nearest neighbor search: finding the points in the space that are most similar to your point.
Pick an example below or type your own query. The results often surface connections that a keyword search would miss entirely.
This is the core idea: the embedding model has already learned something about language. It has read enough text to develop a sense of what words mean, what topics relate to each other, what structures look similar. When we embed our data through this model, we get to look at our data through the lens of everything the model has learned.
You can learn about data by looking at what models have learned. The model is the lens. Your data is what you see through it.
There are many embedding models, each trained on different data with different objectives. One model might be trained for search relevance, another for semantic similarity, another for code understanding. The same text, embedded through different models, produces different numbers and different similarity rankings.
This matters because the choice of model shapes what you see. A model trained heavily on scientific papers might place two chemistry jokes closer together than a general-purpose model would. A model trained for retrieval might care more about topical overlap than tone or structure.
For this series, we use bge-small-en-v1.5, a model that produces 384-dimensional embeddings. It is a general-purpose lens: competent across a wide range of text, not specialized for any one domain. In the next article, we will take these embeddings and project them into two dimensions so we can see the whole landscape at once.