Latent Verse / Part 2
UMAP flattens high-dimensional embeddings into two dimensions so you can see the whole landscape at once.
By Ian Johnson
In the previous article, we looked at embeddings one at a time or in pairs. We could search for similar jokes, but we couldn't see the space they live in. Each joke is a point in 384-dimensional space. We need a way to flatten that space onto a surface our eyes can actually read.
This is what dimensionality reduction does. It takes high-dimensional data and projects it down to two dimensions, trying to preserve the relationships between points. Points that were close in 384 dimensions should stay close in 2D. Points that were far apart should stay apart.
The algorithm we use is called UMAP (Uniform Manifold Approximation and Projection). Here are 1,000 dad jokes, projected from 384 dimensions down to two:
Hover over the dots. You'll find that nearby jokes tend to be about similar things. There may be a clump of animal jokes here, food puns there, science humor in another corner. The model's understanding of meaning creates this structure, and UMAP makes it visible.
UMAP is good at preserving local neighborhoods. If two jokes are close in the original 384-dimensional space, they'll usually be close on the 2D map. This is why you see coherent clusters of related jokes.
But it distorts global distances. The distance between two far-apart clusters on the map doesn't tell you much about how different those topics actually are. UMAP can place unrelated clusters near each other or related clusters far apart, depending on how the algorithm resolves the compression from 384 dimensions to 2.
Trust the neighborhoods, not the distances. If two points are close on a UMAP plot, they're likely similar. If they're far apart, you can't conclude much.
UMAP has two key parameters that change the character of the projection:
n_neighbors controls how many nearby points UMAP considers when deciding where to place each point. Low values (5-10) emphasize very local structure, creating tight, separated clusters. High values (30-50) capture broader patterns, producing more connected, flowing shapes.
min_dist controls how tightly points can pack together. Low values (0.01) allow dense clumps. High values (0.5) spread points out more evenly.
Use the controls below to see how these parameters change the same 1,000 jokes:
There is no single "correct" projection. Each parameter setting reveals different aspects of the data's structure. Tight clusters are useful for finding distinct groups. Spread-out projections show gradients and transitions between topics.
So far we've looked at 1,000 jokes. Here is the full dataset: 53,133 dad jokes, all projected at once. At this scale, the broad topography of the joke-space becomes visible.
There is something important to know about UMAP: it is computed fresh for each dataset. If you add new jokes and re-run UMAP, the whole map reshuffles. The neighborhoods stay mostly intact, but the overall layout changes. Your map of 1,000 jokes looks nothing like your map of 53,000, even though the same jokes appear in both.
This makes UMAP a local tool. It gives you a map of this dataset, not a map of language in general. Every time you project, you get a new map. In Article 5, we'll see how to build a permanent map that gives every text a consistent address. But first, let's find structure in the landscape we can see.