Latent Verse / Part 5
A parametric UMAP gives every text a permanent address. Combined with SAE, it forms a complete atlas of meaning.
By Ian Johnson
In Article 2, we saw that UMAP gives you a map of your data. But that map is disposable. Add new data, re-run UMAP, and the whole layout reshuffles. The neighborhoods stay roughly intact, but every point moves. Two UMAP plots of overlapping datasets can't be compared directly, because they don't share a coordinate system.
This is fine when you just want to explore one dataset. But it creates problems when you want to do any of these things:
What we want is a permanent coordinate system. A map that doesn't change when the data changes. A map where every text already has an address.
The approach is straightforward in concept: instead of running UMAP from scratch on each dataset, train a neural network to learn the UMAP projection. Feed it millions of diverse embeddings along with their UMAP coordinates. The network learns a function: embedding goes in, two coordinates come out.
Once trained, this network can project any new embedding instantly. It doesn't need to see the whole dataset. It doesn't need to recompute. It gives every text a consistent position on a permanent map, because the map was shaped by the vast corpus the network trained on.
This is called parametric UMAP. "Parametric" means the projection is defined by the learned parameters of the network, rather than by the specific dataset you're looking at. Once you have a trained network, you have a basemap: a global map of meaning that any new text can be placed onto.
Here are the same dad jokes plotted two ways: on a fresh local UMAP computed from just these jokes, and on a global basemap trained on millions of diverse texts. Hover to see the jokes. Watch which points move and which stay close.
The local UMAP treats the 100 jokes as the whole universe. The basemap treats them as a small region of a much larger territory. On the basemap, all 100 jokes might end up in a corner of the full space, because "dad jokes" is just one neighborhood within the map of all text. The fine structure within that neighborhood is still visible, but now it sits in a broader context.
Every SAE feature has a characteristic activation pattern: some jokes activate it strongly, others not at all. When we color the map by feature activation, we can see where each concept lives in the coordinate system. Features tend to concentrate in specific regions of the map.
Pick a feature below and watch it light up on the map:
This is what makes the combination powerful. The basemap gives every text a position. The SAE tells us what concepts activate in that text. Put them together, and the coordinate system starts to mean something: a region of the map is a region of meaning, not just an arbitrary cluster of similar dots.
Now we have all the pieces of the instrument:
Click any joke on the map below. The panel that appears shows everything the microscope reveals about it: where it lives, what concepts it activates, and what its nearest neighbors are in the full embedding space.
To make the pipeline concrete, here is what happens to a single joke as it passes through the complete instrument. Each stage transforms the joke in a different way. Pick a joke and watch it travel:
An atlas is more than a map. It's a map plus a vocabulary: the names of places, the features of terrain, the categories that help you navigate. The basemap provides the geography. The SAE taxonomy provides the vocabulary. Together they form a system for understanding what an embedding model has learned.
Different embedding models produce different atlases, just as different lenses show different things. A model trained on scientific literature produces an atlas where chemistry papers form dense continents. A model trained on source code produces an atlas where function signatures cluster together. Within any one atlas, though, everything has a place and a description: a dad joke, a news article, a line of code, each locatable and describable in the same terms.
The microscope is complete. The lens focuses meaning into points. The projector shows where those points live. The filter shows what they are made of. These three instruments, trained to match each other, let you look at any text through a consistent view of the space it inhabits.
This series has walked through the components of a latent space workbench: embedding, projection, clustering, feature decomposition, and global mapping. Each piece is interchangeable. You can swap in a different embedding model, try a different clustering algorithm, train a basemap on a different corpus, or use a different SAE architecture. The pipeline stays the same. The views change.
We used dad jokes because they're fun, small, and accessible. The same tools work on any text. The workbench was built to be applied broadly: to scientific literature, to legal documents, to customer feedback, to code repositories, to anything written down. Every application reveals something different about the data and something different about the model looking at it.
The point has been the same throughout. Models have already learned something about your data. These tools help you see what they learned. Embeddings, projections, clusters, features, maps: none of them are the data, and none of them are the model. They're instruments for looking through one at the other.
Now go put something under the microscope.