A workbench for studying latent spaces.
Language models learn something about the meaning of text. When they embed a sentence, they compress their understanding into a list of numbers, placing it at a specific point in a high-dimensional space. Similar texts land near each other. Related ideas form neighborhoods. Structure emerges that no one explicitly designed.
This series explores that structure. Using 50,000 dad jokes as our specimen, we build up a set of tools for seeing what models have learned: projecting high-dimensional spaces into visible landscapes, finding and naming groups, decomposing embeddings into interpretable features, and constructing permanent maps of meaning.
Think of it as setting up a microscope. Each article adds a piece of the instrument. By the end, familiar data looks completely different.
An embedding model turns text into numbers that encode meaning. Similar texts get similar numbers.
UMAP flattens high-dimensional embeddings into two dimensions so you can see the whole landscape.
Clustering algorithms find groups in the data. Labels give those groups names. Both are choices, not facts.
A Sparse Autoencoder decomposes embeddings into interpretable features. It works on any text, not just this dataset.
A parametric UMAP gives every text a permanent address. Combined with SAE, it forms a complete atlas of meaning.