Before comparing methods, we need to agree on what they are trying to preserve. This article builds intuition for high-dimensional space and the different kinds of structure that a dimensionality reduction method might care about.
An image classifier produces a 2048-dimensional feature vector. A language model produces a 4096-dimensional residual stream activation. A single-cell RNA sequencing assay produces a 20,000-dimensional expression profile. Each row of such data sits in a space we cannot draw, cannot picture, and cannot navigate directly. The whole point of dimensionality reduction is to give us a picture. But a picture of what?
Every dimensionality reduction method throws information away. The interesting question is not whether it distorts the data, but which properties it tries to keep and which it is willing to sacrifice. t-SNE, UMAP, PCA, PHATE, and Mapper are not different answers to the same question. They are answers to different questions. Once you see that, choosing between them becomes a question about your data and your goal, rather than a beauty contest between scatter plots.
This series works through the main families of methods one by one, organized by what they preserve. This first article establishes the shared vocabulary: what high-dimensional space actually looks like, what kinds of structure live in it, and how we can tell whether a projection has kept them or lost them.
Most of our visual intuition for points, distances, and neighborhoods comes from two- and three-dimensional space. Those intuitions do not extend to the spaces where real ML data lives. Before we can reason about methods that summarize high-dimensional data, we have to see how high-dimensional space behaves on its own.
Consider a uniform random cloud of points and measure the distances between every pair. In one or two dimensions, those distances are spread out: some points are close, some are far. As the dimension grows, something counterintuitive happens. The distances bunch up. Every pair of points ends up roughly the same distance apart. The scatter plot below lets you watch this happen.
In everyday three-dimensional space, the nearest point is clearly closer than the farthest point. In fifty-dimensional space drawn from the same uniform distribution, the ratio between them shrinks toward one. Notions we take for granted, like "the closest point is meaningfully close," weaken as dimension grows. This effect is sometimes called the concentration of distances, and it is a core piece of the curse of dimensionality.
Real data is rarely uniform, and this is exactly what saves us. A language model's embeddings do not fill their 4096-dimensional ambient space uniformly. They cluster, they trace curves and manifolds, they leave enormous regions empty. The meaningful information lives on a much lower-dimensional structure embedded in a high-dimensional container. Dimensionality reduction is the craft of finding that structure and rendering it at a size we can look at.
If the goal of dimensionality reduction is to preserve structure, we should be precise about what structure means. There is no single answer. The same dataset has many kinds of structure layered on top of each other, and different methods foreground different layers.
Take a simple two-dimensional dataset with two intertwined crescents. We can look at it through at least five different lenses, shown in the figure below. Each lens emphasizes a different property and ignores others. A dimensionality reduction method is, in effect, a decision about which lens to prioritize when the data gets squished.
The points view shows raw coordinates. Two crescents, each a different color. This is the rawest form of the data.
The distances view draws lines for every pairwise distance, thicker for shorter distances. It shows that each crescent is a tight cluster, but also that some points in one crescent are closer to points in the other crescent than to points at the far end of their own. A method that tries to preserve all pairwise distances will have a hard time with this shape, because the distances lie to you: two points are geometrically close while being meaningfully far apart along the crescent.
The neighbors view draws edges only to each point's k nearest neighbors. The crescents become two connected clusters. This is a much more honest picture of the data's shape, which is why many of the most successful methods we will meet later, t-SNE and UMAP among them, build on neighborhood graphs rather than raw distances.
The density view colors the plane by how many points live nearby. The dense cores of the crescents show as bright regions; the sparse tails fade out. Density is information that most methods discard: clusters get repositioned to be visually separated even when one is actually much denser than another.
The topology view ignores metric detail entirely and summarizes what connects to what. Two crescents show up as two connected components with no loops. Topology is a lossy summary, but it captures properties that survive distortion, and it is what a method like Mapper is designed to preserve.
No single projection can maintain all five views faithfully. When a method collapses 4096 dimensions to 2, it has to pick its battles. The rest of this series is organized around those choices.
Once we know what a method is trying to preserve, we can ask whether it succeeded. The field has settled on a few standard measurements, each of them probing a different kind of honesty.
Trustworthiness asks: of the points that appear close in the 2D picture, how many were actually close in the original space? A trustworthy embedding does not invent false neighbors. Continuity asks the reverse: of the points that were close in the original space, how many are still close in the picture? A continuous embedding does not rip apart neighborhoods that belonged together. A projection can be high on one and low on the other, and usually is. These two metrics together form a minimum viable check.
Stress measures how well all pairwise distances agree between the original space and the embedding. It is the natural metric when a method targets global distance preservation (like MDS), and an unfair one when the method deliberately warps distances in favor of neighborhoods (like t-SNE). Shepard diagrams plot original distances against embedded distances point by point, revealing at a glance where and how the projection distorts.
No single number captures quality, because quality itself is conditional on what you wanted to preserve. When we compare methods in later articles, we will return to these measures and show what each family looks good (and bad) at.
The remaining nine articles each take one family of methods, built around a single answer to "what should we preserve."
The progression is roughly from the simplest idea (preserve variance along orthogonal directions) to the most elaborate (learn a neural network whose latent space is the dimensionality reduction). Along the way, each family makes a new move that the previous family could not: preserve nonlinear distances, preserve neighborhoods rather than distances, preserve topology rather than geometry, preserve density, fit data to a grid, use a non-flat geometry, learn from data. We finish with a capstone article on open problems, where the families we have met suggest directions they have not yet gone.
Each article is self-contained, but they build on each other. The linear family sets up problems that the nonlinear families try to solve. The local-neighborhood family creates evaluation puzzles that the topological and density families respond to. The learned-representation family borrows ideas from everywhere else. Read in order for the narrative; read out of order for reference.