If we want the picture to tell us how far apart things are, we need a method that respects distances. But what counts as "distance" is itself a choice, and the field has produced three increasingly clever answers to it.
Linear methods force the data into a flat subspace. That works when the data is already flat, but most interesting data is not. A ribbon of points that curls through three-dimensional space cannot be laid out on a plane by any rotation. The distances through the ribbon are much longer than the distances through empty space, and it is the distances through the ribbon that carry the meaning.
This article is about methods that take distances seriously, and the key move each of them makes is redefining what distance means. Classical multidimensional scaling starts from a given distance matrix and asks for a low-dimensional arrangement that reproduces it. Sammon mapping adjusts the emphasis so that small distances are preserved more faithfully than large ones. Isomap replaces straight-line distance with the distance along the data. Diffusion maps replace it with the distance traveled by a random walk. Each is a different answer to the same question: what does "far" really mean?
The oldest member of the family, classical multidimensional scaling, takes a distance matrix as input. You hand it the pairwise distances between n items, and it returns a set of coordinates in k dimensions whose pairwise distances match your input as closely as possible.
You can think of MDS as the answer to a natural geometric question: given how far apart your items should be, where should they go? The classical solution is a double-centering of the squared distance matrix followed by an eigendecomposition, very much in the same family of tools as PCA. In fact, when the input distances are Euclidean distances of the original data, classical MDS is PCA, just reached from a different starting point. MDS is the more general tool because the input distances do not have to come from any embedding at all: they might be perceptual distances between colors, edit distances between strings, or any pairwise similarity you can encode.
Classical MDS tries to match all pairwise distances with equal emphasis. That is a reasonable goal only when the data is roughly flat. When it is not, classical MDS ends up compromising every distance and getting none of them right. The next member of the family pushes on this.
Sammon mapping, introduced by John Sammon Jr. in 1969, starts from the observation that not all distance errors are equally bad. If two items are supposed to be 10 units apart and the embedding places them at 11, that is a minor error. If they are supposed to be 0.5 units apart and the embedding places them at 1.5, the topology is wrong: what should have been near neighbors have been pulled apart.
Sammon's stress function divides each squared distance error by the original distance itself, giving small distances higher weight. The effect is that the optimizer fights hard to get local relationships right, tolerating larger errors in the global layout when necessary. If you slide the weighting control in Figure 1, you can watch the embedding shift: under classical MDS, far-apart items have their distances respected at the cost of the close-together ones; under Sammon, the reverse.
Philosophically, Sammon is the bridge from the MDS family to the neighbor-based family we will see in article 4. Once you notice that local relationships matter more than global ones for most real data, you are halfway to t-SNE.
Even Sammon mapping computes distances as Euclidean distances in the original coordinate space. That is a problem for curved data. Consider the classic example: a two-dimensional ribbon rolled up into three dimensions, the Swiss roll. Two points at opposite ends of the ribbon are far apart along the ribbon, but they might be close to each other through the empty air in the middle of the roll. Any method that uses Euclidean distance will treat them as neighbors and fold the ribbon back on itself.
Isomap, introduced by Tenenbaum, de Silva, and Langford in 2000, fixes this with a beautifully simple trick: redefine distance as shortest path in a k-nearest-neighbor graph. Build a graph where each point connects only to its k closest neighbors. The distance between two points is the total length of the shortest path through the graph between them. For a swiss roll with reasonable point density, this geodesic distance respects the curvature: two points at opposite ends of the ribbon are far apart even if they sit next to each other in three dimensions, because the shortest path has to walk all the way along the ribbon.
Feed the geodesic distance matrix into MDS and you have Isomap. The embedding now tries to preserve distances along the manifold, which is usually what we meant by "distance" in the first place.
Isomap's weakness is the same graph that makes it work. If the sampling is too sparse, the shortest-path approximation becomes unreliable. If k is chosen too large, shortcut edges appear: an edge connecting points that are close in 3D but far along the manifold, which crash the geodesic computation back toward a Euclidean one. The method is at its best on densely sampled, smooth manifolds and gets shakier as those assumptions weaken.
Isomap measures distance as the shortest path through a single chain of neighbors. But what if we care about how many paths there are between two points, not just the length of the shortest one? Two regions that are connected by many independent paths should feel "closer" than two regions that are connected by only one thin bridge, even if the single thin bridge is shorter.
Diffusion maps, developed by Coifman and Lafon in 2006, formalize this intuition by defining distance through a random walk. Imagine releasing a cloud of walkers at one point. They wander around the kNN graph, stepping to random neighbors with probabilities weighted by similarity. After a few steps, the cloud has spread: some of it has reached point B, more of it has reached point C. The diffusion distance between two points is the difference between where the walker goes from one vs. where it goes from the other, averaged over all possible destinations.
Diffusion distance is robust to noise (a single bad edge has little effect if there are many good ones) and sensitive to bottlenecks (two well-connected regions linked by a thin bridge are treated as further apart than their graph distance suggests).
Diffusion maps have been influential in single-cell biology, where they handle branching developmental trajectories gracefully, and in analysis of dynamical systems, where the random walk is a natural model of how the system transitions between states. They are also one of the theoretical linchpins for understanding the broader family of spectral methods, because the diffusion operator's eigenvectors are exactly the embedding coordinates.
All four methods take distances as their primary object and try to reproduce them in two or three dimensions. Their family resemblance is strong: each one is essentially MDS applied to a cleverer distance matrix. The progression, from Euclidean to weighted-Euclidean to geodesic to diffusion, is a progression of increasingly realistic definitions of what "close" means.
What the family shares, for better and worse, is a commitment to distance as the governing concept. When distance is a meaningful quantity (spatial data, feature vectors with calibrated scales, perceptual studies), this family is strong. When distance is not meaningful, or when the interesting structure is in who is near whom rather than how far apart, the distance-based methods start to struggle. The next article is about methods that give up on preserving distances and focus on preserving neighbor lists instead. That shift, from distances to neighborhoods, is the single biggest idea in the modern dimensionality reduction landscape.