Finding Structure

Clustering algorithms find groups in the data. Labels give those groups names. Both are choices, not facts.

By Ian Johnson

Your eye already clusters

Looking at the UMAP scatter plot, your eye naturally groups the dots into clumps. Some areas are dense, others sparse. You can already see structure. Clustering algorithms do this systematically: they assign every point to a group (or mark it as noise that doesn't belong to any group).

Here are the same 1,000 jokes, now colored by cluster. The algorithm we use is HDBSCAN, which finds clusters of varying density and size. Gray points are "noise": jokes that don't fit neatly into any group.

Figure 1. HDBSCAN clustering with min_cluster_size=5. Click a colored cluster in the legend to see its jokes. Gray dots are noise points.

Resolution is a choice

How many clusters should there be? This isn't a question with one right answer. Clustering algorithms have parameters that control the granularity. With HDBSCAN, the key parameter is min_cluster_size: the minimum number of points required to form a cluster.

A small value finds many fine-grained groups. A large value merges them into fewer, broader categories. Adjust the slider below and watch the clusters reorganize:

min_cluster_size:

Figure 2. The same jokes clustered at different resolutions. Fine settings find many small groups. Coarse settings find a few large ones. Both are valid ways to look at the data.

At the finest setting, you might find a cluster of just 5 jokes about bicycles. At the coarsest, those jokes get absorbed into a broader "transportation" group or dissolved into noise. Neither view is more correct. They answer different questions. Fine clustering helps you find specific themes. Coarse clustering gives you a high-level map of the territory.

Naming what you find

A cluster is just a set of indices until someone gives it a name. One approach is to sample a few items from each cluster and ask a language model to summarize what they have in common. The model reads 10-20 jokes from a cluster and proposes a label like "animal puns" or "jokes about family meals."

These labels are helpful but imperfect. A cluster might contain jokes that share a structural pattern (question-answer format) rather than a topic. The label the model generates depends on which samples it sees and how it interprets them. Different models produce different labels for the same cluster.

Clustering and labeling are tools for organizing your view of the data, not for discovering ground truth. The question isn't "what are the real groups?" but "what grouping is useful for what I'm trying to understand?"

Many valid decompositions

Everything in this pipeline is interchangeable. You can swap the embedding model (a different lens), the dimensionality reduction algorithm, the clustering method, the labeling approach. Each combination produces a different view of the same data.

This isn't a weakness. It reflects a genuine property of data: there are many valid ways to organize it. A library can be sorted by author, subject, date, or reading level. None is the "true" order. The same principle applies here. The value is in being able to try different decompositions quickly and see what each reveals.

So far, every tool we've used has been local to this dataset. The UMAP projection reshuffles when the data changes. The clusters depend on the projection. In the next two articles, we'll build global tools: a set of features and a map that work the same way on any text, because they're trained to match the lens itself.