Clustering

Why does clustering matter for enterprise AI?

Clustering surfaces patterns that are hard to find by hand, such as customer segments, anomalous transactions, or groups of similar documents, applications of the same unsupervised learning principle behind clustering itself. Reproducing a specific clustering result later, and showing which points landed where and why, is what turns a one-off analysis into something a business can rely on repeatedly.

In financial services, clustering is a standard technique inside anti-money-laundering (AML) transaction monitoring: grouping account behavior into clusters makes it possible to flag activity that doesn't match any established pattern. Because AML systems fall under Bank Secrecy Act compliance requirements, the detection logic built on top of the clustering step needs to be documented and auditable, not just accurate.

In life sciences, clustering identifies patient subgroups from clinical or genomic data, informing the stratified analyses a clinical study report needs to justify. The clustering step itself is exploratory, but the subgroup definitions it produces often need to hold up under the same scrutiny as any other analytical decision in a trial's documentation.

What are the main types of clustering?

• Hierarchical (connectivity-based) clustering: builds a tree of nested clusters from the bottom up or top down. Well suited to data with a natural hierarchy, like an org chart or a product catalog.
• Partitioning (centroid-based) clustering: assigns each point to the cluster with the nearest centroid, based on squared distance. K-Means is the most common example.
• Density-based clustering: groups points in dense regions and treats sparse regions as noise. Clusters can take any shape, and the cluster count doesn't need to be specified in advance.
• Distribution-based clustering: assumes each cluster follows a statistical distribution, such as Gaussian, and assigns points based on the likelihood they belong to that distribution.
• Fuzzy clustering: assigns each point a membership score across multiple clusters instead of a single hard label. Useful when group boundaries are genuinely ambiguous.
• Grid-based clustering: divides the data space into a grid of cells and forms clusters from dense cells, which keeps the method fast on large datasets.
• Model-based clustering: assumes the data was generated by a mixture of underlying probability models and fits each cluster to one of those models.
• Constraint-based clustering: incorporates rules supplied by the user, such as pairs of points that must or must not end up in the same cluster, to guide the result toward a known constraint.

Common clustering algorithms

• K-Means: best for spherical, evenly sized clusters when you already have a rough idea how many you want.
• Agglomerative hierarchical clustering: builds a full hierarchy of clusters, useful for exploring different cluster counts after the fact rather than choosing one upfront.
• BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies): built for very large datasets that don't fit comfortably in memory.
• DBSCAN (Density-Based Spatial Clustering of Applications with Noise): finds arbitrarily shaped clusters and explicitly flags noise instead of forcing every point into a cluster.
• Affinity propagation: picks the number of clusters automatically, based on how strongly points signal each other as representatives.
• Mini-batch K-Means: a faster, approximate version of K-Means for datasets too large to cluster in one pass.
• Mean shift clustering: finds clusters by locating the densest regions of the data directly, without assuming a cluster shape.
• Gaussian mixture models (GMM): assumes clusters come from overlapping distributions and gives each point a probability of belonging to each cluster rather than one hard assignment.

Industry use cases

Financial services Analysts reviewing a flagged cluster need to see which specific transactions and account attributes pulled it out of the normal pattern, which is why AML systems typically pair clustering with an explainability layer rather than shipping a bare cluster label to an investigator.

Life sciences A common application is identifying responder and non-responder subgroups within a clinical trial, where a treatment effect that looks weak across the full population can be concentrated in one cluster that an average-outcome analysis would otherwise miss.

Public sector Government agencies use clustering to group benefits claims, grant applications, or case records, surfacing activity that doesn't match typical claim patterns for fraud or improper-payment investigation, the same anomaly-detection logic AML monitoring uses in financial services. Because these programs are subject to public audit and oversight, the clustering logic behind a flagged case has to be explainable enough for an investigator to defend the decision on record.

How do you run a clustering analysis?

1. Choose a distance or similarity measure. Most algorithms need a way to quantify how alike two points are, commonly Euclidean distance for continuous features or cosine similarity for text and embeddings.
2. Scale the features. Distance-based methods like K-Means are sensitive to feature scale, so standardizing or normalizing inputs first prevents one feature from dominating the distance calculation.
3. Pick an algorithm and, if required, a cluster count. This is where model selection matters most: K-Means and GMM need the number of clusters specified up front, often chosen with the elbow method or silhouette score, while density-based and hierarchical methods don't require this.
4. Fit the model and inspect the result. Look at cluster sizes, check for one dominant cluster absorbing everything else, and visualize the result in two dimensions if the data allows it.
5. Validate the clusters against something external. Where possible, check whether the clusters correlate with a known outcome that wasn't used to form them. That's the strongest evidence the clusters are meaningful rather than an artifact of the algorithm.

A minimal K-Means example in scikit-learn, covering steps 2 through 4:

from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans

X_scaled = StandardScaler().fit_transform(X)
kmeans = KMeans(n_clusters=4, random_state=0, n_init="auto")
labels = kmeans.fit_predict(X_scaled)

FAQ

Is clustering supervised or unsupervised?

Clustering is unsupervised. It works on unlabeled data and infers groupings from the structure of the data itself, rather than learning to predict a labeled target.

What does a -1 label mean in DBSCAN output?

In scikit-learn's DBSCAN documentation, a label of -1 means the point didn't meet the minimum density required to join a cluster, so scikit-learn records it as noise rather than assigning it to a group.

How is clustering different from classification?

Clustering finds groups in data that has no labels. Classification predicts a known label for new data based on examples it was trained on. If the categories already exist and you have labeled examples, that's classification, not clustering.

How do you choose the number of clusters?

Some algorithms, like K-Means, require the number of clusters as an input, often chosen using methods like the elbow method or silhouette score. Density-based and hierarchical methods don't require this to be set in advance.

How is clustering different from factor analysis?

Clustering groups observations based on how similar they are to each other. Factor analysis groups variables based on how they move together, to find the latent factors behind them. If you're trying to group rows of data, that's clustering; if you're trying to explain why columns of data correlate, that's factor analysis.