Topic discovery
Organize a large text corpus into discovered themes.
AI & ML · Const
Clustering
Clustering groups text by similarity using semantically meaningful embedding vectors. It is an unsupervised way to uncover hidden groupings in a large text dataset.
How it works
Clustering applies an unsupervised grouping algorithm to embedding vectors so semantically similar text is assigned to the same group.
Returns
Cluster labels for the embedded text items; the page demonstrates storing them in a Cluster column.
Key points
- Use clustering to organize and interpret large collections of text.
- Generate an embedding for each text item before applying a clustering algorithm.
- Clusters may represent different themes or sentiment patterns, and multiple clusters can describe related aspects of the same dataset.
- K-means requires selecting the number of clusters explicitly in the documented example.
- Fit the clustering model to a matrix whose rows contain the embedding vectors.
Common mistakes
- Because clustering is unsupervised, choose and validate the number and meaning of groups rather than treating labels as predefined ground truth.
Example: Cluster embedding vectors
from sklearn.cluster import KMeans
import numpy as np
embeddings = np.array([
[1.0, 0.0], [0.9, 0.1],
[0.0, 1.0], [0.1, 0.9],
])
model = KMeans(n_clusters=2, init="k-means++", random_state=42)
model.fit(embeddings)
print(model.labels_)This runnable example groups embedding vectors into two clusters with K-means.
When to use it
Review analysis
Separate favorable and unfavorable review patterns.
Semantic segmentation
Identify groups of semantically related documents.
Interview questions
QHow would you cluster text with OpenAI embeddings?
OpenAI embeddings convert text into vectors that preserve aspects of content or meaning. A clustering algorithm can then group vectors according to similarity.
Follow-up: Why are embeddings useful before clustering? Similar text tends to produce vectors that are closer together than unrelated text.
QWhen is clustering appropriate?
Clustering is useful when the dataset has no predefined labels and the goal is to discover latent groups. The documentation presents textual data as the input and embedding vectors as the representation.
Follow-up: What does the output mean? Each item receives a cluster assignment, which should be interpreted by examining the text in each group.
QWhat are the main implementation steps?
A practical workflow is to create embeddings, arrange them into a matrix, select a cluster count, fit K-means, and store the resulting labels. The documentation example uses four clusters and k-means++ initialization.
Follow-up: What should be checked after fitting? Inspect whether the groups are coherent and whether the selected cluster count reflects the task.
Related
EmbeddingsSearchRecommendationsAnomaly detectionClassificationKMeans