Text outlier detection
Detect unusual text records by comparing their vectors with vectors from normal records.
AI & ML · Const
Anomaly Detection
Anomaly detection can use embedding vectors to represent data and identify items that differ from a reference set. OpenAI documents embeddings as useful for anomaly detection, but not a dedicated anomaly-detection API.
How it works
Convert comparable data items into embedding vectors and identify items whose vectors are unusually distant from a reference group. The documentation names anomaly detection as an embedding use case, but does not prescribe a specific detection algorithm or threshold.
Syntax
client.embeddings.create({ model, input })Returns
An embedding vector produced from a text string.
Key points
- An embedding represents an individual data item as a vector.
- The vector is designed to retain relevant content or semantic information.
- Similarity is commonly reflected by smaller separation between embedding vectors.
- Unrelated items tend to be represented farther apart than similar items.
- OpenAI provides models that transform text strings into embedding vectors.
- Anomaly detection is one of the documented downstream applications.
Common mistakes
- The page identifies anomaly detection as a use case for embeddings, not as a complete built-in anomaly-detection API; distance calculation and thresholding remain application responsibilities.
Example: Generate embeddings for anomaly detection
from openai import OpenAI
client = OpenAI()
response = client.embeddings.create(
model="text-embedding-3-small",
input=["normal event", "unusual event"]
)
vectors = [item.embedding for item in response.data]
print(vectors)Generate embedding vectors that can then be compared with a distance metric or used by an anomaly-detection system.
When to use it
Semantic monitoring
Flag semantically atypical support tickets, logs, or events within a collection.
Interview questions
QWhat is an embedding?
An embedding is a vector representation of data intended to preserve aspects of its content or meaning. Items that are similar tend to have embeddings that are closer together than unrelated items.
Follow-up: How could those vectors support anomaly detection? Compare a new vector with vectors representing normal behavior and investigate sufficiently distant results; the documentation does not define the distance metric or cutoff.
QHow can text embeddings be used for anomaly detection?
OpenAI text embedding models accept a text string as input and return an embedding vector. Those vectors provide the representation used by downstream applications such as anomaly detection.
Follow-up: What must be decided before production use? Define the reference population, similarity or distance method, and threshold using representative data, because the page does not supply those policy choices.
QWhat other uses does the documentation list for embeddings?
Embeddings are also useful for search, clustering, recommendations, and classification. These are application patterns built on the vector representation rather than separate anomaly-detection operations documented on the page.
Follow-up: Why might one embedding workflow support several tasks? The same semantic vector representation can be compared or grouped differently depending on the downstream objective.
Related
searchclusteringrecommendationsclassificationtext embedding models