Vector Embeddings

What are embeddings?

An embedding is a list of numbers (typically 256–3,072 dimensions) produced by a model, where semantic similarity becomes geometric closeness: “How do I reset my password?” and “forgot login credentials” end up near each other even though they share no words. Embeddings power semantic search, RAG, recommendations, clustering, deduplication, and classification.

How they’re used

1. Pass text through an embedding model → get a vector.
2. Store vectors in a vector database or a plain array.
3. At query time, embed the query and find nearest neighbors by cosine similarity.

When to use what

• Semantic search / RAG — embeddings are the default first stage.
• Classification with few labels — embedding + logistic regression often beats prompting an LLM, at ~1/1000 the cost.
• Exact lookups (SKUs, error codes, names) — keyword search wins; embeddings blur exact identifiers. Use hybrid search.

Cost reality

Embedding is cheap — fractions of a cent per million tokens — but re-embedding is the trap: switch embedding models and you must re-embed the entire corpus, because vectors from different models aren’t comparable. Budget for that migration before picking a niche provider. Dimension count also drives vector-DB RAM/storage cost roughly linearly.

What people get wrong

• Comparing vectors across models (or even model versions). Never mix.
• Embedding whole documents. Long text averages into mush; embed chunks or summaries.
• Ignoring the mismatch between queries and documents. Short queries vs long docs embed differently — models with query/document modes, or adding synthetic questions per chunk, close the gap.

Try it in code

Semantic similarity in ten lines — no vector database required:

import numpy as np

sentences = [
    "How do I reset my password?",
    "I forgot my login credentials",
    "What is your refund policy?",
]
vecs = [embed(s) for s in sentences]   # any embeddings API returns a list of floats

def cosine(a, b):
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

print(cosine(vecs[0], vecs[1]))  # high (~0.8): same meaning, zero shared words
print(cosine(vecs[0], vecs[2]))  # low  (~0.2): different topic

Under ~100k items, brute-force cosine over a NumPy array is exact and fast — reach for a vector database only after this stops being enough.