Embeddings

Roadmaps › AI Engineering

Scenario

A user asks your support bot: 'how do I recover my account?' Your docs have a paragraph titled 'Password Reset Steps'. No word matches — 'recover' vs 'reset', 'account' vs 'password'. Before you read further, predict: will a keyword search find this paragraph?

Think about it — then read on to understand how embeddings let computers find meaning without shared words.

Mental model

An embedding is a map coordinate for meaning. Every piece of text becomes a point in space, and texts that mean similar things sit close together.

An embedding model takes text and outputs a list of about 1,500 numbers — a vector. The model was trained so that text with similar meaning produces vectors pointing in nearly the same direction. 'Reset my password' and 'recover my account' end up very close in this space, even though they share no words. 'Today's invoice total' lands far away because it means something different. The vector is the meaning, encoded as geometry.

Deep dive

Picture a vast space — not 2D or 3D, but 1,536 dimensions (for `text-embedding-3-small`). Each dimension encodes some feature of meaning that no human ever labeled. The embedding model learned these dimensions from billions of text examples. It discovered that texts with similar meaning should land near each other, and different texts should land far apart.

Embedding space: each dimension is a learned axis of meaning. Similar text = nearby points.

Why not just search for keywords? Keywords break on synonyms ('buy' vs 'purchase'), typos, and paraphrases — all normal ways to say the same thing differently. Semantic search via embeddings does not care about specific words. It encodes meaning directly. Two sentences with zero shared words can be neighbors in embedding space if they mean the same thing.

Embeddings capture meaning, not words — so semantically similar text lands nearby even without shared keywords.

Calling the embedding API is simple: pass a string, receive a vector. Three things to remember. First: the model is a black box — you do not control what each dimension means. Second: the vector only has meaning when compared to other vectors from the SAME model. Third: vectors from different embedding models are incompatible — like GPS coordinates from two different map systems.

Vectors from different models cannot be compared. Always use the same model on both sides.

Two embedding models are widely used today. `text-embedding-3-small`: 1,536 dimensions, cheaper, fast — the default for most RAG apps. `text-embedding-3-large`: 3,072 dimensions, better quality for complex tasks, about 2× the cost. **You choose the model at ingest time and must use the same model at query time — forever — until you re-embed your entire corpus.**

Pick your embedding model at ingest time and commit. Changing later means re-embedding everything.

The deepest insight: the embedding model does not understand English. It is a pure number machine — it maps a token sequence to a float vector. All the 'understanding' lives in patterns learned from training data. The vector store holds billions of these vectors but understands even less — it only compares lists of numbers with fast math. Yet from pure arithmetic on learned patterns, something that looks like semantic understanding emerges.

Pipeline: Docs → Chunk → Embed → Store. The vector store does not understand text — it only compares numbers.

Code examples

Get an embedding vector for a piece of text

from openai import OpenAI
client = OpenAI()

def embed(text: str) -> list[float]:
    resp = client.embeddings.create(
        model='text-embedding-3-small',
        input=text,
    )
    return resp.data[0].embedding  # list of 1,536 floats

vec = embed('How do I recover my account?')
print(len(vec))   # 1536

The output is a plain Python list of floats. You cannot read meaning from the numbers directly — they are only meaningful when compared to other vectors from the same model. The dimensions encode semantic features the model learned from training data.

Measure semantic similarity with cosine similarity

import numpy as np

def cosine_similarity(a: list[float], b: list[float]) -> float:
    a, b = np.array(a), np.array(b)
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

# Same meaning → close to 1.0
v1 = embed('How do I reset my password?')
v2 = embed('Steps to recover your account')
v3 = embed('What is the capital of France?')

print(cosine_similarity(v1, v2))  # ~0.92  — semantically close
print(cosine_similarity(v1, v3))  # ~0.73  — semantically distant

Cosine similarity measures the angle between two vectors — 1.0 means identical direction (same meaning), 0 means perpendicular (unrelated). This is the operation the vector store does millions of times per second.

Embed a batch of texts efficiently — the ingest pattern

from openai import OpenAI
client = OpenAI()

chunks = [
    'Password reset steps for your account.',
    'How to view your invoice history.',
    'Getting started with the REST API.',
    # ... up to 2048 strings per call
]

resp = client.embeddings.create(
    model='text-embedding-3-small',
    input=chunks,  # pass a list — one API call for all chunks
)

# Pair each chunk with its vector
pairs = [(chunk, r.embedding) for chunk, r in zip(chunks, resp.data)]
print(f'Embedded {len(pairs)} chunks in one API call')

The API accepts a list of up to 2,048 inputs per call. Always batch at ingest — calling the API once per chunk is about 100× slower and wastes your rate-limit quota. The response preserves input order, so zipping chunks with `resp.data` is safe.

Key points

Common mistakes

Glossary

embedding
A list of numbers that acts like a map coordinate for the meaning of a piece of text. Texts with similar meaning get coordinates that are close together.
vector
The list of numbers that represents text. The word 'vector' and 'embedding' are used to mean the same thing in most AI contexts.
embedding model
An AI that reads text and converts it into a vector. It is trained so that similar-meaning texts produce vectors that are close to each other.

Recall questions

Questions & answers

You built a RAG system last year using text-embedding-3-small. Now you want to upgrade to text-embedding-3-large for better quality. What must you do beyond changing the query call?

Re-embed the entire document corpus with `text-embedding-3-large` and rebuild the vector index. Vectors from `3-small` (1,536 dimensions) and `3-large` (3,072 dimensions) are in different spaces — querying a 1,536-dim index with a 3,072-dim vector throws a hard error. There is no shortcut: you must re-embed everything.

Approach: Emphasize the same-model invariant. The migration cost — re-embedding the whole corpus — is the correct answer, not 'just change the query model'.

A user complains the support bot cannot find answers that clearly exist in the docs. You check the vector similarity scores — the best match for the user's query is 0.71. What are two likely root causes?

The similarity score is low, which points to problems upstream of the vector search. First possibility: model mismatch — the query was embedded with a different model than the chunks. Second: the embedding model quality is insufficient for the domain. Secondary causes include the docs not containing the answer, or chunking being too coarse.

Approach: Low similarity scores point upstream — to the embedding model — not to the vector search itself. Name the model-mismatch bug and model quality as the two main causes.

Continue learning

Previous: Tokens & tokenization

Next: Grounding with Retrieval

Next: Similarity Metrics

Next: Embed & Retrieve Top-k

Return to AI Engineering Roadmap