Metadata Filtering

Scenario

A user asks: 'What is the status of the Acme project?' Your system retrieves chunks about the Alpha project — because both projects use similar words like 'Q3 milestones' and 'budget'. Before you read on, predict: can you fix this by changing the system prompt?

Think about whether the LLM can reliably enforce boundaries — then read on to see why the fix must happen at the database level.

Mental model

Filter by exact rules first, then search by meaning second.

Vector search only understands semantic closeness — it has no concept of ownership, dates, or categories. When you need strict rules like 'only documents from 2024' or 'only this user's documents', semantic fuzziness becomes a bug. Metadata filtering combines a hard database rule with vector similarity, so the semantic search only runs over documents that are allowed.

Deep dive

Vector search is fuzzy by design. If you search for 'red shirt', it might retrieve 'pink shirt' because the meanings are close in vector space. But if a user selects `category = red` from a filter button, returning pink is a logic error. You need a way to enforce hard boundaries.

Semantic search cannot natively enforce strict boolean rules.

The fix is to attach structured tags — called metadata — to each text chunk when you store it. Tags like `author`, `date`, `category`, or `tenant_id` are stored alongside the vector. At query time, you apply a filter on these tags.

Store structured tags alongside every embedded chunk at ingest time.

When you query, the database applies the metadata filter first. Only chunks that pass the filter are considered for vector similarity. Chunks that fail the filter are ignored completely, no matter how semantically close they are.

Apply the strict filter, then run vector search only on the remaining chunks.

Where the filter runs matters. **Pre-filtering** applies the rule inside the database before the vector search, so you always get back the full number of results you asked for. **Post-filtering** fetches the top results first and then removes the invalid ones in your application code — which is dangerous. If all top results fail the filter, you end up with zero results and the LLM has nothing to work with.

Pre-filter inside the database. Post-filtering in application code risks returning zero valid chunks.

**In multi-tenant apps, metadata filtering on `tenant_id` is a hard security requirement.** Without it, a user's question can retrieve private documents belonging to a different user, because vector similarity has no concept of ownership. A system prompt saying 'only answer about this user' is not a reliable substitute — the LLM will eventually ignore it.

Metadata filtering is the correct way to enforce data boundaries in a multi-tenant RAG system.

Code examples

pgvector: strict SQL filter + semantic search in one query

SELECT chunk_text
FROM documents
WHERE tenant_id = 42           -- 1. Hard metadata filter
  AND created_at > '2024-01-01'
ORDER BY embedding <=> :qvec   -- 2. Semantic vector search
LIMIT 5;

The SQL `WHERE` clause enforces the hard boundaries. The `ORDER BY` clause runs the semantic search only on rows that passed the filter. Postgres handles the optimization. This is the pre-filtering pattern.

Pinecone: metadata filter in the query payload

response = index.query(
    vector=query_vec,
    top_k=5,
    filter={                     # Hard metadata filter
        'tenant_id': {'$eq': 42},
        'year': {'$gte': 2024}
    }
)

Dedicated vector databases use JSON-style filter objects. The database applies this filter during the ANN index traversal, so you always get exactly `top_k` valid results back.

Common mistakes

• Relying on the LLM for access control: Adding a system prompt like 'Only answer using Acme documents' does not stop the model from reading unauthorized context that was already injected. Access control must happen at the database level via hard metadata filters before the LLM ever sees the chunks.
• Post-filtering in application code: Fetching the top 10 results and then filtering them in a Python loop is dangerous. If none of the top 10 pass your filter, you pass zero context to the LLM. Push the filter into the database query so the database keeps searching until it finds enough valid results.

Glossary

metadata filter

A strict rule that limits which documents the vector search is allowed to look at — for example, 'only documents from 2024' or 'only documents for this user'.

pre-filtering

Applying the strict filter inside the database before the vector search runs, so you always get the full number of valid results back.

post-filtering

Fetching results from the vector search first, then removing invalid ones in your application code — a risky approach that can leave you with zero results.

Recall questions

• Explain what metadata filtering does in a vector database, as if to a classmate.
• What is the difference between pre-filtering and post-filtering, and which is safer?
• Why is metadata filtering on tenant_id a security requirement in multi-tenant RAG?
• What happens if you retrieve the top 5 semantic results and then filter them in Python — and all 5 fail the filter?

Questions & answers

A user in your SaaS application asks a question and the RAG bot replies with confidential data belonging to a completely different customer. How did this happen and how do you fix it?

The system ran a global vector search without checking document ownership. The other customer's vectors were semantically closest to the query, so they were retrieved and shown to the wrong user. Fix: add a hard metadata filter on `tenant_id` to the database query so the semantic search only runs on the current user's documents.

Approach: Recognize that vector similarity has no concept of permissions. Identify metadata filtering as the correct architectural fix for multi-tenant data isolation.

You retrieve the top 5 semantic results from your vector database. In Python, you filter out any documents older than 2024. Users report the bot often says 'I don't have enough context'. What is the bug?

You are post-filtering. If all 5 semantically closest results happen to be from before 2024, your Python filter removes them all, leaving zero chunks for the LLM. Push the date filter into the database query as a pre-filter so the database keeps searching until it finds 5 valid results.

Approach: Identify post-filtering as the cause of result starvation. Advocate for pushing filters into the database query tier.

Continue learning

Previous: Embed & Retrieve Top-k

Return to AI Engineering Roadmap