Vector Search, Chunking, and Grounded Answers

What This Is

This lesson focuses on the pre-generation layer of RAG: how documents are split, embedded, retrieved, and used to support grounded answers.

Why This Matters in Production

If retrieval quality is poor, the generator is forced to hallucinate or overfit to irrelevant snippets. Most “RAG is bad” complaints are actually retrieval design failures.

Mental Model

Generation quality is downstream of retrieval quality. Retrieval quality is downstream of document structure, chunking strategy, metadata discipline, and ranking logic.

Deep Dive

Chunking is not a mechanical preprocessing step. It determines what semantic unit is even retrievable. Too small and you lose context. Too large and you dilute relevance. Metadata matters because filters often decide whether a result is even eligible. Grounded answers matter because the user should be able to trace claims back to source fragments instead of trusting the model’s confidence tone.

Worked Example

A security policy document chunked by arbitrary character count may split the exception clause from the rule. The retriever finds half the truth, and the answer becomes misleading even if the model is obedient.

Common Failure Modes

Common failures include naive chunking, no source attribution, retrieving top-k blindly, and never measuring whether relevant chunks actually appear in the candidate set.

article

Chunking Strategies

Useful practical framing of chunking tradeoffs.

official-doc

Weaviate RAG Concepts

Helps connect retrieval to generation interfaces.

official-doc

Evaluate Retrieval

Tie retrieval design to later eval discipline.