Week 4: RAG, Context, and Agentic Systems
Why RAG Exists and When to Use It
RAG is a product design choice, not a mandatory AI ingredient.
Objective
Decide when to use plain LLM calls, classical ML, or RAG based on the problem shape.The lesson is public. The pressure loop lives inside the app where submissions, revision, and review happen.
Deliverable
A retrieval architecture brief and an agent threat model.Each lesson contributes to a week-level artifact and eventually to the shipped AI-native SaaS.
Preview
Public lesson preview.
Lesson Preview
Why RAG Exists and When to Use It
RAG is a product design choice, not a mandatory AI ingredient.
This lesson explains why retrieval-augmented generation exists and, more importantly, when it is the wrong answer.
RAG adds operational complexity. If you do not need external knowledge grounding, retrieval may just add latency, cost, and new failure modes.
Use RAG when the model needs fresh or proprietary context at inference time and the answer quality depends on retrieving the right evidence first.
What This Is
This lesson explains why retrieval-augmented generation exists and, more importantly, when it is the wrong answer.
Why This Matters in Production
RAG adds operational complexity. If you do not need external knowledge grounding, retrieval may just add latency, cost, and new failure modes.
Mental Model
Use RAG when the model needs fresh or proprietary context at inference time and the answer quality depends on retrieving the right evidence first.
Deep Dive
RAG solves a specific class of problem: the model lacks access to the right knowledge at the right time. It is not a cure for weak prompt design, poor product scope, or problems that are really structured prediction tasks. The mature question is not “can we add vector search?” It is “what failure does retrieval eliminate, and what new failure does it introduce?”
Worked Example
A support assistant answering over a living knowledge base benefits from RAG. A churn classifier over stable structured data does not. A simple FAQ page with ten fixed answers probably does not either.
Common Failure Modes
Typical failures include using RAG as a prestige layer, retrieving too much context, and ignoring whether the source material is trustworthy or current.
References
Further reading the machine expects you to use properly.
official-doc
RAG Conceptual Guide
Use this as a modern framing of the pattern.
Open referencearticle
Elastic RAG Overview
Helpful for search-oriented intuition.
Open referenceofficial-doc
LlamaIndex RAG Overview
Useful for comparing library mental models.
Open reference