Module 1: RAG Mental Model
Why RAG Exists
Learning objectives
- Explain the core mental model behind Why RAG Exists
- Apply Why RAG Exists within RAG Mental Model
- Identify important boundaries, trade-offs, and failure modes
- Produce concrete evidence from the practice exercise
Related: RAG What Is RAG | RAG Core Concepts | RAG Architecture Overview | RAG Index
The Limits of Parametric Knowledge
LLMs store knowledge in their weights. This is called parametric knowledge. It has three fundamental limits:
Limit 1: Cutoff Date
Models know nothing after their training data cutoff. If your product launched last month, the model has never heard of it.
RAG solution: Index your docs and retrieve them at query time.
Limit 2: Hallucination
When uncertain, models confabulate. They generate plausible-sounding but false statements.
RAG solution: Ground answers in retrieved documents. The model is instructed to use only the provided context.
Limit 3: Proprietary Data
Models have never seen your internal wiki, your customer's support tickets, or your legal contracts.
RAG solution: Your data becomes the corpus.
What RAG Adds
| Capability | Plain LLM | RAG |
|---|---|---|
| Up-to-date facts | No | Yes (if corpus is current) |
| Source attribution | No | Yes (via metadata) |
| Proprietary knowledge | No | Yes |
| Reduced hallucination | Partial | Better |
| Cost (per query) | Low | Higher (retrieval + longer prompt) |
When Not to Use RAG
- General knowledge Q&A. "Who wrote Hamlet?" — the model already knows.
- Creative writing. RAG constrains the model to retrieved facts; creativity may suffer.
- Ultra-low-latency inference. Retrieval adds 50–500ms.
- Small, static knowledge bases. If you have 10 facts, fine-tuning or few-shot prompting may be simpler.
Related
- RAG What Is RAG — the working definition
- RAG Core Concepts — the vocabulary
- RAG Architecture Overview — the system design
- RAG Index — full topic map
Practice lab
Implement the smallest measurable RAG experiment for Why RAG Exists. Keep the corpus and query set fixed, change one variable, and compare retrieval evidence and answer quality before and after.
Review questions
- What problem does Why RAG Exists solve, and what assumptions does it rely on?
- Which boundary or failure case is easiest to miss, and how would you expose it?
- What alternative design would you consider, and what trade-off would change the decision?
- What artifact, trace, test, or metric proves that your implementation is correct?
Completion evidence
- A working artifact, annotated trace, or reproducible experiment
- At least one normal case and one deliberately failing or boundary case
- A concise explanation of the design choice and its trade-offs
- Saved output showing how correctness was evaluated