RAG Common Pitfalls

Advanced RAG Engineering · 2/13

Module 1: Failure-Oriented Architecture

RAG Common Pitfalls

Learning objectives

  • Explain the core mental model behind RAG Common Pitfalls
  • Apply RAG Common Pitfalls within Failure-Oriented Architecture
  • Identify important boundaries, trade-offs, and failure modes
  • Produce concrete evidence from the practice exercise

Related: RAG Core Concepts | RAG Chunking Strategies | RAG Evaluation Metrics | RAG Index

Bad Chunking

Splitting mid-sentence or mid-table destroys meaning.

Fix: Use RecursiveCharacterTextSplitter with appropriate chunk sizes.

Wrong Embedding Model

Using a model trained on English for Chinese documents.

Fix: Match the embedding model to your language and domain.

Too Many Chunks in Context

Stuffing 20 chunks into the prompt dilutes attention.

Fix: Rerank to top-5 or use MMR.

No Source Attribution

Users cannot verify answers.

Fix: Include metadata in the prompt and cite sources.

Stale Index

Documents updated but index not refreshed.

Fix: Incremental indexing with SQLRecordManager.

Related

  • RAG Core Concepts — fundamentals
  • RAG Evaluation Metrics — detecting pitfalls
  • RAG Index — full topic map

Practice lab

Implement the smallest measurable RAG experiment for RAG Common Pitfalls. Keep the corpus and query set fixed, change one variable, and compare retrieval evidence and answer quality before and after. Add an operational constraint such as concurrency, recovery, security, latency, or cost, and defend the resulting design trade-off.

Review questions

  1. What problem does RAG Common Pitfalls solve, and what assumptions does it rely on?
  2. Which boundary or failure case is easiest to miss, and how would you expose it?
  3. What alternative design would you consider, and what trade-off would change the decision?
  4. What artifact, trace, test, or metric proves that your implementation is correct?

Completion evidence

  • A working artifact, annotated trace, or reproducible experiment
  • At least one normal case and one deliberately failing or boundary case
  • A concise explanation of the design choice and its trade-offs
  • Saved output showing how correctness was evaluated