Learn · RAG: chunking, embeddings & vector search · CCAR-F

The RAG pipeline

Do you even need RAG? If you do: ingest, chunk, contextualise, embed, index, then retrieve, rerank, generate and evaluate.

Key ideas

• RAG finds the few relevant passages in a large collection and gives only those to Claude with the question.
• Under about 200K tokens (roughly 500 pages), skip RAG: put the corpus in the prompt and use prompt caching.
• Offline steps: ingest, chunk, contextualise (optional, recommended), embed, index vectors plus BM25.
• Online steps: retrieve with vectors and BM25, rerank about 150 candidates to 20, generate with citations.
• Evaluate retrieval recall and answer faithfulness separately on a test set; most failures are retrieval failures.

Worked example: The nine steps

A checklist you can say out loud in an interview. Steps 1-5 run when documents arrive; steps 6-8 run per question; step 9 runs whenever you change anything.

Exam traps

• Building a vector database for a corpus that fits in the context window with prompt caching.
• Thinking RAG is only "embed and search": chunking, reranking and evaluation decide most of the quality.
• Evaluating only final answers, so you can't tell retrieval failures from generation failures.

Official docs

• Introducing Contextual Retrieval
• Embeddings
• RAG recipe (cookbook)

Practise CCAR-F questions · Next lesson: Chunking strategies