Learn · RAG: chunking, embeddings & vector search · CCAR-F

Chunking strategies

Chunking decides what a passage is. Split by structure, keep tables whole, carry the heading path and store metadata.

Key ideas

• Too small loses meaning; too large makes retrieval imprecise and prompts costly. Anthropic's article describes chunks of no more than a few hundred tokens.
• Fixed-size with 10-20% overlap is a quick baseline but cuts sentences and tables.
• Structure-aware chunking (headings, then paragraphs, then sentences) suits policies, manuals and reports.
• Never split a table; repeat the heading path in every chunk; keep numbers with their labels and periods.
• Store metadata (document, section, page, effective date, entity, version) to filter and cite.
• Parent-child chunking searches small chunks but returns the larger parent section.

Worked example: Structure-aware chunking in Python

Split on Markdown-style headings, track the heading path, then split long sections by size with overlap. Each chunk starts with its heading path and carries metadata.

Exam traps

• Fixed-size chunks that split tables or clauses mid-way.
• Chunks that hold a number without its label, unit or period.
• Dropping heading paths and metadata, so chunks can't be filtered, cited or understood on their own.
• Choosing a chunk size by intuition instead of measuring recall on test questions.

Official docs

• Introducing Contextual Retrieval
• RAG recipe (cookbook)

Practise CCAR-F questions · Next lesson: Embeddings and vector search