Learn · RAG: chunking, embeddings & vector search · CCAR-F

Contextual retrieval and reranking

Prepend a short Claude-written context to every chunk, index it twice, then rerank 150 to 20. Anthropic measured 67% fewer failures.

Key ideas

Chunks lose context ("the limit is ₹7,500": which limit?); contextual retrieval adds it back before indexing.
Claude writes a short context for each chunk from the whole document; it is prepended for both the embedding index and BM25.
Prompt caching keeps it cheap: about $1.02 per million document tokens in Anthropic's estimate.
Reported top-20 failure rate: 5.7% baseline, 3.7% contextual embeddings, 2.9% with contextual BM25, 1.9% with reranking.
Rerank: retrieve about 150 candidates, score them with a reranker, keep the top 20; top 20 beat 5 or 10 in the tests.

The added line is what Claude generates. It makes the chunk findable by both meaning and keywords.

Adding context only to the embedding index and forgetting the BM25 index.
Generating contexts without prompt caching, paying to resend the whole document for every chunk.
Sending all ~150 retrieved candidates to Claude instead of reranking to the best 20.
Assuming fewer chunks is always better; in Anthropic's tests the top 20 beat the top 5 or 10.