Learn · RAG: chunking, embeddings & vector search · CCAR-F

Embeddings and vector search

Embeddings put similar meanings close together; vector databases find neighbours fast. Add keyword search for exact identifiers.

Key ideas

• An embedding is a vector (often 1,024 numbers) that captures meaning; similar passages land close together.
• Anthropic has no embedding model of its own; its docs use Voyage AI (e.g. voyage-4, voyage-finance-2) and advise assessing several vendors.
• Use input_type="document" for passages and "query" for questions. Voyage vectors are normalised, so dot product equals cosine similarity.
• Vector databases use approximate nearest neighbour indexes (HNSW, IVF) plus metadata filters; start with what you already run.
• Embeddings miss exact identifiers, so production systems add BM25 and merge with reciprocal rank fusion.
• Fewer dimensions or int8/binary quantisation cut storage 4-32× at some accuracy cost; changing models means re-indexing.

Worked example: Embedding and nearest-neighbour search

Passages and questions are embedded with different input_type values, then ranked by dot product. In production a vector database replaces the NumPy line.

Exam traps

• Omitting input_type, or embedding questions as documents.
• Relying on vector search alone for account codes, invoice numbers or section references.
• Comparing vectors from two different embedding models after a model upgrade.
• Spending the design effort on choosing a database when chunking, embeddings and reranking matter more.

Official docs

• Embeddings
• Introducing Contextual Retrieval

Practise CCAR-F questions · Next lesson: Contextual retrieval and reranking