Retrieval engineering: why RAG hits or misses
You gave the model the documents, and it still cited the wrong one and missed an obvious fact. RAG was supposed to fix this.
The idea inside
Retrieval quality is engineered: chunking, hybrid (keyword + semantic) search, reranking, citations, and freshness each decide whether the right context even reaches the model.
After this lesson
You can diagnose why a RAG answer fails and fix it at the retrieval layer (chunking, hybrid search, reranking, citations, freshness) instead of blaming the model.
Where it leads
Retrieval now finds the right text. But answering is still passive, what about acting?
Inside this lesson
That's the real lesson stage, paused. Claim your pass to operate it.
See how AI actually works, end to end.
This lesson is one stop on the full arc. Unlock all of it, and keep it for life.
What you get
- The 34-lesson main path, a finishable route from a word to agents
- Goal tracks for using AI at work and building AI features
- Boss labs that make you apply a whole act, not just recognize it
- Spaced recall that brings each idea back before you forget
- Course memory: every term defined, with links to where it first appears
- A shareable capability card when you finish the main path
- Lifetime access on every device, every future lesson included
Not videos to watch. You predict, operate the machine, then prove it. That is why it stays.
99 interactive lessons and challenges. No videos, no code.
Free launch pass: lifetime access, no card needed
New here? The first lessons are free to try. Start with lesson 0.1
What this lesson shows
Retrieval quality is engineered: chunking, hybrid (keyword + semantic) search, reranking, citations, and freshness each decide whether the right context even reaches the model.
The question it opens with
You gave the model the documents, and it still cited the wrong one and missed an obvious fact. RAG was supposed to fix this.
The walkthrough, in the lesson's own words
- Tune the retrieval pipeline until the bot answers 30 days, grounded in the right doc.
- Semantic search nails meaning. Predict whether it can find an exact part number.
- When RAG misses, the model is rarely the problem. Retrieval is.
- Semantic alone ranks the general returns chunk first, it never sees the exact code. Keyword alone finds the code but can't tell the live policy from the archived one. Use Hybrid search, turn Reranking on to pull the exact match to the top, and turn the Freshness filter on to drop the stale 14-day duplicate.
- Scores here are illustrative, not a real ranking engine, but the failure modes they show are real.
- Semantic search is great at meaning. Will it reliably find a document by an exact part number like RX-2024?
- Embeddings capture meaning, so an exact token like RX-2024 gets blurred into nearby ideas. Exact tokens and IDs need keyword (lexical) matching. Hybrid search runs both, lexical to catch the literal code, semantic to catch the meaning, then merges the results.
- Chunking is the cut before search even runs. Too large and the answer sentence is buried in noise the embedding averages away. Too small and the answer is split across chunks, so no single one holds it.
- RAG just means giving the model the documents.
- Retrieval is engineered: chunking, hybrid (keyword + semantic) search, reranking, citations, and freshness decide whether the right context even reaches the model.
- A RAG support bot confidently cites the wrong document, or misses a fact that is plainly in the docs. Where is the bug, almost certainly?
- In retrieval, not the LLM. The model can only answer from what reached its context. Bad chunks, semantic-only search that misses exact IDs, no reranking, or a stale index means the right passage never showed up. Fix retrieval before you blame or swap the model.
- What if the question and the docs are in different languages?
- A common failure. Keyword search breaks first, a Spanish question won't match English docs word-for-word. Embeddings help, since a multilingual model puts 'return policy' and 'política de devoluciones' near each other, but only if the embedding model is genuinely multilingual. Cross-language retrieval is a real gap, so test it with bilingual documents, not just same-language ones.
- You can diagnose why a RAG system hits or misses by reading the retrieval pipeline: chunking, hybrid search, reranking, citations, and freshness, instead of assuming the model is at fault.
- Hybrid found the exact code, reranking pulled it to #1, and the freshness filter dropped the stale 14-day duplicate. The answer is now grounded in the right doc.
- The answer is wrong or ungrounded. Read the top result above, then change a control to fix what is reaching the bot.
- Too small: the answer sentence is split across two chunks, so neither one holds it whole.
- Too large: the answer is one line buried in a page of unrelated text the embedding averages away.
- Balanced: the answer sits whole inside one focused chunk, easy to retrieve and cite.
Key takeaway
You fixed a failing answer by tuning retrieval: hybrid search caught the exact code, reranking pulled the right chunk to the top, and a freshness filter dropped the stale one.
What you can do after this lesson
You can diagnose why a RAG answer fails and fix it at the retrieval layer (chunking, hybrid search, reranking, citations, freshness) instead of blaming the model.
Check yourself: In a RAG system, what most often decides whether the answer is right?
- Whether retrieval surfaced the right passage (chunking, search, reranking)(correct)
- How large the language model is
- The temperature setting
- How long the prompt is
In RAG, the answer is usually only as good as the retrieval. Chunking, hybrid search, and reranking decide whether the right passage even reaches the model; a bigger model can't fix a bad retrieve.
Where it leads: Retrieval now finds the right text. But answering is still passive, what about acting?
This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.