Agentic RAG: Turning Retrieval Into a Tool So the Model Decides Whether to Search, How Many Times, and Whether to Start Over
A single fixed retrieval pass can't handle multi-hop questions or retrieval failures. Today wrap retrieval as a tool inside an agent loop, implement self-reflection and result correction, handle multi-hop queries, and set a call-count and budget cap on the loop so it doesn't keep searching forever.
Today's Goals
- Wrap retrieval as a tool inside an agent loop, and design a tool description that tells the model when it should be used
- Implement self-evaluation and correction of retrieval results: rewrite the query and retry when the material turns out irrelevant, with a retry cap
- Explain how much extra latency and how many extra calls Agentic retrieval costs over a fixed pipeline, and in which scenarios that cost isn't worth paying
For eleven days you built a pipeline: a question goes in, retrieval, reranking, assembly, and generation happen, and an answer comes out. Today bends that straight line into a loop. Come back and tick off the three goals.
Plain-Language Walkthrough
The researcher, versus the research procedure
There are two ways to find material in a library.
The first is a procedure: you hand the topic to the front desk, the desk searches the catalog, fetches five books onto your table, and walks away. Whether those five are right or enough is not the procedure's concern.
The second is a person: a practiced researcher takes the topic, searches once on the plainest keywords, opens a book and finds it is about the neighboring subject, and searches again with different wording; opening the next, the book says "this requires the platform team lead's approval" while the topic asks what that lead's name is — the book has no name and it gave a clue, so they return to the catalog with "platform team lead" and search a second round.
Eleven days built the first kind. It performs well on most questions, because most questions are answered by one search. And it has a structural flaw: it has no try again action. It does not know when it searched wrongly, nor when it searched incompletely.
Agentic retrieval fills that gap. The name is intimidating and peels down to one sentence: turn retrieval from a station on the pipeline into a tool the model can call repeatedly, and have the model judge after each call whether that is enough. Enough and it stops; not enough and it searches again with different wording.
The cost is equally plain: a researcher costs more than a front desk, and each extra round is one more retrieval, one more model call, and more latency. So today's difficulty is not writing the loop — that code is under a hundred lines — but three questions: how the manual is written so the model neither skips searching when it should nor searches wildly when it should not; how the "is that enough" judgment avoids being the model congratulating itself; and most importantly, how it is stopped from searching forever.
Interview questions
You are exposing retrieval to a model as a tool. How do you write the tool description, and what concrete failure modes appear when you write it badly?把检索包成一个工具交给模型,这个工具的描述该怎么写?写不好会导致哪些具体的错误行为?
Common in ChinaCommon overseasBasic#tool-design#agentic-rag#prompting🔒 Unlock this chapter to see the answer keys.
Self-reflective retrieval rewrites the query and retries. How do you guarantee it terminates instead of spinning on the same query forever?自反思式检索会反复改写查询重试。你怎么保证它一定会停下来,而不是在同一个查询上原地打转?
Common in ChinaCommon overseasIntermediate#agentic-rag#self-reflection#reliability🔒 Unlock this chapter to see the answer keys.
In multi-hop retrieval a wrong first hop poisons every hop after it. How would you design for that?多跳检索里第一跳查错了,后面全跟着错。你会怎么设计容错?
Common in ChinaCommon overseasDeep dive#multi-hop#error-propagation#agentic-rag🔒 Unlock this chapter to see the answer keys.
When would you refuse to make a RAG system agentic, and what data would you use to convince your team?什么情况下你会拒绝把一个 RAG 系统做成 Agentic 的?拿什么数据说服你的团队?
Common in ChinaCommon overseasIntermediate#agentic-rag#cost#engineering-judgement🔒 Unlock this chapter to see the answer keys.
