Dayward AIWeChat Mini Program
RAG in 14 Days: From Retrieval to Trustworthy Answers
Week 2 · D10🔒 LockedAbout 5 hours

Query-Side Optimization: Rewriting, Hypothetical Document Embeddings, Multi-Query, Step-Back Prompting, and Intent Routing

A user's question is often short and vague, and a retrieval miss isn't always the index's fault. Today put the effort into what happens after the query comes in and before retrieval happens: rewriting, generating a hypothetical answer and retrieving on that, splitting into multiple sub-queries, stepping back to ask a more general question, and judging intent first to decide which path to take.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Explain what shortfall each of query rewriting, hypothetical document embeddings, multi-query, and step-back prompting patches
  2. Implement an intent router that sends chat, retrieval-needed questions, and multi-hop questions down different handling paths
  3. Prove the net benefit of each query-side technique with evaluation, and work out the extra latency and call count each one adds

Yesterday's effort all went into the retriever: two-path recall, fusion, reranking. Today switches ends — not one character of the retriever changes, only the sentence going into it. Come back and tick off the three goals.

Plain-Language Walkthrough

The patient says it hurts, and the doctor has to ask where

At a hospital you are not wheeled into the CT room on arrival. The doctor asks first: where does it hurt, when did it start, does pressing make it worse. That round of questioning produces not a diagnosis but an accurate chief complaint — "persistent dull pain in the lower right abdomen, two days, worse under pressure." With that sentence, the later imaging knows where to look.

In a retrieval-augmented generation system, the sentence a user types is the patient's opening "it hurts": short and vague. Everything of the first nine days — chunking, indexing, hybrid search, reranking — is equipment in the CT room. However good the equipment, an inaccurate complaint gets an image of the wrong place.

Today's work is adding a round of questioning after the query comes in and before retrieval happens. It has a collective name: query transformation. Four common techniques each patch one shortfall:

  • Query rewriting patches an incomplete statement: unresolved references, colloquialism, two questions packed into one sentence.
  • Hypothetical document embeddings patch a question and an answer not looking alike: the two were never in the same register.
  • Multi-query patches one phrasing not fetching everything: the documents may use a different word for the same thing.
  • Step-back prompting patches a question asked too specifically, so specifically that the corpus contains no such sentence.

Plus one more layer, intent routing: judge first whether this question needs retrieval at all.

One thing must be said in advance: today's product is not turning all five on but a default configuration. Each technique costs at least one extra model call and most cost several extra retrievals. So each one reports three bills along the way: how much the metric rose, how much latency rose, and how many extra calls. The scale was built on day eight.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • Why does HyDE (hypothetical document embeddings) work, and when does it steer retrieval in the wrong direction?假设文档嵌入(HyDE)为什么有效?它在什么情况下会把检索带偏?
    Common in ChinaCommon overseasIntermediate#hyde#query-transformation#retrieval-quality

    🔒 Unlock this chapter to see the answer keys.

  • How do you handle coreference in multi-turn RAG, and what is the classic failure when you skip it?多轮对话里怎么处理指代?不做指代消解最典型的翻车场景是什么?
    Common in ChinaCommon overseasBasic#coreference#multi-turn#query-rewriting

    🔒 Unlock this chapter to see the answer keys.

  • Query rewriting adds a model call per question and doubles end-to-end latency. How do you decide whether it is worth paying?上线查询改写之后每问多了一次模型调用,端到端延迟涨了一倍,你怎么判断这笔开销值不值?
    Common in ChinaCommon overseasDeep dive#cost-tradeoff#latency#query-rewriting

    🔒 Unlock this chapter to see the answer keys.

  • What happens when intent routing misclassifies, and how would you design the fallback?意图路由判错了会怎样?你会怎么设计兜底?
    Common in ChinaCommon overseasIntermediate#intent-routing#fallback#observability

    🔒 Unlock this chapter to see the answer keys.

Comments