Dayward AIWeChat Mini Program
From Frontend Engineer to Agent Engineer in 30 Days
Week 4 · D24🔒 LockedAbout 6 hours

RAG, Level Up: Hybrid Search, Reranking, Citations, Recall Evaluation

Upgrade day twelve's memory retrieval into a hybrid-search-plus-rerank combination, add source citations, and evaluate retrieval quality with a recall metric.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Implement hybrid search combining keyword search and vector search
  2. Add a reranking step to the retrieval result to improve ranking quality
  3. Design a recall evaluation method that measures whether retrieval recalled what it should have

Yesterday made integrating a capability cheap, so you will naturally wire in a pile of knowledge bases and retrieval services. Today goes back to fix upstream — because however much you wire in, it cannot rescue a retrieval whose recall is low to begin with.

Plain-Language Walkthrough

Those two rounds of resume screening are exactly retrieval's two rounds

A company hiring one person receives eight hundred resumes. HR does not read each in detail — that would take two weeks. The real process is two rounds.

The first round is broad and high-volume, running two channels at once. One is a keyword sift: search the system for "Kubernetes" and whatever hits enters the pool. It is precise, with one fatal flaw — a candidate who wrote it as "K8s" is missed entirely. The other is a recruiter's referral: they have met this person and it "feels right," without being able to name which word matched. They understand synonyms and context, and asked to find somebody who has used model SF-3000 they will hand you a pile of people who worked on similar models while missing the one who actually wrote SF-3000.

Each channel has its own blind spot, and the blind spots do not overlap — so both channels' people go into the second round together. The second round is a technical interviewer reading each in detail, twenty minutes apiece. Precise, and expensive, so only a few candidates get it.

Retrieval is exactly those two rounds. Keyword search (the BM25 road) is HR's search box: literal matches, superb on model numbers, error codes, and order numbers, and helpless with synonyms. Vector search is the recruiter: it compresses text into numbers and compares semantic distance, understanding that "shipping fee" and "postage" are the same thing, while for a semantically empty string like E4032 it can only hand you a pile of things that look like identifiers. Reranking is that technical interviewer: take a few dozen candidates, compare each against the question in detail, and reorder.

The analogy's most valuable point comes last: recall is whether the genuinely suitable people made it into the first round. Whoever the first round missed cannot be rescued by however good the second is — an interviewer can only order the list handed to them, not conjure somebody who never entered the pool.

So the chain's shape is fixed: broad upstream so downstream can tighten. Take 20 from each road, fuse, and after reranking keep only 5 for the context.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • Why isn't pure vector search enough — what does keyword search add?为什么单纯的向量检索不够,还要加一路关键词检索?
    Common in ChinaCommon overseasBasic#rag#hybrid-search#retrieval

    🔒 Unlock this chapter to see the answer keys.

  • How do you merge two retrieval rankings, and why not just take a weighted sum of the scores?两路检索结果怎么合并?为什么不能直接加权求和?
    Common in ChinaCommon overseasIntermediate#rag#rrf#ranking

    🔒 Unlock this chapter to see the answer keys.

  • How is reranking usually implemented, what problem does it solve, and what does it cost?重排(rerank)一般怎么实现?它解决了初步检索的什么问题,代价是什么?
    Common in ChinaCommon overseasIntermediate#rag#rerank#latency

    🔒 Unlock this chapter to see the answer keys.

  • How do you evaluate retrieval quality in a RAG system, and how should the evaluation set be built?怎么评估一个 RAG 系统的检索效果?评估集应该怎么构造?
    Common in ChinaCommon overseasDeep dive#rag#evaluation#recall

    🔒 Unlock this chapter to see the answer keys.

Comments