Hybrid Search and Reranking: Two-Path Retrieval, Reciprocal Rank Fusion, Then Re-Ranking the Top Results With a Cross-Encoder
Keyword search and vector search each have their own blind spots; combining both paths with Reciprocal Rank Fusion is often an instant win. Then use a cross-encoder to precisely re-rank the top few dozen results, and use day eight's evaluation to prove how much each step actually contributed.
Today's Goals
- Explain the blind spots of keyword search and vector search respectively, and give a query only one of them can hit
- Implement Reciprocal Rank Fusion and explain why it's more stable than weighting by raw score
- Explain the structural difference between bi-encoders and cross-encoders, and why reranking can only be applied to the top few dozen results
Yesterday you fitted week one's system with a scale and measured 87.5% recall and an already-saturated mean reciprocal rank. Today starts weighing things with it: fuse two retrieval paths, add a reranking layer, and measure tier by tier what each is worth. Come back and tick off the three goals.
Plain-Language Walkthrough
Two detectives: one works the literal clues, one works the motive
A case goes to two detectives. The first recognizes only physical evidence: fingerprints at the scene, shoe prints, a receipt with a name on it, locking on when one character matches and looking away when it does not. The second ignores physical evidence and studies motive: who has been short of money lately, who had a grievance with the deceased, whose account keeps changing — they cannot name one piece of hard evidence pointing at the killer and they can circle a set of that kind of person.
Keyword search is the first detective. Ask "how many unused annual leave days can carry into next year at most" and it goes to the inverted index for those terms, ranking by who contains more of them and whose terms are rarer. The formula and its two correction terms were taken apart on day 1 and are not repeated today. Its strength is precision: say a name and it finds that name and never returns a different one. Its blind spot is right there — rephrase and it goes blind.
Vector search is the second detective. It compresses query and document each into a string of numbers and compares direction rather than characters. "Passphrase" and "password" sit close in vector space even sharing no characters. Its blind spot is imprecision: ask for a person by name and it may hand you a batch of "the team lead" passages, because the name of the lead is genuinely close in meaning.
This is not which is better but that the two are good at different cases. I verified one pair on this day's thirty-document corpus (chunk-level retrieval, 134 chunks, MOCK=1 hash vectors):
- Asked how long troubleshooting records are kept before being purged, with the answer in
doc-022(log retention), the keyword path ranks it 3rd and the vector path does not have it in the top 10 at all. - Asked whether an employee's old passphrase still works after switching to the company's unified identity, with the answer in
doc-002(single sign-on and password policy), the keyword path has not one entry in the top 10 — because "passphrase" never appears in the corpus while "unified identity" and "employee" are too common to discriminate — and the vector path ranks it 3rd.
Same corpus, same indexes, and two queries each stumped one detective. So do not let them work alone — the question is how two people's conclusions get onto one sheet of paper.
Interview questions
Why do hybrid retrieval systems usually use reciprocal rank fusion instead of normalizing both scores and adding them with weights? When does the weighted approach break down?混合检索为什么普遍用倒数排名融合,而不是把两路分数归一化之后加权相加?加权那条路在什么情况下会失控?
Common in ChinaCommon overseasIntermediate#hybrid-search#rank-fusion🔒 Unlock this chapter to see the answer keys.
Why is a cross-encoder more accurate than a bi-encoder? And if it is more accurate, why not just use it to search the whole corpus directly?交叉编码器为什么比双编码器准?既然更准,为什么不干脆拿它直接检索全库?
Common in ChinaCommon overseasBasic#cross-encoder#bi-encoder🔒 Unlock this chapter to see the answer keys.
You replaced pure vector retrieval with hybrid search plus reranking, and after shipping it your eval metrics went down. How do you investigate?你把纯向量检索换成了混合检索加重排,上线之后评估指标反而掉了。你会怎么排查?
Common in ChinaCommon overseasDeep dive#hybrid-search#evaluation🔒 Unlock this chapter to see the answer keys.
Adding a reranker costs you 200 ms of extra latency per question plus a per-search fee. How do you decide whether that spend is worth it?加上重排之后每次提问多了两百毫秒延迟,还多了一笔按次计费的开销。你怎么判断这笔钱该不该付?
Common in ChinaCommon overseasDeep dive#rerank#cost-tradeoff🔒 Unlock this chapter to see the answer keys.
