Dayward AIWeChat Mini Program
RAG in 14 Days: From Retrieval to Trustworthy Answers
Week 2 · D11🔒 LockedAbout 5 hours

Advanced Indexing: Parent-Child Documents, Summary Indexes, Contextual Retrieval, and the Trade-Offs of Tree Aggregation vs. Graph Retrieval

The same set of documents can support several index structures. Today implement parent-child and summary indexes, land contextual retrieval — a technique with a very good cost-to-benefit ratio — then explain exactly what problems tree-based recursive aggregation and graph retrieval each solve, what they cost, and when not to use them.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Implement parent-child and summary indexes, and explain which category of question each makes answerable
  2. Land contextual retrieval: add contextual descriptions to each chunk, and work out the one-time cost of building the index
  3. Explain what problems tree-based recursive aggregation and graph retrieval each solve, and under what conditions their cost isn't worth it

Yesterday's effort went into what happens after a query arrives; today switches sides: the query stays and the index changes. Come back and tick off the three goals.

Plain-Language Walkthrough

One book, several catalogs

A thick book usually has more than one listing: contents by chapter at the front, a subject index at the back. Not one word of the book changed; what changed is what the cards are filed by. For what chapter seven covers, the contents are fastest; for every mention of the recycle bin, the subject index is.

Retrieval is the same. For ten days we built one index: cut documents into chunks, one entry each, and search that set. That works for a question like how many days the retention period is, and has almost nothing for another three kinds:

  • The hit chunk is too small and the model sees half a sentence (context completeness);
  • A chunk lost its belonging when cut, so alone it does not say whose retention period (chunk self-sufficiency);
  • The question wants a summary of the whole store rather than a few passages (global questions).

Today builds several more catalogs over the same documents so each of those three has somewhere to go. The index structure changes and the retriever needs not one line changed — the precondition for every comparison here.

Which raises the question: with three indexes built over one corpus, what decides which one a query takes?

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • Parent-child indexing and contextual retrieval both patch the same problem — chunks losing their context. What actually distinguishes them?父子索引和上下文检索都在补『块被切碎』这个问题,它们的差别到底在哪?
    Common in ChinaCommon overseasIntermediate#indexing#contextual-retrieval#chunking

    🔒 Unlock this chapter to see the answer keys.

  • Contextual retrieval needs one model call per chunk. How do you estimate that one-off cost, and what levers bring it down?上下文检索要给每个块调一次模型,这笔一次性成本怎么估?有哪些办法能压下来?
    Common in ChinaCommon overseasDeep dive#contextual-retrieval#prompt-caching#cost

    🔒 Unlock this chapter to see the answer keys.

  • What kind of question actually requires graph retrieval? Give one concrete case where it is justified and one where it is not.什么样的问题必须上图检索?给一个该上的具体例子和一个不该上的例子。
    Common in ChinaCommon overseasDeep dive#graph-rag#multi-hop#cost

    🔒 Unlock this chapter to see the answer keys.

  • You have built three different indexes over the same corpus. How do you decide which one a query goes to?同一份语料建了三套索引,检索时你怎么决定走哪一套?
    Common in ChinaCommon overseasIntermediate#index-routing#evaluation#architecture

    🔒 Unlock this chapter to see the answer keys.

Comments