Dayward AIWeChat Mini Program
RAG in 14 Days: From Retrieval to Trustworthy Answers
Week 2 · D13🔒 LockedAbout 5 hours

Going to Production: Incremental Sync and Deduplication, Permission-Based Filtering, Cache Layering, Tracing, and the Cost-Latency Ledger

One stretch remains before handing the system to real users: documents change, people have different permissions, repeated questions shouldn't be recomputed every time, and when something breaks you need to find which step was slow and expensive. Today finish all four in one pass.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Design a content-fingerprint-based incremental sync scheme that only recomputes the chunks affected by a document change
  2. Build access control into retrieval itself, and explain why filtering only at generation time is wrong
  3. Add tracing and cost metering to the whole pipeline, and locate where latency and spend each concentrate

Twelve days made this system steadily more accurate while it lived inside three assumptions: the corpus is static, everyone sees the same copy, and every question is computed from scratch. Today all three come down. Once you have read this and finished the lab, scroll back to the top and tick off the three goals.

Plain-Language Walkthrough

The collection changes, and recataloguing is too expensive

A library's card catalog is not finished when it is finished. Every week brings new books, lost books removed, and revised editions replacing old ones. If every change meant recataloguing the whole building, the librarian would do nothing else for a career. What actually happens is a stocktake: take a current list of holdings, reconcile it card by card against the catalog, and touch only the cards that disagree.

An index is the same. Your corpus comes from a drive, a ticketing system, an internal wiki, and they change daily. Rebuilding everything nightly is perfectly fine at thirty documents and becomes a large daily payment at thirty thousand — vectorization is billed by token, and a full rebuild buys the entire knowledge base again every day. Worse, during the rebuild the index is half new and half old, and a question asked then gets an unreproducible result.

Reconciliation handles three kinds of change, none omissible: present in the source and absent from the index is an addition; present in both with differing content is a modification; present in the index and gone from the source is a deletion. Written out it is a dozen lines, and the key is that all three branches must be there:

sync.js
// Three-way reconciliation: compare the source's full set against the index's, touching only disagreements
export function planSync(indexed, incoming) {
  const plan = { added: [], modified: [], unchanged: [], deleted: [] }
  for (const doc of incoming) {
    const known = indexed.get(doc.id)
    if (!known) plan.added.push(doc.id)
    else if (known.contentHash !== contentHash(doc.raw)) plan.modified.push(doc.id)
    else plan.unchanged.push(doc.id)
  }
  // Deletion must iterate the index instead: a source never produces a "this one is gone" record
  const incomingIds = new Set(incoming.map((d) => d.id))
  for (const docId of indexed.keys()) {
    if (!incomingIds.has(docId)) plan.deleted.push(docId)
  }
  return plan
}

In today's lab that reconciliation recognizes one addition, one modification, and one deletion, and vectorizes only 6 chunks; a full rebuild vectorizes 151. Same corpus, a bill twenty-odd times apart.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • After a document changes, how do you recompute only the affected chunks? And how do you guarantee a deleted document really disappears from the index?文档更新之后,你怎么做到只重算受影响的块?被删掉的文档又怎么保证一定从索引里消失?
    Common in ChinaCommon overseasIntermediate#incremental-sync#content-hash#index-maintenance

    🔒 Unlock this chapter to see the answer keys.

  • Why can't access control be applied at the generation step? What exactly leaks if you put it there?为什么权限过滤不能放在生成阶段做?放在那里会泄露什么?
    Common in ChinaCommon overseasDeep dive#access-control#filter-pushdown#multi-tenancy

    🔒 Unlock this chapter to see the answer keys.

  • What can be cached in a RAG system, and what are the invalidation conditions for each?RAG 系统里有哪些东西可以缓存?各自的失效条件是什么?
    Common in ChinaCommon overseasIntermediate#caching#invalidation#cost-optimization

    🔒 Unlock this chapter to see the answer keys.

  • You need to switch embedding models. How do you migrate a live system without downtime and without losing recall?要换一个 embedding 模型,线上系统怎么迁移才能不停机也不掉召回?
    Common in ChinaCommon overseasDeep dive#embedding-migration#zero-downtime#rollout

    🔒 Unlock this chapter to see the answer keys.

Comments