Capstone Project and Retrospective: A Multi-Tenant Enterprise Knowledge-Base Q&A, a RAG Decision Map, and an Interview Deep Dive
Turn thirteen days of work into a portfolio-ready project: a multi-tenant, cited, evaluation-dashboarded enterprise knowledge-base Q&A system. Then compress the whole course into one decision map that answers the questions interviewers most often follow up on.
Today's Goals
- Assemble the previous thirteen days' modules into one complete multi-tenant project with citations and an evaluation dashboard, and explain its architecture
- Draw a RAG decision map that, given a new requirement, points to which configuration to use and why
- Handle interview follow-up questions, explaining every technical choice using this course's data and trade-offs
No new theory today. Every one of the thirteen days left a part and a set of numbers, and today does three things only: assemble the parts, gather the numbers into a map, and turn that map into something you can say out loud in an interview. Come back and tick off the three goals.
Plain-Language Walkthrough
One: three criteria for a portfolio project
An unwelcome fact first: "I built a RAG project" carries nearly zero information in an interview. Everybody has built one. What separates people is the three follow-ups: how do you know it works, which decision would you change now, and if you could do only three more things, which three?
So a project fit for a portfolio is judged not by feature count but by these three:
One, it demonstrates. Within five minutes you must be able to feed in a document, ask a question, click a citation to see the original, and open a dashboard showing metrics. Without that sequence, your project is a README in somebody else's eyes. That is also why today's lab ends by asking you to write a demo script — the demo path is itself a design review, and where it stumbles is usually where the design is wrong.
Two, it explains its trade-offs. Every configuration entry needs a "because of this, therefore that," and that because is best measured by you. Thirteen days already gathered the raw material: why BM25 came before vectors, why reranking applies only to the top twenty, why the gate sits on the raw score rather than the fused one — each has a measurement behind it.
Three, it has reproducible data. Not "it works well" but "on this thirty-document corpus, these twenty questions, and a six-hundred-token budget: 93.8% recall, 75.0% on the multi-hop tier, 0.0% refusal rate on unanswerable questions." That last 0.0% especially must be written down — a project description reporting only good numbers is more suspicious than one reporting none.
The hardest of the three is the second. The raw material for trade-offs is already yours, and how do dozens of conclusions scattered across thirteen days become something portable, opened whenever a new requirement arrives? That takes a map.
Interview questions
You are handed a knowledge base of five million documents that must answer in about a second, with accuracy as the top priority. How would you design it?给你一个五百万文档、要求秒级响应、准确率优先的知识库场景,你会怎么设计这套系统?
Common in ChinaCommon overseasDeep dive#system-design#scaling#latency-budget🔒 Unlock this chapter to see the answer keys.
Looking back at the RAG project you built, which decision would you change now, and why?你做过的这个 RAG 项目里,哪个决定你现在会改?为什么?
Common in ChinaCommon overseasDeep dive#retrospective#evidence#chunking🔒 Unlock this chapter to see the answer keys.
How do you convince a non-technical stakeholder that your retrieval system actually got better?怎么向不懂技术的业务方证明你的检索系统真的变好了?
Common in ChinaCommon overseasBasic#evaluation#stakeholder-communication#abstention🔒 Unlock this chapter to see the answer keys.
Users report that your live RAG system 'answers inaccurately'. What is your triage order?RAG 系统上线后用户反馈「答得不准」,你的排查顺序是什么?
Common in ChinaCommon overseasIntermediate#debugging#failure-modes#observability🔒 Unlock this chapter to see the answer keys.
If you could only fund three changes to improve an existing RAG system, which three would you pick and why those three?如果预算只够做三件事来提升一个已有 RAG 系统的效果,你选哪三件?为什么是这三件?
Common in ChinaCommon overseasIntermediate#prioritization#evaluation#abstention🔒 Unlock this chapter to see the answer keys.
