Self-Verification: The End-to-End Gate, Premature Completion Claims and Automatic Intervention on Doom Loops

Interview questions

• An agent reports a task done - what evidence should your harness use to decide whether to believe it?Agent 报告任务完成，你的 harness 该用什么依据决定信不信？

  Common in ChinaCommon overseasIntermediate#self-verification#false-completion#gates

  How to reason about it · think before answering

  1. This question is about who owns the completion criterion. 'Have it double-check' and 'tell it in the prompt to run the tests first' both score zero, because the evidence still comes from the party under review. An inspector does not read the contractor's self-assessment. The shallow check used for the first four days has exactly this shape: the patch landed in the file, therefore it passed. It can only prove something was written, never that it is right.
  2. Start by dropping the word 'lying'. The model is not lying - it can only see its context, it has never run that service, so how would it know that what it wrote does not work. False completion is not a character flaw in the model, it is a direct consequence of the constraint that it only sees context. Which means any fix built on prompt wording, including 'please report honestly', is void from the start.
  3. Separate the three shapes of a false completion: only the shell was written (the route is registered, it returns 200, the side effect never happens), the tests were not run (it says they were), and the tests ran but the result was misread. All three slip through code review easily, and they share one property - only actually starting the service and sending a real request catches them.
  4. So the evidence has to be produced by the harness itself, which in practice means three gates in a fixed order: syntax, then it boots, then the cases. Syntax is the cheapest, so it runs first. The three fail for entirely different reasons, and in the report a human reads, 'the syntax is broken' and 'the behavior is wrong' are two different things. Collapsing them into one 'verification failed' leaves you with nowhere to start in the morning.
  5. One threshold people skip is worth volunteering: if not a single case covers this feature, it cannot count as passed. Absence of evidence is not a pass - a gate that returns success on an empty suite looks permanently green while verifying nothing. That is the most common entrance to a tautological gate.
  6. The measured numbers are worth memorizing because they run against intuition. Same code, same model: on the shallow-check run the checklist claimed 33 items, 24 actually worked, 9 were false. On the end-to-end run it claimed 20, 20 actually worked, 0 were false. The shallow numbers look better. In the morning you see the prettier progress first and only then discover the service misbehaves - with nothing on the checklist flagged as suspect.
  7. Expected follow-up: why re-run everything already passed instead of just this item? Because one class of defect passes its own case. F07 in this course forgets the guard for 'no tag supplied means pass the list through'; queried with a tag it filters perfectly, and what it breaks is everybody else - eight features that were built correctly, measured. A gate that only checks the current item is blind to it; the regression suite is what catches it.

  分析过程 · 先想清楚再作答

  1. 这题考的是**完成判据归谁**。答「让它再自查一遍」「提示词里要求它先跑测试」都拿不到分，因为这两条给出的证据仍然由被审查的一方生产。监理不看施工队的自我评价。前四天的浅检查就是这种形态：补丁写进文件就算过，**它只能证明「写了」，证明不了「对」**。
  2. 拆的第一步是把「谎报」这个词先摘掉。**模型不是在撒谎**——它只能看见上下文，它没有跑过那个服务，凭什么知道自己写的东西不 work。所以谎报完成不是模型品行问题，是「只能看见上下文」这条约束的必然结果，任何靠改提示词、加一句「请如实汇报」的方案都从根上不成立。
  3. 谎报有三种形态要分开：只写了壳（路由注册了、返回 200，副作用根本没发生）、测试没跑（它说跑了）、跑了但看错了结果。三种在代码 review 时都很容易漏掉，共同点是**只有把服务真的起起来发一个真请求才拦得住**。
  4. 于是判据只能是 harness 自己产出的证据，落地是三道门：**语法 → 起得来 → 用例**，顺序不能换。语法那道最便宜先跑；三道门的失败原因完全不同，给人看的报告里「语法坏了」和「行为不对」是两件事，混成一句「验证失败」会让早上排查时无从下手。
  5. 有一条容易被跳过的门槛要主动说：**一条用例都没覆盖到这条 feature 时不能算通过**。没有证据不等于通过——一个「套件为空就返回通过」的闸门看起来永远绿，而它什么都没验。这是恒真闸门最常见的入口。
  6. 实测数字值得原样背下来，因为它反直觉：同一份代码、同一个模型，浅检查那趟清单**声称 33 条、实测真能用 24 条、谎报 9 条**；端到端闸门那趟**声称 20 条、实测 20 条、谎报 0 条**。**浅检查的数字更好看。** 早上你会先看到一个更漂亮的进度，然后才发现服务的行为是错的，而清单上没有任何一条写着可疑。
  7. 可预期的追问是「验这一条就行了，为什么要把已通过的全部重跑」。因为有一类缺陷自己的用例是过的：本课的 F07 忘了「没传 tag 就原样放行」的守卫，带 tag 查时它筛得好好的，**被它弄坏的是别人**——实测连累了八条本来做对的 feature。只验当前这一条的闸门对它完全无能为力，抓住它的是回归套件。

  Key points

  • The completion criterion cannot be produced by the party under review: self-reports are not evidence.
  • A shallow check proves something was written, never that it is right.
  • False completion is not lying: the model never ran the service, so it cannot know.
  • The only trustworthy evidence is booting the service and sending real requests: syntax, boots, cases.
  • The three gates fail for different reasons, so the report must keep them apart.
  • No case covering the feature means no pass - absence of evidence is not a pass.
  • Measured: shallow claims 33 / 24 usable / 9 false; end-to-end 20 / 20 / 0 - and shallow looks better.

  答题要点

  • 完成判据不能由被审查的一方生产：模型的自我报告不是证据。
  • 浅检查只能证明写了，证明不了对：补丁落地就算过是故意留的破洞。
  • 谎报不是撒谎：模型没跑过那个服务，凭什么知道自己写的不 work。
  • 唯一可信的依据是真把服务起起来发真请求，三道门顺序固定：语法、起得来、用例。
  • 三道门失败原因不同，报告里语法坏了和行为不对必须分开写。
  • 没有用例覆盖到就不能算通过——没有证据不等于通过。
  • 实测：浅检查声称 33 / 真能用 24 / 谎报 9，端到端 20 / 20 / 0，浅检查数字更好看。
• Browser automation versus HTTP-only end-to-end verification - what does each buy and cost?端到端验证用浏览器自动化和只打 HTTP 接口，各有什么得失？

  Common in ChinaCommon overseasIntermediate#end-to-end#trade-offs#capability-boundary

  How to reason about it · think before answering

  1. This is about choosing a verification layer, not picking a winner. 'Browsers are obviously more realistic' and 'API tests are faster and more stable' each cover one half. The interviewer wants to hear what each layer cannot see, and which conditions forced the choice. Stating the boundary out loud says more about engineering maturity than which side you land on.
  2. Break it open with two conditions: does the target have a user interface, and where does this gate have to run? The target here is a pure HTTP service, and the repository imposes a hard rule on labs - they must run offline without asking the reader to install docker or a browser driver. Together those leave HTTP as the only option: a trade-off forced by constraints, not a preference.
  3. Name the cost before you are asked: rendering defects are entirely invisible at the HTTP layer. A blank page, a collapsed layout, a button that does not respond - the endpoint still returns 200, the probes still go green, the gate still lets it through. The Anthropic article uses browser automation precisely to cover that layer. Choosing HTTP means accepting that this class of defect survives until morning.
  4. The browser side has real costs too: another runtime dependency, slower runs, more flakiness - and that cost multiplies by item count, because the end-to-end gate runs a regression suite on every single feature, not once a night. At that multiplier, flakiness turns into a genuine problem, and the next question covers why it is worse than having no gate at all.
  5. The conclusion compresses into one line: switching to a browser driver changes nothing about the structure taught here - set up the environment, run the cases, and only a pass may flip the state. All three steps hold verbatim; only how the cases execute inside step three changes. These are not two designs, they are two execution layers of one gate, and which layer you pick depends on which layer your defects show up in.
  6. One more boundary, stated plainly: a probe may speak only through requests - no spawning processes, no reading source files, no touching git. A 'probe' that greps the source to see whether some function exists proves nothing about the service's behavior; it quietly degrades the gate you just built back into a shallow check - end-to-end in form, day one in substance.
  7. Expected follow-up: how do you combine them in a real project? In layers. HTTP carries the full regression (cheap enough to run per item), the browser covers a small set of critical paths (too expensive to run per item). The criterion is not which is more realistic but which layer a given class of defect becomes visible in - split that way, the two layers cover things that do not overlap.

  分析过程 · 先想清楚再作答

  1. 这题考的是**验证层次的取舍**，不是选型站队。答「当然浏览器更真实」或者「接口测试更快更稳」都只说了半边；面试官想听的是你能不能把各自**测不到什么**说清楚，以及这个选择被什么条件逼出来。能力边界说在明处，比选了哪一边更能说明工程成熟度。
  2. 拆法是先问两个条件：**靶子有没有界面**，以及**这套闸门要在什么环境里跑**。本课的靶子是纯 HTTP 服务，而本仓库对 lab 有一条硬约束——必须离线可跑、不依赖读者装 docker 或浏览器驱动。两个条件叠起来，HTTP 层是唯一选项，这是被约束逼出来的取舍而不是偏好。
  3. 代价必须主动交代，不能等对方问：**渲染类缺陷在 HTTP 层一个都测不到。** 页面白屏、样式塌掉、按钮点不动——接口照样返回 200，探针照样全绿，闸门照样放行。Anthropic 那篇原文用的是浏览器自动化，正是为了盖住这一层。你选了 HTTP 就等于承认这一类缺陷会一路漏到早上。
  4. 反过来浏览器那侧也有实打实的代价：多一套运行时依赖、慢、更容易偶发失败，而这个成本要乘以条数——端到端闸门在**每一条 feature** 上都跑一次回归套件，不是整晚跑一次。偶发失败在这个量级上会被放大成一个真正的麻烦，下一题会讲它为什么比没有闸门更糟。
  5. 结论可以压成一句话答出去：**换成浏览器驱动时，这一天讲的结构一个字都不用改**——起环境、跑用例、只有过了才允许翻状态，三步原样成立，变的只是第三步里用例的执行方式。所以这不是两套设计，是同一个闸门的两种执行层，选哪一层取决于你的缺陷会落在哪一层。
  6. 还有一条边界顺带说清楚：**探针只许通过发请求说话**，不 spawn 进程、不读源码、不碰 git。一条会去 grep 源码看某个函数在不在的「探针」证明不了服务的行为，它只会把刚建好的闸门重新退化成浅检查——形式上还是端到端，实质上回到了第一天。
  7. 可预期的追问是「真实项目里怎么配」。分层：HTTP 层做全量回归（便宜、可以每条都跑），浏览器只在少数关键路径上跑（贵、跑不起全量）。判据不是哪个更真实，而是**这一类缺陷会在哪一层显形**——照这个判据分，两层各自负责的东西是不重叠的。

  Key points

  • It is a choice of verification layer; the point is naming what each cannot see.
  • HTTP was forced here by a hard rule: labs run offline, with no docker or browser driver.
  • State the cost: rendering defects are invisible - blank pages and dead buttons still return 200.
  • The browser side costs dependencies, speed and flakiness, multiplied by a per-feature regression suite.
  • Swapping in a browser driver changes only how cases execute, not the structure of the gate.
  • Probes may speak only through requests; one that reads source code reverts the gate to a shallow check.
  • In practice, layer them: HTTP for full regression, browser for a few critical paths.

  答题要点

  • 这是验证层次的取舍，重点是说清各自测不到什么，而不是站队。
  • 本课选 HTTP 是被硬约束逼出来的：lab 必须离线可跑、不装 docker 与浏览器驱动。
  • 代价明写：渲染类缺陷全盲——白屏、样式塌、按钮点不动，接口照样 200。
  • 浏览器侧的代价是依赖、慢与偶发失败，而闸门在每条 feature 上都跑一次回归套件。
  • 换成浏览器驱动时结构一个字不用改，变的只是用例的执行方式。
  • 探针只许通过发请求说话：去读源码的探针会把闸门退回浅检查。
  • 真实项目分层：HTTP 跑全量回归，浏览器只覆盖少数关键路径。
• How do you detect an agent spinning in place, and what do you do once you have?你怎么发现一个 Agent 在原地打转？发现之后该做什么？

  Common in ChinaCommon overseasDeep dive#stall-detection#intervention#rollback

  How to reason about it · think before answering

  1. The second half is where this question separates people. Detecting a spin is already-solved ground - the course where you build a coding agent by hand has a stall signal for a checklist that has not moved in ten rounds, but it only detects, because a human is sitting right there and will take over. Unattended, detection without intervention is worthless: who is the alarm for? The hard part is the automatic decision that follows.
  2. Still answer the detection half properly: the checklist stops moving, the same feature keeps being reopened, the same file keeps being edited. All three point at one thing - steps going up while progress does not. Note that each is a fact the harness can observe on its own, with no cooperation from the model, which is the same principle as the previous question.
  3. There are exactly three interventions: retry, skip to another item, halt and alert. The criterion is whether the workspace is dirty, not how many times it failed. This is the one people get backwards. An empty patch can fail three times with the workspace still spotless, and reverting there throws away verified neighboring work. One line to remember: a rollback is not a punishment for failure, it is a repair of the foundation - if the foundation is intact, do not revert.
  4. Retries need a ceiling, and a low one. The reasoning is blunt: feed the same input to the same model and why would the second attempt differ? A retry only means something when this input differs from the last one, because a failure reason has been added to the context. Unlimited retries are not resilience, they are burning a whole night on one feature. By the same logic, once the number of skipped items passes a threshold, halt - constant skipping means the premise is wrong and continuing just burns money.
  5. This course measured what happens when the rollback step is removed, and the numbers sting: the end-to-end run drops from 20 completed items to 6, and the skipped set changes from F07, F18, F23 to F07, F08, F09. The last two are innocent. F07's bad code stays in the workspace filtering the whole list down to empty, F08 and F09 then fail verification one after another, and the harness skips two correctly built features as if they were impossible. The third skip hits the ceiling and the run halts. One uncleaned bad patch leaves six items for the night.
  6. That experiment also answers a common objection: 'reverting is too aggressive, just let it keep fixing'. Continuing to fix presumes a clean foundation. Once the foundation is dirty, every failure signal the harness receives no longer refers to the feature it thinks it does, and it starts punishing the innocent. So the criterion is not how aggressive you are, it is whether this failure contaminated already-verified work.
  7. Expected follow-up: should 'skipped' be a separate field in the state? No - derive it from the attempt counts: anything at the attempt ceiling that is still not in the completed set was skipped. It then persists alongside the state for free, and a crash-resume run needs no second record. If an existing authoritative record can answer the question, do not build a second record.

  分析过程 · 先想清楚再作答

  1. 这题的分水岭在后半句。**发现打转是已经被解决过的问题**——手搓 Coding Agent 那门课有「停滞信号：清单十轮不动」，但它**只做发现**，因为人就坐在旁边，发现了你会接手。本课人不在场，**发现而不干预等于白发现**：报警报给谁看？所以难的不是检测，是检测之后那个自动决定。
  2. 发现这一半仍然要答完整：清单长时间不动、同一条 feature 反复重开、同一个文件被反复改。三个信号指向同一件事——**步数在涨，进度不涨**。注意它们都是 harness 侧能独立观察到的事实，不需要模型配合，这一点和上一题是同一条原则。
  3. 干预只有三种：再试一次、换一条、停机报警。**判据是工作区脏不脏，不是失败了几次。** 这是最容易做反的一条：空补丁失败三次，工作区仍然是干干净净的，这时回退会把旁边已经验证过的工作一起丢掉。一句话记住——**回退不是对失败的惩罚，是对地基的修复。地基没坏就不该退。**
  4. 重试必须有上限，而且不该大。理由很直白：**同一条输入喂给同一个模型，凭什么第二次会不一样。** 重试有意义的前提是这次的输入和上次不同（上下文里多了一条失败原因）。无限重试不是韧性，是把一整夜烧在同一条 feature 上。同理，被换掉的条数超过阈值就该停机——一直在换说明前提出了问题，继续跑只是烧钱。
  5. 把回退这一步去掉会怎样，本课做过变异实测，数字很刺眼：端到端那趟完成条数从 **20 条掉到 6 条**，而且被换掉的从 F07 F18 F23 变成 **F07 F08 F09**。**后两条是无辜的**——F07 那段坏代码留在工作区里把整张列表过滤成了空，F08 F09 的验收接连失败，harness 把两条本来做对的 feature 当成做不出来的换掉了，第三条到顶就停机。一条没清干净的坏补丁让整晚只剩六条。
  6. 这个实验还顺带回答了一个常见反驳：「回退太激进了，不如让它接着修」。接着修的前提是**地基是干净的**；地基已经脏了还接着修，harness 收到的每一个失败信号都不再指向它以为的那条 feature，于是它开始惩罚无辜者。判据因此不是激进不激进，而是这次失败有没有污染已经验证过的工作。
  7. 可预期的追问是「被换掉这件事要不要在状态里单独存一个字段」。不用：它从每条的尝试次数推导——到了上限却仍然不在已完成集合里的，就是被换掉的那些。这样它天然跟着状态一起落盘，崩溃续跑时不用再补一份记录。**能让现成的权威记录回答的问题，不要另建一套记录。**

  Key points

  • Detection is solved; intervention is the hard part - unattended, detecting without acting is worthless.
  • Signals: the checklist stops moving, one item keeps reopening, one file keeps changing.
  • Exactly three interventions: retry, skip, halt and alert.
  • The criterion is a dirty workspace, not a failure count: an empty patch fails clean, so do not revert.
  • A rollback is not punishment for failure, it repairs the foundation; intact foundations stay put.
  • Retries need a ceiling: same input to the same model has no reason to behave differently.
  • Measured mutation: removing the rollback drops 20 items to 6 and skips two innocent features.

  答题要点

  • 发现是已解决的问题，难的是干预：人不在场时发现而不干预等于白发现。
  • 发现信号：清单长时间不动、同一条反复重开、同一文件反复改——步数在涨进度不涨。
  • 干预只有三种：再试一次、换一条、停机报警。
  • 判据是工作区脏不脏，不是失败了几次：空补丁失败三次仍然干净，就不该回退。
  • 回退不是对失败的惩罚，是对地基的修复；地基没坏就不该退。
  • 重试必须有上限：同一条输入喂给同一个模型，凭什么第二次会不一样。
  • 变异实测：去掉回退，20 条掉到 6 条，被换掉的从 F07 F18 F23 变成 F07 F08 F09，后两条无辜。
• A verifier built to catch false completions - how do you confirm it is not tautological itself?一个防谎报的验证器，你怎么确认它自己不是恒真的？

  Common in ChinaCommon overseasDeep dive#assertions#mutation-testing#probe-design

  How to reason about it · think before answering

  1. This question is about the credibility of the gate itself. A verifier that always returns pass is worse than no verifier: without one you at least know you have no evidence, whereas an always-green gate hands you an authoritative-looking false report that you then act on. So the gate must itself be verified before it goes live - who verifies the verifier is the real subject here.
  2. Split it into two directions, both required. First, prove it is not always green: run once with the gate switched off and nine false completions surface immediately (F02, F08, F09, F10, F11, F12, F15, F18, F29), while the run with the gate on reports zero. The phenomenon appears and disappears with the switch, which is what proves the gate does something. Second, prove it is not always red: on a complete target with all forty patches applied, all forty probes must pass; any red one means the probe itself is wrong.
  3. Always-green has three common entrances, all worth naming. One is passing on an empty suite - if a feature no case covers counts as passed, the gate is permanently green for every new feature. Two is verifying only the current item: F07 in this course passes its own case while breaking other people's (eight features, measured), and a single-item gate is blind to it. Three is a probe that reads source code, which reverts end-to-end into a shallow check - still verifying in form, back to 'it was written, therefore it passed' in substance.
  4. Always-red, and especially intermittent false red, is just as fatal and better hidden: a flaky red is worse than no gate at all, because it teaches people to ignore red lights. Once a team defaults to re-running whatever turns red, that habit swallows the genuine reds too. Hence the discipline that every probe is self-sufficient: it creates its own data, never asserts on global counts, cleans up after itself, and depends on neither execution order nor anything another probe left behind.
  5. This course hit a real instance worth retelling. Three probes originally used tag filtering to pick their own data out of the whole store, which looked clean - until the run loop skipped the tag-filtering feature because it failed verification. That code was simply not in the workspace, the filter parameter was ignored, the isolation evaporated, and all three probes went red because somebody else's feature was missing, halting the run at item eight. The rule tightened into one line: a filter may appear in a probe only when that feature is the filter, never because the probe needs to pick out its own data.
  6. The portable conclusion: assertions must be written in both directions. Asserting only that false completions are absent is a tautology trap, because an implementation that does nothing satisfies it too; it must be paired with confirming that the false completions really come back once the defense is switched off. Other courses in this repository have hit the same trap, so this is not a local quirk.
  7. Expected follow-up: how do you know the mutation test itself works? Look at how specific the red is. With the rollback disabled, the self-check does not merely fail - it names the unrunnable item that slipped onto the checklist. An assertion that says 'something is wrong' and one that says which thing is wrong differ by an order of magnitude in value at eight in the morning.

  分析过程 · 先想清楚再作答

  1. 这题考的是**闸门自己的可信度**。一个永远返回通过的验证器比没有闸门更糟：没有闸门时你至少知道自己没有证据，而一个恒绿的闸门会给你一份带着权威感的假报告，然后你照着它做决定。所以闸门上线前必须先被验一次——**谁来验验证器**，这是这题真正的题面。
  2. 拆成两个方向，缺一不可。一是证明它**不会恒绿**：把闸门关掉跑一趟，九条谎报当场冒出来（`F02 F08 F09 F10 F11 F12 F15 F18 F29`），开上闸门那一趟谎报是 0 条——现象随着开关出现和消失，说明它真的在起作用。二是证明它**不会恒红**：在一个四十条补丁全部打好的完整靶子上跑全部四十条探针，必须全绿；任何一条红了都是探针自己写错了。
  3. 恒绿有三个常见入口，都值得点名。第一个是**空套件返回通过**——没有任何用例覆盖到这条 feature 时判它通过，这条闸门对新增的 feature 永远是绿的。第二个是**只验当前这一条**：本课的 F07 自己的验收用例是过的，被它弄坏的是别人（实测连累八条），只验一条的闸门对它全盲。第三个是**探针去读源码**，那等于把端到端退回浅检查，形式上还在验、实质上又回到了「写了就算过」。
  4. 恒红和偶发假红同样致命，而且更隐蔽：**偶发的假红比没有闸门更糟，它会教人学会忽略红灯。** 一旦团队默认「红了先重跑一次」，真正的红也会被这个习惯吞掉。所以探针的纪律是每条**自给自足**：自己造数据、不断言全局条数、跑完收拾干净，不依赖执行顺序，也不依赖别的探针留下的东西。
  5. 本课在这里踩过一个真实的坑，值得原样讲出去：有三条探针最初用标签筛选把自己造的数据从全库里挑出来，看起来很干净——直到运行循环把**验不过的标签筛选那条 feature 换掉了**，工作区里根本没有那段代码，筛选参数无人理会，隔离当场失效，三条探针一起因为**别人的 feature 缺席**而变红，整个运行在第八条就停机。判据由此收紧成一句：筛选出现在探针里，只能是因为**这条 feature 本身就是它**，不能是因为我需要把自己的数据挑出来。
  6. 结论是一句可迁移的话：**断言必须双向写。** 只断言「谎报没有出现」是恒真陷阱，因为一个什么都不做的实现同样能让它成立；必须配上「关掉防线之后谎报确实回来了」。这条在本仓库的另外几门课上也被反复踩到过，不是本课特有的。
  7. 可预期的追问是「怎么知道变异检验本身有效」。看它红得**具体不具体**：关掉回退之后自检不只是转红，它直接点出清单里混进了跑不通的那一条。一条只会说「有问题」的断言，和一条能指出是哪一条出了问题的断言，在早上排查时的价值差一个数量级。

  Key points

  • An always-green gate is worse than none: it hands you an authoritative-looking false report.
  • Prove both directions: false completions return when the gate is off, and all forty probes pass on a complete target.
  • Three entrances to always-green: passing an empty suite, verifying only the current item, probes reading source.
  • Intermittent false reds are just as fatal - they teach people to ignore red lights.
  • Every probe is self-sufficient: own data, no global counts, no dependence on order or leftovers.
  • A real trap: using another feature as an isolation tool, which failed once that feature was skipped.
  • Write assertions both ways, and make the red specific enough to name the offending item.

  答题要点

  • 恒绿的闸门比没有闸门更糟：它给你一份带权威感的假报告。
  • 两个方向都要证：关掉闸门谎报回来（不恒绿），完整靶子上四十条探针全绿（不恒红）。
  • 恒绿的三个入口：空套件返回通过、只验当前这一条、探针去读源码。
  • 偶发假红同样致命——它会教人学会忽略红灯。
  • 每条探针必须自给自足：自己造数据、不断言全局条数、不依赖执行顺序或别人留下的数据。
  • 真实踩过的坑：拿别的 feature 当隔离工具，那条被换掉后三条探针一起假红、运行第八条停机。
  • 断言必须双向写，而且要红得具体——能指出是哪一条出了问题。

Comments