先问后做：结构化提问工具、只读探索模式与计划审批

面试题库

• 什么时候 Agent 该停下来反问用户？问得太多和不问各有什么代价？When should an agent stop and ask the user a question? What does asking too much cost, and what does never asking cost?

  国内高频海外高频基础#clarification#human-in-the-loop

  分析过程 · 先想清楚再作答

  1. 这题看着是产品题，其实在考「你有没有一条能写进代码的判据」。答「不确定的时候就问」等于没答——模型对什么都不确定。
  2. 怎么拆：先给判据。**只有当两种合理读法会导致完全不同的改动时才问**，能自己读代码查明的事不许问。「让 divide 更安全一点」是前者（抛错和返回 null 改出来的代码不一样，而且测试期望的是哪一种也不确定）；「divide 在哪个文件里」是后者，grep 一下就有。这条判据的好处是它可判定，能直接写进工具的 description 里约束模型。
  3. 再说两端的代价，它们不对称但都真实。问太多：用户很快学会一路回车，于是那些提问全部失效——**一个被无脑通过的问题比不问更糟**，因为你以为自己确认过了。不问：你付的是返工，而返工的成本不在改代码，在于你要重新读一遍它改了什么再把它推回去。
  4. 结论：宁可少问，但每一问都要是真的岔路口；而且**提问必须便宜**——给选项、给默认值、支持直接回车，把回答的成本压到一次按键。默认值这一项最容易被忽略：用户在终端里最常做的动作就是直接回车，他心里想的是「按你说的默认那个来」。
  5. 可预期的追问：无人值守（CI、别的 Agent 调你）时怎么办？这时候没有提问渠道，**不许假装问过、也不许自动选第一个**。正确做法是回灌一条明确的结果：「当前环境无法向用户提问，请按最稳妥的一种做法继续，并在回答里说明你替用户做了哪个假设。」这条和审批门的口径一致——无人值守时不许自动同意。

  How to reason about it · think before answering

  1. This looks like a product question but really tests whether you have a criterion you could put in code. Answering ask when you are unsure says nothing — the model is unsure about everything.
  2. How to break it down: give the criterion. Ask only when two reasonable readings would lead to genuinely different edits; anything the agent can determine by reading the code is off limits. Make divide safer is the first kind — throwing versus returning null produce different code, and which one the test expects is also unsettled. Where does divide live is the second kind; a grep answers it. The value of this criterion is that it is decidable, so it can go straight into the tool description.
  3. Then the two costs, asymmetric but both real. Asking too much: users quickly learn to hit enter on everything, which voids every question — a question that gets rubber-stamped is worse than no question, because you believe you confirmed something. Never asking: you pay in rework, and the cost of rework is not the edit, it is re-reading what it changed and pushing it back.
  4. Conclusion: ask rarely, but only at real forks — and make asking cheap. Offer options, offer a default, accept a bare enter, so answering costs one keystroke. The default is the piece people forget: hitting enter is the most common thing a user does in a terminal, and what they mean is use the default you suggested.
  5. Likely follow-up: what about unattended runs, in CI or when another agent drives you? There is no channel to ask, and you must neither pretend to have asked nor silently pick the first option. Feed back an explicit result: this environment cannot ask the user, so proceed with the safest option and state which assumption you made on their behalf. That matches the approval gate's rule — never auto-approve when nobody is watching.

  答题要点

  • 判据是「两种合理读法会导致完全不同的改动」，能自己查明的事不许问
  • 问太多的代价是用户一路回车，于是提问全部失效——比不问更糟，因为你以为确认过了
  • 不问的代价是返工，而返工贵在你要重读它改了什么并把它推回去
  • 提问必须便宜：给选项、给默认值、支持直接回车，把回答压到一次按键
  • 无人值守时不许假装问过也不许自动选第一个，要明确回灌「这里问不了人，请说明你的假设」

  Key points

  • The criterion is two reasonable readings leading to different edits; never ask what code reading can settle
  • Asking too much makes users rubber-stamp everything, voiding the questions — worse than not asking, because you think you confirmed
  • Never asking costs rework, and rework is expensive because you must re-read the changes and push back
  • Asking must be cheap: options, a default, and a bare enter, so answering is one keystroke
  • Unattended, never fake an answer or auto-pick; feed back that nobody can be asked and require the assumption be stated
• 计划模式怎么实现？它和权限系统是什么关系？How would you implement plan mode, and what is its relationship to the permission system?

  国内高频海外高频进阶#plan-mode#permissions

  分析过程 · 先想清楚再作答

  1. 这题区分度最高，因为多数人会答「加一个 mode 字段，然后在该判的地方判一下」——而「该判的地方」有多少处，正是这题真正在问的。
  2. 怎么拆：先说结论。**计划模式不是一套新的权限系统，它是既有权限规则表的一个预设。** 进入模式只做一件事：把规则表整体换成两条——所有工具一律 deny，外加「交计划」这一个工具判 ask。只读工具根本走不到规则表（判定函数第一行就是「只读直接放行」），所以读文件、搜内容、反问用户全部照常可用，一个新分支都不用写。
  3. 为什么不能加 mode 字段到处判：你会在循环里判一次、在审批门里判一次、在每个写工具里再判一次，得到三处可能不一致的判断。它们不一致的那天，表现是「计划模式下它偷偷改了文件」——这种 bug 极难在测试里覆盖，因为它取决于哪条路径先命中。**权限的判定入口只能有一个**，这是比任何功能都硬的一条。
  4. 一个能显出你真写过的细节：**换规则表要整体替换，不能追加。** 如果匹配规则是「更具体的规则赢」（一个好的规则表都该这样），那么原表里那条「跑测试允许」带工具名带命令模式，比无条件的 deny 具体得多；把 deny 追加上去，计划模式下测试照跑，而且没有任何提示。原表存起来、退出时放回去即可。
  5. 顺着这条还有一个口径要表态：**计划模式该不该给「无害的命令」开口子。** 我的答案是不开——「哪条命令算只读」无法穷举（跑测试会不会写快照？会不会碰缓存？），而这个模式的全部价值就在于边界一句话说得清：只读工具全过，其余全挡。
  6. 可预期的追问：审批那一步要不要给协议加事件类型（比如一个 awaiting_approval 事件）？不要。把「交计划」做成一个**非只读工具**，既有的审批门就会自动拦住它去问用户；批准就是工具照常执行，拒绝就是回灌一条「没有被执行」的工具结果让模型改道。事件类型、循环、渲染层一个字都不用动，只有「摆给用户看的内容」要特判——而那本来就是渲染层的职责。

  How to reason about it · think before answering

  1. This has the most signal, because most people answer add a mode flag and check it where needed — and how many places need it is exactly what the question is probing.
  2. How to break it down: lead with the conclusion. Plan mode is not a new permission system; it is a preset of the existing rule table. Entering the mode does one thing: swap the table for two rules — deny every tool, plus ask for the single submit-a-plan tool. Read-only tools never reach the table at all, because the decision function's first line passes anything read-only, so reading files, searching, and asking the user all keep working with no new branch.
  3. Why not a mode flag checked everywhere: you end up checking it in the loop, in the approval gate, and inside every write tool — three judgments that can disagree. The day they do, the symptom is it quietly edited a file in plan mode, and that bug is nearly impossible to cover in tests because it depends on which path hits first. There must be exactly one place where permission is decided.
  4. A detail that shows you built it: swapping the table must replace, not append. If the matcher is most specific rule wins — as any good rule table should be — then an existing rule like allow running the test command carries a tool name and a command pattern and is far more specific than an unconditional deny. Append the deny and tests still run in plan mode, silently. Save the old table and restore it on exit.
  5. That leads to a stance worth stating: should plan mode carve out exceptions for harmless commands? I say no. Which command counts as read-only cannot be enumerated — does running tests write snapshots, or touch a cache? — and this mode's entire value is a boundary you can state in one sentence: read-only tools pass, everything else is blocked.
  6. Likely follow-up: does approval need a new protocol event, like awaiting_approval? No. Make submit-a-plan a non-read-only tool and the existing gate stops it to ask the user. Approval means the tool simply runs; rejection means feeding back a was-not-executed tool result so the model reroutes. Event types, loop and renderer stay untouched; only what is shown to the user needs a special case, and that was always the renderer's job.

  答题要点

  • 计划模式是既有权限规则表的一个预设，不是新的权限系统
  • 进入模式只做一件事：规则表整体换成「全部 deny + 交计划判 ask」两条
  • 只读工具走不到规则表（判定第一行就放行），所以探索能力是白拿的
  • 换表要整体替换不能追加，否则原表里更具体的规则会打败无条件的 deny
  • 权限判定入口只能有一处；审批走既有的门，不给事件协议加类型

  Key points

  • Plan mode is a preset of the existing permission rule table, not a separate system
  • Entering it does one thing: swap the table for deny-everything plus ask on the submit-plan tool
  • Read-only tools never reach the table, so exploration comes for free
  • Replace the table rather than appending, or a more specific existing rule beats the unconditional deny
  • There must be one permission decision point; approval reuses the existing gate with no new event types
• 批准了一份计划之后，怎么保证执行没有偏离？After a plan is approved, how do you ensure the execution did not drift from it?

  国内高频海外高频深入#plan-verification#drift-detection

  分析过程 · 先想清楚再作答

  1. 题眼在「保证」。多数人答到「让模型按计划执行、最后让它自己汇报」就停了，而那是让被检查的人写检查报告。这题要的是一个不依赖模型自觉的机制。
  2. 怎么拆：先说清偏离长什么样，以及为什么它抓不住。两种典型偏离是「多改了计划外的文件」和「没跑验证就宣布成功」，而它们的共同点是**都不是错误**：工具全部成功、循环正常结束、终端上一片对勾。静默的失败只能靠主动检查暴露。
  3. 所以第一步在**计划的格式**上：计划必须可核对，否则后面无从对起。四个字段——目标、每步改哪几个文件、怎么验证、有什么风险——其中「改哪几个文件」是核对的抓手，「怎么验证」是判断「做成了」的硬判据。自然语言计划（「优化一下边界处理」）改一个文件和改五个文件都算符合，等于没有计划。而且不合格的计划要**整份退回、不要就地补全**：我们替它补的验证方式是我们编的，用户批准之后没人对它负责。
  4. 第二步是**授权范围**：批准之后按计划里那几个文件逐个放行，一个不多。全开等于批准「随便改」；什么都不放行，则每个文件执行时还要再问一遍，用户很快学会一路按同意。**批准的是这份计划，不是无限授权。** 于是计划外的写入天然会撞回审批门——这是第一层防线，而且它是拦住的，不是事后发现的。
  5. 还有一条容易被忽略的红线：**按计划放行不能越过 deny。** 规则表里若有硬规则（例如不许改测试目录，因为不能靠改测试让它变绿），批准时无脑给计划里每个文件加放行，模型只要把测试文件写进计划就能借一次审批绕开硬规则。所以放行前要用原规则表判一次，本来 deny 的保持 deny 并明确告知模型。
  6. 第三步是**事后核对**：用一层透明包装（把事件原样传下去、路过时记一笔）采集这一轮真的成功写过哪些文件、真的跑过哪些命令，然后与计划对三列——计划内改了哪些、有哪些计划外的、有哪些计划里写了却没动的，再加一条「验证方式跑过没有」。只记成功的写入：被门挡下的、执行失败的不算实际改动。
  7. 可预期的追问：怎么证明这套核对不是摆设？做对照。同一份计划走两条支线，一条按计划执行，另一条故意多改一个文件且不跑验证，看核对表能不能抓到那两处。抓不到的核对表就是个装饰。另外要划清边界：核对只回答「这一次执行做了什么」，不回答「改动对不对」——后者是验证方式（跑测试）的职责。

  How to reason about it · think before answering

  1. The hinge is ensure. Most answers stop at have the model follow the plan and report back, which is asking the inspected party to write the inspection report. This question wants a mechanism that does not rely on the model's good faith.
  2. How to break it down: describe what drift looks like and why it hides. The two typical forms are edited files outside the plan and declared success without running the verification, and what they share is that neither is an error: every tool succeeded, the loop ended normally, the terminal is full of checkmarks. Silent failures only surface through an active check.
  3. So step one is the plan format: a plan must be checkable or there is nothing to check against. Four fields — goal, which files each step touches, how to verify, what the risks are — where the file list is the handle for reconciliation and the verification is the hard criterion for done. A prose plan like improve the boundary handling is satisfied by editing one file or five, which is the same as having no plan. And an unqualified plan must be returned whole rather than patched up: a verification step we invented is ours, and after approval nobody owns it.
  4. Step two is the authorization scope: on approval, grant exactly the files the plan lists, no more. Granting everything means approving arbitrary edits; granting nothing means every file is asked about again during execution, and the user learns to approve reflexively. What was approved is this plan, not unlimited authority. Writes outside the plan then hit the approval gate naturally — a first line of defense that blocks rather than merely reports.
  5. One easily missed red line: a plan-based grant must not override a deny. If the table has a hard rule — say, never edit the test directory, because you cannot make tests pass by editing tests — then blanket-granting every planned file lets the model smuggle a test file into the plan and launder it through one approval. Re-evaluate each file against the original table first, keep the denials, and tell the model explicitly.
  6. Step three is reconciliation after the fact: a transparent wrapper that passes events through while recording which files were actually written and which commands actually ran, compared against the plan on three axes — planned and touched, touched but unplanned, planned but untouched — plus whether the verification ran. Only successful writes count; gate-blocked and failed calls are not real changes.
  7. Likely follow-up: how do you prove the reconciliation is not decorative? Build a control. Run the same plan down two branches, one following it and one deliberately touching an extra file and skipping verification, and see whether the table catches both. A reconciliation that catches nothing is an ornament. Also draw the boundary: reconciliation answers what this execution did, not whether the change is correct — the latter is the verification step's job.

  答题要点

  • 偏离不是错误（工具全成功、终端一片对勾），静默失败只能靠主动检查暴露
  • 计划必须可核对：四个字段里「改哪些文件」是抓手、「怎么验证」是硬判据；不合格整份退回不要补全
  • 批准之后按计划里的文件逐个放行，一个不多——批准的是这份计划，不是无限授权
  • 按计划放行不能越过 deny，否则模型能把硬规则里的文件写进计划来洗白
  • 事后用透明包装采集实际写入与实际跑过的命令，核对计划内 / 计划外 / 漏做三列加一条验证跑没跑，并用对照支线证明它真能抓到偏离

  Key points

  • Drift is not an error — tools succeed and the terminal looks clean — so it only surfaces via an active check
  • The plan must be checkable: the file list is the handle, the verification is the hard criterion; return unqualified plans whole instead of patching them
  • On approval grant exactly the planned files — what was approved is this plan, not unlimited authority
  • A plan-based grant must never override a deny, or hard rules can be laundered through one approval
  • Afterwards, collect actual writes and commands with a transparent wrapper and reconcile planned, unplanned and untouched plus whether verification ran, proving it with a control branch

评论