Tool Call Visualization and Humans in the Loop: Interruption Is Not Local Interception

Interview questions

• Why implement human-in-the-loop approval as run interruption and resumption rather than a frontend modal that blocks the request?为什么人在回路的审批要做成运行中断加恢复，而不是前端弹窗拦住请求？

  Common in ChinaCommon overseasDeep dive#human-in-the-loop#security#protocol-design

  How to reason about it · think before answering

  1. This separates candidates sharply, because almost everyone's first instinct is a frontend modal. That answer is not entirely wrong, but it does not survive the first follow-up: what if the user calls your API directly?
  2. Name the two independent problems with local interception. First, timing: the decision to call a dangerous tool happens on the server, and by the time the event stream reaches the browser the server may already be executing. A modal cannot stop it.
  3. Second, and more serious, security: the dangerous-tool list lives in frontend code, so editing the request or calling the endpoint from a terminal makes the confirmation vanish. **Treating the frontend as a security boundary is a classic mistake**; the frontend can advise, not enforce.
  4. The correct model makes approval part of the protocol: before a dangerous tool, the server deliberately ends the run with an interrupt outcome carrying an interrupt id. The frontend renders an approval UI. Once the user decides, the frontend issues a **new request** carrying that decision in a resume entry, and only then does the server decide whether to execute.
  5. This buys three things: the server is the sole executor so the gate is trustworthy; interrupt state can be persisted so the user can approve tomorrow; and each decision is a real request carrying an id, so the audit trail exists by construction.
  6. Expect the follow-up on what the frontend approval UI is still for: explaining consequences and collecting the decision, both of which the server cannot do. Worth adding that approval buttons should have no default selection, since a default decides for the user.

  分析过程 · 先想清楚再作答

  1. 这题的区分度极高，因为绝大多数人的第一直觉就是「前端弹个框」。答这个不算完全错，但接不住第一个追问：那用户绕过前端直接调你的接口呢。
  2. 先说清楚本地拦截错在哪，而且是两个独立的问题。第一个是时序：模型决定调用危险工具这件事发生在服务端，等事件流到前端时，服务端那边随时可能已经执行了，前端的框拦不住它。
  3. 第二个是安全，而且更致命：危险工具清单写在前端代码里，用户改改请求、或者直接用命令行调接口，那个确认框就完全不存在了。**把前端当安全边界是典型错误**，前端能做的只是提示，不是管控。
  4. 正确模型是把审批变成协议的一部分：服务端执行到危险工具前主动结束这次运行，结局标成中断并带上一个中断标识；前端据此渲染审批界面；用户决定后前端发起**新的一次请求**，在恢复条目里带上这个决定；服务端读到之后才决定要不要执行。
  5. 这个模型顺带解决了三件事：服务端是唯一执行方所以门是可靠的；中断状态可以持久化，用户关了浏览器明天再批也行；每个决定都是一次带标识的真实请求，审计记录天然就有了。
  6. 可预期的追问是「那前端的审批界面还有什么用」——它负责两件服务端做不了的事：把后果解释清楚，以及采集决定。还可以顺带提一句界面细节：审批按钮不该有默认选中项，默认值等于替用户做了决定。

  Key points

  • Local interception has two separate problems: the run is already executing server-side, and frontend checks can be bypassed.
  • The right model ends the run with an interrupt outcome; the frontend renders approval and sends the decision back as a resume entry on the next request.
  • The server is the sole executor and therefore the only trustworthy gate; the frontend explains consequences and collects the decision.
  • Interrupt state can persist so approval can happen later, and each decision is an identified request, giving audit for free.
  • UI detail: no default selection on approval buttons, since a default decides for the user.

  答题要点

  • 本地拦截有两个独立问题：运行已经在服务端跑起来了，以及前端判断可以被绕过。
  • 正确模型是服务端主动以中断结局结束运行，前端渲染审批界面，决定随下一次请求的恢复条目发回。
  • 服务端是唯一执行方，所以它才是可靠的门；前端只负责解释后果与采集决定。
  • 中断状态可持久化，用户可以晚些再批；每个决定是一次带标识的请求，审计记录天然具备。
  • 界面细节：审批按钮不设默认选中项，默认值等于替用户做了决定。
• Tool call arguments arrive as streamed fragments. What should the UI display before they are complete?工具调用的参数是一段段流式拼出来的，界面在参数还没拼完时应该显示什么？

  Common in ChinaCommon overseasIntermediate#tool-calling#streaming#ui-state

  How to reason about it · think before answering

  1. It looks simple but tests whether you have handled streaming tool calls for real. People who have not say 'show the arguments', which is exactly how half-formed JSON ends up on screen.
  2. State the fact first: the delta on an arguments event is a **fragment of a JSON string**, not an object. You may hold `{"path":"reports/20`, which will throw if parsed.
  3. So rule one is accumulate the raw text and parse only once the end event arrives, with error tolerance. Models do occasionally emit invalid JSON; do not let the UI crash. Mark the call failed and show the raw text, which beats a blank screen.
  4. As for what to display: **never show the half-formed JSON**. Seeing `{"path":"repo` makes users think the app broke. Show something like 'preparing arguments', optionally with a character count — honest without being alarming.
  5. A related state design point: keep 'arguments still arriving' and 'tool executing' as separate states rather than one spinner. The first is usually hundreds of milliseconds, the second can be tens of seconds, and they mean different things to the user.
  6. Expect a follow-up on testing this: assert on **what the user can see**, not internal fields. My first version asserted that the parsed field stayed null while incomplete, which turned out to be vacuously true since parsing partial JSON fails and returns null anyway. Asserting the displayed text made it meaningful.

  分析过程 · 先想清楚再作答

  1. 这题看着简单，考的其实是你有没有真处理过流式工具调用。没做过的人会答「显示参数」，而这恰恰会在界面上露出半截 JSON。
  2. 先说清楚事实：参数事件的增量是**一段 JSON 字符串的片段**，不是对象。你可能收到 `{"path":"reports/20` 这种东西，它 parse 一定抛异常。
  3. 所以第一条规则是**边收边存原文，不要边收边 parse**，等结束事件到了再一次性解析。而且解析要容错——模型偶尔真的会吐出不合法的 JSON，这时候不要让界面崩，把状态标成失败并把原文留给用户看，比白屏有用得多。
  4. 回到题目问的显示：**绝对不要把半截 JSON 原文显示出来**。`{"path":"repo` 出现在界面上，用户会以为程序崩了。合理的做法是显示「正在准备参数」，可以带一个已收到的字符数，既诚实又不制造恐慌。
  5. 顺带一个状态设计：「参数还在传」和「工具正在执行」要分成两个状态，不要合并成一个加载中。前者通常几百毫秒，后者可能几十秒，对用户的含义完全不同。
  6. 可预期的追问是「那怎么测这件事」——测试要盯**用户看得见的输出**，而不是内部变量。我自己写这段的测试时第一版断言的是「参数没收齐时内部字段为 null」，结果它恒真：提前 parse 半截 JSON 本来就失败返回 null。改成断言界面文案之后才真正有效。

  Key points

  • Argument deltas are fragments of a JSON string, not objects, so mid-stream parsing always throws.
  • Accumulate raw text and parse once at the end event, tolerating failure rather than crashing.
  • Never render partial JSON; show 'preparing arguments' with a character count instead.
  • Keep 'arguments arriving' and 'tool executing' as distinct states; their durations and meanings differ.
  • Assert on user-visible output, since asserting internal intermediate state easily produces vacuously true tests.

  答题要点

  • 参数增量是 JSON 字符串的片段而不是对象，中途 parse 必然抛异常。
  • 边收边存原文，结束事件到了再一次性解析，且解析失败要容错不要崩。
  • 界面绝不显示半截 JSON 原文，改显示「正在准备参数」加已收字符数。
  • 「参数在传」和「工具在跑」要分成两个状态，两者的时长量级和含义都不同。
  • 测试要断言用户可见的输出，断言内部中间状态容易写出恒真的假绿。
• The user rejects a tool call. What should your system do next?用户拒绝了一个工具调用，你的系统接下来应该怎么处理？

  Common in ChinaCommon overseasBasic#ux#error-handling#human-in-the-loop

  How to reason about it · think before answering

  1. This tests product judgment rather than technical difficulty. Many implementations fail the run outright, which is a UX failure.
  2. The deciding principle fits in a sentence: **the user declined an action, they did not ask for the conversation to break**. They most likely want the agent to propose something else. Turning refusal into failure punishes the user for exercising the control you gave them.
  3. The right move is to return the refusal as a **tool result**, stating that the user rejected it and why. The model can then understand what happened and offer an alternative, such as archiving the reports instead of deleting them.
  4. The run therefore still ends successfully rather than in error, and that distinction is visible in the events: a run finished with a success outcome, not a run error.
  5. Reflect it in the UI too: do not render rejection as a red error. Users read that as having done something wrong, when they merely made a choice. Give it a distinct neutral state.
  6. State the transferable rule out loud: **use the error state only when the system actually failed. A user's choice, whatever it is, is not a failure.** The same applies to form validation, permission denials, and cancelled payments.

  分析过程 · 先想清楚再作答

  1. 这题考产品判断，不是技术难点。很多实现直接让这次运行报错结束，而这是个体验事故。
  2. 判断依据一句话就能说清：**用户只是不想执行这个操作，不是想让对话崩掉**。他多半希望 Agent 换个方案继续。把拒绝做成失败，等于惩罚用户行使了你给他的权利。
  3. 正确做法是把拒绝也当成一个**工具结果**回给模型，内容写明被用户拒绝以及原因。模型拿到这个结果就能理解发生了什么，并给出替代方案，比如从「删除这批报告」改成「先归档这批报告」。
  4. 整个运行因此仍然是成功结束的，不是错误结束。这一点在事件上是有区别的：应该是带成功结局的运行结束，而不是运行错误。
  5. 界面上也要区分：拒绝状态不要渲染成红色报错。用户会以为自己做错了什么，而他只是做了个选择。给它一个独立的中性状态。
  6. 一条可迁移的判断标准，值得主动说出来：**只有系统真的出故障时才用错误态。用户的选择，无论是什么，都不是故障。** 这条在表单校验、权限拒绝、支付取消等场景同样适用。

  Key points

  • Rejection is not failure: the user declined an action, not the conversation.
  • Return the refusal as a tool result with the reason so the model can propose an alternative.
  • The run still ends with a success outcome rather than emitting a run error.
  • Give rejection its own neutral state in the UI instead of a red error.
  • General rule: reserve the error state for actual system failures; a user's choice is not one.

  答题要点

  • 拒绝不是失败：用户只是不想执行这个操作，不是想让对话崩掉。
  • 把拒绝作为工具结果回给模型，写明被拒绝与原因，让它给出替代方案。
  • 整个运行仍以成功结局结束，而不是发出运行错误事件。
  • 界面上给拒绝一个独立的中性状态，不要渲染成红色报错。
  • 通用判据：只有系统真出故障才用错误态，用户的选择不是故障。
• An agent fires three tool calls concurrently and their events interleave on arrival. How do you keep them straight?Agent 并发发起了三个工具调用，事件交错着到达前端，你怎么保证它们各自归位？

  Common in ChinaCommon overseasIntermediate#streaming#state-management#tool-calling

  How to reason about it · think before answering

  1. This checks whether you know not to rely on arrival order in a streaming protocol. Pushing to an array works perfectly with one tool call and breaks entirely under concurrency, in a way that only shows up in production.
  2. The answer is direct: **group by tool call id**, using a map keyed on that id rather than an array. Every event carries the id, so any argument fragment or result can find its card.
  3. Worth generalizing: yesterday message deltas merged by message id, today tool events merge by tool call id, tomorrow subagent events merge by subagent run id. **Everything in a streaming protocol merges by identifier, never by order** — that is the transferable lesson.
  4. A protocol detail worth mentioning: tool events also carry a parent message id, which lets the UI place the call under the right message in the timeline instead of piling every tool card at the end.
  5. Implementation-wise, mind the timeline itself: messages and tool cards interleave, so alongside the two maps you usually need an ordered list of what appeared when, or you have the data but no rendering order.
  6. Expect the follow-up about the same tool being called twice: each call has its own id, so you get two cards naturally. The real risk is keying on the tool name yourself.

  分析过程 · 先想清楚再作答

  1. 这题在考你有没有意识到「流式协议里不能依赖到达顺序」。按数组顺序 push 的实现在单个工具调用时完全正常，一并发就全乱，而且是那种线上才复现的乱。
  2. 答案本身很直接：**按工具调用标识归组**，用一个以该标识为键的表，而不是数组。每个事件都带这个标识，所以任何一片参数、任何一个结果都能找到自己的卡片。
  3. 值得多说一句的是这条规则的普适性：昨天处理消息增量用消息标识归并，今天处理工具用工具调用标识归并，明天处理子 Agent 用子运行标识归并。**流式协议里的一切归并都靠标识，不靠顺序**——这是一条能迁移的判断。
  4. 顺带提一个协议设计细节：工具事件上还带父消息标识，这让界面能知道这次调用挂在哪条消息下面，从而在时间线上正确排版，而不是把所有工具卡片堆在末尾。
  5. 实现上还要注意时间线本身：消息和工具卡片是混排的，所以除了两张表之外通常还需要一个记录出现顺序的列表，否则你有数据但不知道该按什么顺序渲染。
  6. 可预期的追问是「同一个工具被调用两次怎么办」——每次调用有各自独立的标识，所以天然是两张卡片，不需要特殊处理。真正需要小心的是你自己生成键的时候不要用工具名当键。

  Key points

  • Group by tool call id using a map, never an array, and never rely on arrival order.
  • It generalizes: streaming protocols merge by identifier — message id for messages, subagent run id for subagents.
  • The parent message id on tool events places the card under the right message instead of at the end.
  • Messages and tool cards interleave, so keep an ordered timeline list alongside the maps.
  • Two calls to the same tool naturally produce two cards since each has its own id; never key on the tool name.

  答题要点

  • 按工具调用标识归组，用以标识为键的表而不是数组，绝不依赖到达顺序。
  • 这是通用规则：流式协议里的归并一律靠标识，消息靠消息标识，子 Agent 靠子运行标识。
  • 工具事件带的父消息标识用来把卡片排到正确的消息下面，而不是堆在末尾。
  • 消息与工具卡片混排，所以还需要一个记录出现顺序的时间线列表。
  • 同一工具调用两次天然是两张卡片，因为每次调用有独立标识；不要用工具名当键。

Comments