Dayward AI
中文
Week 1 · D7About 6 hours

Packaging It as a Service: Fastify + SSE + Docker (dg P07); Week One Retrospective

Package the agent built over the previous six days into a Fastify service other frontends and services can call, streaming output over SSE and packaged with Docker, wrapping up week one.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Expose a chat endpoint that supports SSE streaming with Fastify
  2. Write the agent service as a Dockerfile, and build and run it successfully locally
  3. Summarize in one sentence the key concepts learned in week one, from the LLM API to the Pi SDK

Everything you wrote over the past six days ran the same way: type a command in your own terminal, watch the output, hit Ctrl+C, and serve exactly one person. Today it becomes something other people can call — with an address, an interface contract, a frontend able to connect to it, and an image you can carry to any machine. This is the last step of W1 and the starting point for W2's architecture split.

Plain-Language Walkthrough

From script to service: publish an interface and every assumption changes

Cooking in your own kitchen, you never think about opening hours, five tables arriving at once, or a guest ordering something that is not on any menu. The moment you open for business none of that is avoidable — your cooking did not change, the way it is used did.

A CLI script becoming a service is the same. Three unchallenged assumptions hide in the past six days of code. One user, so that history is yours by nature and nobody has to ask whose message this is. Serial execution, so two things never modify the same state at once. Trusted input, because you typed the arguments yourself and they are never a ten-megabyte string or JSON missing a field. All three fail in a service, and the first fails fastest and is hardest to find — because it behaves perfectly in single-person local testing.

There is a subtler one, and you solved a version of it only yesterday: on D6 you wrote sessions into .sessions/*.jsonl and a separate process could read them back. That road closes the moment you become a service. To handle concurrency you will run two instances, each writing its own local disk, and when the load balancer sends a user's second request to the other machine that disk has none of their history. D6 solved process restart; becoming a service introduces multiple instances, and those are not the same problem. So today we take a step back, put the history in process memory keyed by session, and get the service running. Moving it somewhere independent of any single instance is D8's job, and it is exactly the motivation for splitting gateway and worker next week.

session-store.js
// The script era: there was one user in the world, so a module-level array was enough
// const history = []
// With two people chatting in a service, both write into that same array and A sees B's conversation
 
// The right way: isolate by session
const histories = new Map()
 
function historyOf(sessionId) {
  if (!histories.has(sessionId)) histories.set(sessionId, [])
  return histories.get(sessionId)
}
// Note: the Map is only a step-back stopgap. D6's local file fails across instances,
// and D8 replaces this with Postgres

Four versions solve one problem, and the languages differ in how dangerous their defaults are: JavaScript's single thread lets you luck out of part of the concurrency problem, Java forces you to choose a concurrent container explicitly, and Swift's actor forbids the mistake at the type level. When you build a service in a new language, the first question to ask is who owns concurrency safety.

Session isolation is only the entry ticket. Interface design still has four decisions: the shape of the endpoint (return one complete JSON, or push as you generate), how a session is identified (the client carries a sessionId, or the server issues a cookie), auth and rate limiting, and how errors are expressed.

The last one is the most overlooked and the most damaging: a streaming endpoint cannot express an error with an HTTP status code. Once the 200 and the first byte are out, the status code is already on the wire; a subsequent model timeout, insufficient balance, or upstream 500 can only be reported as an agreed error event inside the stream. So the interface documentation has to name the three kinds of event — normal, finished, failed — and you must validate everything validatable before you start streaming, because that is your last chance to speak in status codes.

Once the contract is settled, the genuinely hard part is that long-lived connection itself: get the response headers wrong and the browser refuses it outright, sit idle for two minutes and a gateway silently cuts it, and if the user closes the page you keep burning money generating tokens for them. Let us work through it, starting with bringing the service up.

Bringing up Fastify: why a streaming endpoint has to sidestep the framework

A television station airing a pre-recorded program follows a fixed process: the finished cut is delivered, its runtime is known, it goes into the schedule, and it airs on the hour. Switch to live and none of that applies — nobody knows when the event ends, the signal can only be pushed out as it happens, and all the gallery can do is get out of the way and hand the line to the crew on site.

A web framework is that studio process, and SSE is the live line. The framework's normal path is: the handler returns an object, the framework serializes it to JSON, computes a Content-Length, writes it in one go, and closes the connection — every step assuming the response is one complete result of known length. SSE is the opposite: unknown length, many writes, and the connection stays open after the first batch. We choose Fastify not for benchmark numbers but because it turns getting out of the way into an explicit API: after reply.hijack() it stops managing that response and you write to reply.raw yourself. It also ships JSON Schema validation, which gives "validate before streaming" somewhere to live.

What sits behind this /chat is nothing new: it is the agent loop the Pi SDK absorbed on D3, plus D4's model-calling layer and D5's self-correcting tools. Only the entrance and exit changed: from reading a line of stdin to reading an HTTP request body, and from printing to a terminal to writing into an SSE stream. Becoming a service does not change the agent kernel, it gives it a different shell.

server.js
import Fastify from 'fastify'
 
const app = Fastify()
 
app.get('/healthz', async () => ({ ok: true }))
 
app.post('/chat', async (request, reply) => {
  const message = String(request.body?.message ?? '').trim()
  // Before streaming starts is the last chance to speak in status codes
  if (!message) return reply.code(400).send({ error: 'message must not be empty' })
 
  reply.hijack() // hand over control: from here I write raw bytes myself
  const res = reply.raw
  res.writeHead(200, SSE_HEADERS)
 
  for await (const chunk of streamChat(message)) {
    res.write(`event: delta\ndata: ${JSON.stringify({ text: chunk })}\n\n`)
  }
  res.write('event: done\ndata: {}\n\n')
  res.end()
})
 
// Listen on 0.0.0.0 rather than 127.0.0.1, or nothing can reach it inside a container
await app.listen({ port: 3000, host: '0.0.0.0' })

The cross-language comparison is worth noticing: Spring WebFlux makes SSE first-class and you never touch a frame; Node and Swift assemble the strings themselves. That changes where you look when something breaks — in the hand-assembling languages, "the client receives nothing" is nearly always a malformed frame, while in the framework-assembled ones the same symptom usually comes from the proxy layer.

Notice the two-line /healthz route as well. It looks pointless, but container orchestration and load balancers rely on it entirely to decide whether an instance may take traffic — a service with no health endpoint is a black box as far as an orchestrator is concerned.

The server side of SSE: headers, heartbeats, disconnects

Picture the pass-through window in a restaurant kitchen: dishes go out one at a time and the guest eats while waiting. Three things have to be right at that window. It has to stay open. Something has to happen every so often, or the people outside conclude the kitchen has gone quiet. And if the guest leaves midway you need to know, so you stop cooking for that table.

This section and D1 are two sides of one thing. D1 stood on the collection side: how to receive chunk by chunk, how to stitch a half-line, how to ask for the rest after a drop. Today we stand inside the window: how to emit, what to declare before emitting, and how to keep the connection alive while emitting. D1 already covered what the wire format looks like, so here we only cover the side that produces it.

Headers first. Four of them, none optional:

HeaderWhat happens without it
Content-Type: text/event-streamthe browser's EventSource declares the connection failed, and does not reconnect
Cache-Control: no-cache, no-transforman intermediate proxy may cache the whole response, or helpfully compress and rewrite the body
Connection: keep-alivestates explicitly that this connection is meant to stay open
X-Accel-Buffering: noNginx buffers by default until a buffer is full, and enough buffering turns streaming back into one-shot

That last one is the textbook "fine locally, gone in production": curl is perfect on your machine, and behind a reverse proxy the user still gets a spinner followed by the whole answer at once. Eight times out of ten a streaming fault is not in your code, it is in the layer in between.

Now the frame format. A frame is several lines: event: is the event name, data: is the content, id: is the number, and a line starting with a colon and no field name is a comment. The crucial part is that a blank line is what ends a frame. Omit one newline and the client waits forever, presenting as "the server is stuck" while the server's logs look entirely healthy.

TextText
id: 1
event: delta
data: {"text":"To"}
 
: this is a comment line, the client ignores it
 
event: done
data: {"chunks":103}

While we are here, claim the event names' lineage: delta / done / error are not invented today, they correspond to model:delta and run:end / run:error from D5's internal event stream. On D5 they only printed to a terminal; today those three get an SSE envelope and travel across the network. D5's tool events such as tool:start have no frame yet, and putting them on the wire is left for later. Design the event system well and becoming a service is only adding an encoding layer to it.

Then heartbeats: write a comment line into the connection periodically. Load balancers and gateways generally have an idle timeout, commonly 60 to 120 seconds, and close a connection with no bytes flowing. An agent has plenty of silent stretches — the model is thinking, calling a tool, waiting on a slow endpoint — and going tens of seconds without a token is routine. Use a comment line rather than a custom event because every client ignores it silently.

Finally, disconnects. The moment the user closes the page the server does not stop by itself: the model keeps generating and the tokens keep billing, with nobody receiving them. This is the most expensive oversight in a streaming service, and a test environment cannot surface it at all.

lifecycle.js
// Heartbeat: a comment line the client ignores, but to a gateway it is a byte saying "alive"
const timer = setInterval(() => res.write(': ping\n\n'), 15000)
timer.unref()
 
const controller = new AbortController()
// Listen for close on the response object, not the request - in Node the request's close
// fires once the request body has been read, which misreads a normal request as a drop
res.on('close', () => {
  if (!res.writableEnded) controller.abort() // cut the upstream, stop burning money for someone who left
})
 
try {
  for await (const chunk of streamChat(message, controller.signal)) {
    res.write(`event: delta\ndata: ${JSON.stringify({ text: chunk })}\n\n`)
  }
} finally {
  clearInterval(timer)
  if (!res.writableEnded) res.end()
}

One thing today can only be done halfway: numbering the events. The id: in a frame is the number the client remembers, and the browser's native EventSource sends it back in Last-Event-ID on reconnect. But as D1 established, an LLM endpoint has to be POSTed to, EventSource can only GET, real frontends hand-parse, and that automatic reconnection is unavailable. What the server can do is number properly and keep a second copy of what it pushed — genuine resumption needs somewhere to store the stream that exists independently of this connection, and that is what W2 builds.

The three layers of reconnection: not the same problem1/5
Layer 1 · Native browser EventSourceGET only
ConnectedConnection established; the server timestamps every event with id:
Layer 2 · LLM API (fetch + POST)EventSource can't help here
ConnectedA POST goes out with messages; the response body is SSE-formatted
Layer 3 · The user closes the tab and comes backThe client is already gone
ConnectedThe server writes the stream into Redis as it generates, and records an activeStreamId
The three kinds of "disconnect" are actually three completely different problems. First, the normal state: all three paths are running along fine.

Putting the service in a shipping container: a minimal Dockerfile

Before the shipping container, cargo came in every shape: barrels, sacks, loose bulk, and every transfer between ships meant restacking by hand. The container itself contains no clever technology; its entire value is that it standardized the dimensions and the crane interface — after which every ship, truck, crane, and port was designed around one interface and handling costs fell by an order of magnitude.

Docker does the same for software: it standardizes "how to run this program" into one interface, and whether Node, Python, or Java sits inside, the outside is one image plus one docker run. It does not solve a performance problem, it solves a handover problem — what you hand over stops being an installation document.

dockerfiledockerfile
# Pin the version, never use latest: an image's value is that another machine gets the same result
FROM node:22-alpine
RUN corepack enable
WORKDIR /app
 
# The key move: copy only the dependency manifests first, install, then copy the source.
# Docker caches layer by layer, so editing one line of src invalidates only the last two
# layers while the dependency layer still hits. COPY . . up front means one character
# reinstalls everything
COPY package.json pnpm-lock.yaml .npmrc ./
# That trailing flag is not filler; the reason is in the next paragraph
RUN pnpm install --frozen-lockfile --config.strictDepBuilds=false
 
COPY tsconfig.json ./
COPY src ./src
 
ENV NODE_ENV=production
ENV PORT=3000
ENV HOST=0.0.0.0
EXPOSE 3000
USER node
 
CMD ["node", "--import", "tsx", "src/index.ts"]

Twenty lines hide five frequent interview points. First, layer caching: an image is stacked layer by layer and a layer whose inputs are unchanged is reused, so instruction order decides whether a build takes seconds or minutes. Second, pinning versions: node:latest means your image quietly moves to the next major version one day, and reproducibility is gone. Third, the bind address: inside a container you must listen on 0.0.0.0, because listening only on 127.0.0.1 is reachable only from inside the container and no host port mapping will help — the number one beginner trap, precisely because it works perfectly on your machine. Fourth, do not run as root: a container shares the host kernel, and running your workload as root maximizes the damage surface after an escape.

The fifth relates directly to SSE and is the most overlooked: PID 1 and signals. The first process in a container is PID 1, and docker stop sends SIGTERM. Write the start command as pnpm start and PID 1 is the package manager, SIGTERM may never reach node, graceful shutdown never runs, and you wait out a ten-second timeout before SIGKILL. For an SSE service that means every in-flight stream is cut hard, and users see a reply stop mid-sentence. So start node directly in array form, and on SIGTERM stop accepting new connections, give in-flight streams a moment to finish, and then exit.

The --config.strictDepBuilds=false at the end of the install line needs a word: pnpm 11 exits non-zero when a dependency's build script is ignored, which locally is one extra line of output and inside a Dockerfile is a failed RUN that kills docker build on the spot. A container build promotes every non-zero exit to a fatal error, which is the most common way a script fails when it moves into an image.

Two more disciplines: .dockerignore must exclude node_modules (host binaries do not run in a Linux container) and .env (a key baked into an image is a key given to everyone who can pull it; pass it at runtime with --env-file).

W1 retrospective: seven days were one line

Looking back, this week was not seven separate topics but one agent growing step by step, each day forced by the day before:

  • D1 established what talking to a model actually is: the messages array, three roles, the context window, streaming, temperature. The conclusion was that the model has no memory and no hands, and only continues text.
  • D2 therefore hand-wrote a loop and tools so it could act — reading the stop reason to decide whether to continue, and turning a tool result into a message fed back in.
  • D3 noticed that after hand-writing it, this loop looks identical in every agent, so the Pi SDK absorbed it into three API calls and we compared what the framework does for you.
  • D4 found two things inside that the framework should not decide: which vendor's model, and what the persona is. So we pulled out a model-calling layer with fallback and wrote the system prompt explicitly.
  • D5 faced tools that crash and arguments filled in wrong, producing parameter validation and error feedback for self-correction, plus event subscription to expose internal state.
  • D6 watched tool results and multi-round conversation burst messages, producing context compression and session persistence — the agent grew a memory.
  • D7 packed all of it into a service today: an endpoint, streaming, a container image, and finally something other people can call.

If you take away only three sentences, I suggest these. One, everything outside the model is yours to build. The loop, the tools, the memory, the reliability, the observability — none of it comes with the API, which also answers what an agent engineer does all day. Two, every layer of abstraction exists to shut one kind of change into one room. The model layer shuts in vendor differences, tool schemas shut in the uncertainty of model output, context compression shuts in the window ceiling, and the service layer shuts in caller differences; a layer whose change you cannot name is probably surplus. Three, every technical decision maps to a bill. Multi-round cost grows with rounds, fallback pays for the same prompt twice, and not stopping generation after a user leaves is a pure loss.

One question for next week. The session history is currently sitting in process memory, so it falls apart the instant you run a second instance, and D6's local file cannot save it either. A single-process service's ceiling is not performance, it is state kept in the wrong place.

Source Reading

Hands-On Lab

🧪 D7 lab: a containerizable agent-service

Code location: labs/agent-30days/day-07-agent-service-docker

Acceptance criteria:

  1. All four self-checks under MOCK=1 SELFTEST=1 pnpm start pass (starter/ as-is passes 1 and 2 and fails 3 and 4, so run it once and remember what that looks like).
  2. Check 3 shows Content-Type: text/event-stream; charset=utf-8, receives 103 delta events plus 1 done event, and reassembles into one complete sentence.
  3. Check 4 reports that the client disconnected after reading 5 fragments while the server generated 6 in total — far short of 103, proving the server really stopped once the user left.
  4. In another terminal, curl -N -X POST shows frames appearing one at a time rather than the screen filling all at once after a few seconds; pnpm typecheck passes with no any.
  5. (Optional, only with Docker) docker build succeeds, the four self-checks inside the container match the host, and docker stop returns within a second rather than hanging for the full ten.

This lab differs from the previous days: it is a long-running service, and pnpm start does not exit by itself, so it adds a self-check mode. SELFTEST=1 makes the process issue a few requests to itself after startup, print the results, and exit, so one command is the whole acceptance run. The four exercise points in starter/ map to this chapter's four traps: headers, frame format, heartbeat, disconnect handling. The first two self-checks (health endpoint, empty argument returning 400) are already scaffolded and pass from the start; the blanks affect the last two. Remember to give curl -N.

One more word on the lab's boundary, so you do not think W1 was wasted: it focuses on the service layer, and the agent kernel is replaced by a minimal stand-in — a streamChat that only emits a reply character by character, with no tools and no history refill. Plugging the full agent from the past six days in means replacing that one function, with no change to the interface shape, the frame format, or the disconnect handling. Each of these four service-layer traps can fail independently, and mixing agent logic in only makes it harder to tell which layer you are looking at.

  1. Run MOCK=1 SELFTEST=1 pnpm start as-is first and see what the failures on checks 3 and 4 look like (the first two are scaffolded and already pass); then complete the SSE headers and frame format until check 3 passes.
  2. In a second terminal run MOCK=1 pnpm start and hit POST /chat with curl -N, confirming with your own eyes that the reply arrives frame by frame rather than all at once.
  3. Add the heartbeat timer and the disconnect handling, then rerun the self-checks: the fragments generated by the server in check 4 must fall from 103 to single digits.
  4. Build an image with docker build, bring it up with docker run -p 3000:3000, curl it from the host, then docker stop and confirm it exits within a second.
  5. Write a W1 retrospective note: one sentence per day, then answer which day you still cannot explain clearly — that one is your weekend homework.

Interview Questions

Today's five questions are in the bank below, covering SSE versus WebSocket selection, the interface design of turning a script into a service, the key Dockerfile decisions, heartbeats and graceful shutdown on long connections, and finally a self-introduction. Read the analysis before the key points — question 5 has no model answer, but you will reuse its derivation framework repeatedly over these two months.

Checklist and Tomorrow

  • Expose a chat endpoint that supports SSE streaming with Fastify
  • Write the agent service as a Dockerfile, and build and run it successfully locally
  • Summarize in one sentence the key concepts learned in week one, from the LLM API to the Pi SDK
  • Say what each of the four SSE response headers guards against, and why a heartbeat uses a comment line
  • Explain why a streaming endpoint cannot report an error with a status code once streaming has begun, and name the alternative
  • All 5 acceptance criteria of the lab pass (skip the Docker one without an environment)
  • Answer at least 3 of the 5 interview questions without looking at the key points

Tomorrow (D8) begins week two, and the first job is taking today's single-process service apart: the gateway handles only auth, rate limiting, and delivery, the worker concentrates on running the agent loop, and a queue decouples them. Why this order? Because only after hand-writing a service that carries everything itself do you know what the split is meant to solve — and today's in-memory session Map becomes three Postgres tables tomorrow: sessions, runs, and messages.

Interview questions

  • For streaming LLM responses, would you pick SSE or WebSocket, and why?流式返回大模型回复，你会选 SSE 还是 WebSocket？为什么？
    Common in ChinaCommon overseasBasic#sse#streaming#api-design

    How to reason about it · think before answering

    1. The hinge is 'how would you pick', not 'what is the difference'. Reciting 'SSE is one-way, WebSocket is two-way' scores nothing — that is the first paragraph of any doc.
    2. Ask one question that nearly decides it: does the client need frequent upstream messages on this connection? Chat completion is one request followed by a long push, which is exactly SSE's shape. Collaborative editing, realtime games and voice are what WebSocket is for.
    3. Give three practical wins for SSE: it is ordinary HTTP, so auth headers, cookies, rate limiting, logging, CDNs and reverse proxies all keep working; the server just writes bytes into a response, with no separate connection lifecycle to manage; and the wire format is plain text, so curl is your debugger. WebSocket runs an upgraded protocol where most of that tooling has to be rebuilt.
    4. Volunteer SSE's two real limits before they are raised. First, the browser's native EventSource can only issue GET, while model endpoints require POST, so real frontends hand-roll the parser with fetch and the spec's Last-Event-ID auto-reconnect never applies. Second, HTTP/1.1 caps concurrent connections per origin, so several tabs each holding a stream compete; HTTP/2 largely removes this.
    5. Land on a decision rule: one-way push means SSE, high-frequency bidirectional means WebSocket, and when unsure start with SSE — its escape hatch is adding one upstream endpoint, while WebSocket's escape hatch is rebuilding your infrastructure.
    6. Expect the follow-up: what about the 'stop generating' button? It does not need the same connection — send a plain POST carrying the run id, have the server abort upstream, and the SSE stream ends on its own. This one separates people who shipped it from people who read about it.

    分析过程 · 先想清楚再作答

    1. 这题的题眼是「怎么选」，不是「有什么区别」。只背出「SSE 单向、WebSocket 双向」拿不到分，因为那是文档第一段。
    2. 先问自己一个问题，它几乎决定了答案：这条连接上客户端需不需要频繁上行？聊天补全是「一次请求、一路往回推」，上行只有最开始那一次，完全落在 SSE 的形状里；协同编辑、实时游戏、语音这种双向高频才轮到 WebSocket。
    3. 然后给 SSE 的三条实际好处：它就是普通 HTTP，鉴权头、Cookie、限流、日志、CDN、反向代理这一整套现成设施全部照用；服务端只是往响应里写字节，不需要额外的连接管理；协议是纯文本，出问题 curl 一下就能看。WebSocket 走的是升级后的独立协议，前面那套东西大多要重做一遍。
    4. 接着说 SSE 的两个真实限制，主动说破比被问出来强：一是浏览器原生的 EventSource 只能发 GET，而大模型接口必须 POST，所以真实前端都是 fetch 手写解析，规范里那套 Last-Event-ID 自动重连一行都用不上；二是 HTTP/1.1 下同域并发连接数有限制，多个标签页各开一条长连接会互相挤占，HTTP/2 之后这条基本消失。
    5. 结论要落到一句可判断的话：单向推送选 SSE，双向高频选 WebSocket；拿不准就先用 SSE，因为它的退路是加一个上行接口，而 WebSocket 的退路是重做整套基础设施。
    6. 可以预期的追问：那大模型产品里的「停止生成」按钮怎么办？答案是它根本不需要走同一条连接——另发一个普通的 POST 请求带上这次生成的 id，服务端收到就中止上游，SSE 那条连接自然结束。这个追问很能区分有没有真做过。

    Key points

    • Decide by upstream frequency: one request plus a long push (chat completion) fits SSE; high-frequency bidirectional traffic needs WebSocket
    • SSE is plain HTTP, so auth, rate limiting, logging, proxies and CDNs all still apply, and curl is enough to debug it
    • Name SSE's limits yourself: EventSource is GET-only while model endpoints need POST, so spec auto-reconnect does not apply; HTTP/1.1 also caps per-origin connections
    • When unsure start with SSE — adding one upstream endpoint is cheaper than rebuilding infrastructure around WebSocket
    • A stop button does not need the same connection: POST the run id and abort upstream, and the stream ends by itself

    答题要点

    • 先判断上行频率：一次请求、一路往回推的场景（聊天补全）用 SSE，双向高频（协同编辑、语音）用 WebSocket
    • SSE 就是普通 HTTP，鉴权、限流、日志、代理、CDN 这套设施全部照用，排查时 curl 就够
    • SSE 的限制要主动说：EventSource 只能 GET，而模型接口必须 POST，所以自动重连用不上；HTTP/1.1 下同域连接数有限
    • 拿不准先选 SSE：加一个上行接口就能补足，而换 WebSocket 要重做整套基础设施
    • 「停止生成」不用走同一条连接，另发一个 POST 带 run id 让服务端中止上游即可
  • When turning a local agent script into a production service, what does the interface layer have to get right?把一个本地跑的 Agent 脚本改造成生产服务，接口层要重点考虑哪些事？
    Common in ChinaCommon overseasIntermediate#api-design#service-architecture#streaming

    How to reason about it · think before answering

    1. This tests whether you can name the assumptions hidden in a script. A generic checklist (auth, logging, monitoring) scores nothing; name the assumptions that silently break.
    2. List them first: one user (so history can live in a module-level variable), serial execution (no two requests mutating the same state), trusted input (you typed the arguments yourself), and a process whose life equals the session's. All four break in a service, and the first is hardest to catch because single-user local testing looks perfect.
    3. Then give the four decisions: response shape (single JSON versus streamed events), session identity (client-supplied id versus server cookie, and where history is stored), authentication and rate limiting (who may call, how often, and the per-call token ceiling), and how errors are expressed.
    4. Expand the last one — it is where this question is actually won. Once a streaming endpoint has written 200 and the first byte, the status code is already on the wire, so a later timeout, out-of-credit or upstream 500 can only surface as an agreed error event inside the stream. Validate everything you can before the first byte, because that is your last chance to speak in status codes.
    5. Add a production note: ship a health endpoint. Without one, orchestrators and load balancers cannot tell whether an instance is ready, and rolling deploys send traffic to a process that has not finished booting.
    6. Expect the follow-up: why cap tokens per request at the interface layer? Because agent cost is triggered by the caller and paid by you — no cap means handing your wallet to the client. Rate limiting is about money per call, not just QPS.

    分析过程 · 先想清楚再作答

    1. 这题考的是「你知不知道脚本里有哪些隐含假设」。答成一份笼统的清单（鉴权、日志、监控）拿不到分，要说出脚本时代默认成立、服务里立刻不成立的那几条。
    2. 先把假设列出来，这是最能体现工程视角的一步：只有一个用户（历史可以放模块级变量）、串行执行（不会有两个请求同时改一份状态）、输入可信（参数是自己敲的）、进程和会话同生共死（Ctrl+C 之后不用交代）。四条在服务里全部不成立，而第一条最难查，因为它在本地单人测试时表现完美。
    3. 然后给出四个必须做的决定：接口形状（一次性 JSON 还是流式推送）、会话标识（客户端带 sessionId 还是服务端发 cookie，以及历史存哪里）、鉴权与限流（谁能调、多久能调一次、单次 token 上限）、错误怎么表达。
    4. 第四条要单独展开，它是这题真正的区分点：流式接口一旦写出 200 和第一个字节，状态码就已经发出去了，之后模型超时、余额不足、上游 500，都只能在流里补发一个约定好的 error 事件。所以推流之前必须把能校验的全部校验完，那是你最后一次能用状态码好好说话的机会。
    5. 再补一条生产视角：服务要有健康检查接口。没有它，编排系统和负载均衡就没法判断这个实例能不能接流量，滚动发布时会把请求打给一个还没起好的进程。
    6. 可以预期的追问：单次请求的 token 上限为什么要在接口层限制？因为 Agent 的成本是请求方触发、你来买单，不设上限就等于把钱包交给调用方——限流限的不只是 QPS，还有每次调用能烧多少钱。

    Key points

    • A script's four assumptions all break in a service: single user, serial execution, trusted input, and a process that dies with the session
    • Session state must be keyed by session id, and in-process storage means data is lost on restart and blocks horizontal scaling
    • Four interface decisions: response shape, session identity, auth and rate limiting including a per-call token ceiling, and error semantics
    • A streaming endpoint cannot report errors by status code after the first byte, so define an in-stream error event and move all validation ahead of it
    • Expose a health endpoint, or orchestrators cannot tell whether the instance is ready for traffic

    答题要点

    • 脚本的四个隐含假设在服务里全部不成立：单用户、串行、输入可信、进程与会话同生共死
    • 会话状态必须按 sessionId 隔离，且要意识到放进程内存意味着重启即丢、无法水平扩容
    • 四个接口决定：响应形状、会话标识、鉴权与限流（含单次 token 上限）、错误表达方式
    • 流式接口推流之后无法用状态码报错，必须约定一个流内的 error 事件，并把校验全部前置到第一个字节之前
    • 提供健康检查接口，否则编排系统无法判断实例能不能接流量
  • What are the key decisions in a Dockerfile that packages a Node service?把一个 Node 服务打包成 Docker 镜像，Dockerfile 里有哪些关键决定？
    Common in ChinaCommon overseasIntermediate#docker#deployment#nodejs

    How to reason about it · think before answering

    1. It looks like a recipe question, but it tests whether you have ever traded off build speed against security. Reading FROM, COPY, RUN, CMD in order is the least differentiating answer.
    2. The first decision is instruction order, the only one with an immediately measurable payoff. Images are stacked layers and a layer whose inputs are unchanged is reused, so copy the manifest and lockfile first, install, then copy source. Editing one line of code then invalidates only the last two layers instead of forcing a full reinstall.
    3. Second, pin the base image. Using latest means the image silently jumps a major version some morning, which destroys the reproducibility that was the whole reason to containerize.
    4. Third, runtime configuration: bind to 0.0.0.0 inside a container. Binding 127.0.0.1 leaves the service reachable only from inside, so a published port still refuses connections — and it works perfectly on your laptop, which is why it is so common. Also note EXPOSE only documents intent; the port is actually published by docker run -p.
    5. Fourth, security: run as a non-root user, since containers share the host kernel and root widens the blast radius of an escape. Keep node_modules out via .dockerignore (host binaries will not run in a Linux container and the build context balloons) and keep .env out too, passing secrets at runtime with --env-file.
    6. Expect the follow-up, and it is the one a streaming service should volunteer: use the exec-form CMD to launch node directly so it becomes PID 1. With pnpm start, PID 1 is the package manager, SIGTERM from docker stop may never reach node, your graceful shutdown never runs, and the container is SIGKILLed after the timeout — cutting every in-flight SSE stream.

    分析过程 · 先想清楚再作答

    1. 这题看着是背步骤，其实考的是「你有没有为构建速度和安全性做过取舍」。把 FROM、COPY、RUN、CMD 顺着念一遍是最没有区分度的答法。
    2. 第一个决定是指令顺序，也是唯一能立刻量化收益的：镜像是逐层叠出来的，某层的输入没变就复用缓存。所以先只拷 package.json 和 lockfile、装完依赖再拷源码——改一行业务代码只让最后两层失效，依赖那层照旧命中；反过来一上来就 COPY 全部，改一个字都要重装依赖。
    3. 第二个是基础镜像钉版本。写 latest 等于让镜像在某天悄悄升到下一个大版本，可复现性当场归零，而可复现正是用容器的全部理由。
    4. 第三个是运行时配置：容器里必须监听 0.0.0.0，只听 127.0.0.1 的话它只在容器内部可达，宿主机做了端口映射也连不上——这个坑在本机跑的时候完全正常，所以特别常见。另外 EXPOSE 只是声明意图，真正开端口的是 docker run 的 -p。
    5. 第四个是安全：用非 root 用户跑业务进程（容器和宿主机共用内核，逃逸后 root 的破坏面大得多），.dockerignore 排除 node_modules（宿主机的二进制在 Linux 容器里跑不起来，还会让构建上下文暴涨）和 .env（密钥打进镜像等于发给每个能拉到镜像的人，运行时用 --env-file 传）。
    6. 可以预期的追问，也是长连接服务最该主动说的一条：CMD 要用数组形式直接起 node，让它当 PID 1。写成 pnpm start 的话 PID 1 是包管理器，docker stop 的 SIGTERM 未必传得到 node，优雅退出代码永远不执行，只能等十秒超时被 SIGKILL——对 SSE 服务，那意味着所有在途的流被硬切。

    Key points

    • Instruction order drives cache hits: copy the manifest, install, then copy source, so code edits do not reinstall dependencies
    • Pin the base image instead of latest — reproducibility is the entire point of containerizing
    • Bind 0.0.0.0 inside the container; EXPOSE only documents intent while docker run -p publishes the port
    • Run as a non-root user, and keep node_modules and .env out via .dockerignore, injecting secrets at runtime
    • Use exec-form CMD to run node as PID 1 so SIGTERM reaches it and graceful shutdown actually executes

    答题要点

    • 指令顺序决定缓存命中：先拷依赖清单装依赖，再拷源码，改代码不会触发重装依赖
    • 基础镜像钉版本不用 latest，可复现是用容器的全部理由
    • 容器里监听 0.0.0.0；EXPOSE 只是声明，真正开端口靠 docker run -p
    • 用非 root 用户运行；.dockerignore 排除 node_modules 与 .env，密钥运行时用 --env-file 注入
    • CMD 用数组形式直接起 node 让它当 PID 1，SIGTERM 才能传到进程，优雅退出才有效
  • For a long-lived SSE service in production, what problems do heartbeats, disconnect handling and graceful shutdown each solve?一个 SSE 长连接服务上线，心跳、连接断开处理和优雅退出分别在解决什么问题？
    Common in ChinaCommon overseasDeep dive#sse#reliability#deployment

    How to reason about it · think before answering

    1. The discriminator is that the three have completely different failure symptoms. Someone who can describe each symptom has shipped one; 'they all improve stability' is a non-answer.
    2. Heartbeats prevent middleboxes from killing you. Load balancers and gateways commonly close idle connections after 60 to 120 seconds, and agents are full of silent gaps while the model reasons, calls a tool or waits on a slow API. The symptom is a stream that dies halfway for no visible reason and never reproduces against a local server. Implement it as an SSE comment line, which clients silently ignore, so no client change is needed.
    3. Disconnect handling is about money. When a user closes the tab the server does not stop on its own: the model keeps generating and tokens keep billing with nobody receiving. It is the most expensive oversight in streaming services, and staging never reveals it because nobody closes tabs mid-run. Watch for the response closing, distinguish a premature close from a normal finish, and abort the upstream request.
    4. One detail must be right or it exposes you immediately: in Node listen on the response object's close, not the request's. The request emits close once its body has been read, so using it as a disconnect signal misfires on every normal request and you see streams stopping after one or two chunks.
    5. Graceful shutdown is about deploys cutting live requests. On SIGTERM the process should stop accepting new connections, give in-flight streams a short window, then exit; otherwise users watch a reply stop mid-sentence. This assumes the signal actually reaches the process — if the container's PID 1 is a package manager, SIGTERM never arrives and the runtime kills you on timeout.
    6. Expect: how long is the window? Shorter than the orchestrator's termination grace period (10s by default in Docker, 30s in Kubernetes), or you get SIGKILLed anyway; and refuse new connections immediately so the load balancer drains traffic away.

    分析过程 · 先想清楚再作答

    1. 这题的区分度在于三件事各自的失败现象完全不同，能分别说出现象的人一定真上过线。答成「都是为了稳定性」等于没答。
    2. 心跳解决的是「被中间设施误杀」。负载均衡和网关普遍有空闲超时，常见 60 到 120 秒，一段时间没有字节流动就关连接；而 Agent 天生有大量静默期——模型在思考、在调工具、在等慢接口。现象是连接莫名其妙断在一半，且本地直连时完全复现不了。实现上用 SSE 的注释行（冒号开头）做心跳，客户端会安静忽略，不用改客户端代码。
    3. 连接断开处理解决的是「花钱」。用户关掉页面之后服务端不会自动停，模型继续生成、token 继续计费，只是没人接收。这是流式服务里最贵的疏忽，而且测试环境暴露不出来，因为没人会中途关页面。做法是监听响应对象的关闭事件，判定是被掐断而不是正常收尾，就把上游请求一起中止。
    4. 这里有个必须说对的细节，说错会当场暴露没写过：Node 里要监听的是响应对象的 close，不是 request 的——request 的 close 在请求体读完时就触发，拿它当断线信号会把每一条正常请求都误判成客户端跑了，现象是每次只推出一两个片段就停。
    5. 优雅退出解决的是「发布时切断在途请求」。容器收到 SIGTERM 后应当先停止接受新连接，给在途的流一点收尾时间再退出，否则用户看到的是回复说了一半突然没了。前提是信号真的能传到进程——CMD 写成包管理器的话 PID 1 不是 node，SIGTERM 传不到，只能等超时被强杀。
    6. 可以预期的追问：收尾时间给多久？答案是要小于编排系统的终止宽限期（Docker 默认十秒、K8s 默认三十秒），超过就会被 SIGKILL，等于白设计；同时新连接要立刻拒绝，让负载均衡把流量挪走。

    Key points

    • Heartbeats defeat idle timeouts in middleboxes, since agent silence often exceeds a gateway's 60 to 120 seconds; SSE comment lines do it transparently
    • Disconnect handling stops waste: after a user closes the tab, an unaware server keeps burning tokens, and staging never shows it
    • In Node listen on the response's close, not the request's — the latter fires when the body is read and misclassifies normal requests as disconnects
    • Graceful shutdown stops deploys from cutting live streams: on SIGTERM refuse new connections and drain, within the orchestrator's grace period
    • It only works if the signal reaches the process, so PID 1 must be node itself rather than a package manager

    答题要点

    • 心跳防的是中间设施的空闲超时，Agent 的静默期常常超过网关的 60 到 120 秒，用 SSE 注释行实现，客户端无感
    • 断开处理防的是浪费：用户关页面后服务端不停就是纯烧 token，测试环境暴露不出来
    • Node 里要监听响应对象的 close 而不是 request 的——后者在请求体读完时就触发，会把正常请求误判成断线
    • 优雅退出防的是发布切断在途流：SIGTERM 后先停收新连接、给在途流收尾时间，收尾窗口要小于编排系统的终止宽限期
    • 前提是信号能传到进程：容器的 PID 1 必须是 node 本身，不能是包管理器
  • In a two-minute self-introduction, how do you convey the value of an agent project?自我介绍时，怎么在两分钟里讲清楚一个 Agent 项目的价值？
    Common in ChinaCommon overseasBasic#interview-prep#communication

    How to reason about it · think before answering

    1. There is no model answer, but there is a clear failure mode: opening with a tool list. Interviewers do not remember stacks; they remember problems and numbers.
    2. Use a fixed structure that fits two minutes: one line on who you are and where you are heading, one line on the business problem (who suffers, in what situation), three or four lines on your key technical decisions and what each bought you, and one closing line with a verifiable result.
    3. Choose decisions that involved a trade-off, not decisions that merely involved implementation. 'We stream over SSE rather than WebSocket because upstream traffic is a single request, which lets us keep existing auth, rate limiting and logging' shows you knew the alternative and priced it — far stronger than naming ten tools.
    4. Attach numbers wherever you can, even self-measured ones: time-to-first-token dropping from seconds to a few hundred milliseconds, tiered routing cutting daily spend by more than half, multi-provider fallback removing a single vendor from your availability ceiling. If the numbers are from a test environment, say so; inventing them collapses after two follow-ups.
    5. A common mistake is presenting a learning project as production. Position it yourself: a complete system built to understand production agent architecture, at self-test scale, where every decision was made against real constraints. Interviewers forgive honest scoping far more readily than inflated claims.
    6. Expect: what was the hardest part? Prepare one concrete story with a process — for example, discovering that a streaming endpoint cannot report errors by status code once it has started pushing, and redesigning around an in-stream error event plus front-loaded validation.

    分析过程 · 先想清楚再作答

    1. 这题没有标准答案，但有明确的失败模式：从技术栈开始报菜名（我用了 Fastify、SSE、Docker、向量库……）。面试官记不住工具清单，他记得住的是问题和数字。
    2. 用一条固定结构去组织，两分钟正好够：一句话说你是谁和转型方向，一句话说项目解决的业务问题（谁在什么场景下受什么苦），三到四句说你的关键技术决定和它换来了什么，最后一句给可验证的结果。
    3. 关键技术决定要挑「有取舍的」讲，不要讲「有实现的」。比如「流式用 SSE 而不是 WebSocket，因为上行只有一次，这样鉴权限流日志这套现成设施全部照用」——这种句子同时展示了你知道有别的选项、也知道选它的代价，比列出十个工具有效得多。
    4. 结果要尽量带数字，哪怕是自测数据：首字延迟从几秒降到几百毫秒、分层路由把日成本从 300 元降到 125 元、多 provider 冗余让可用性不再取决于单家厂商。没有生产数据就诚实说明是自测环境，编数字是最危险的做法，追问两句就穿帮。
    5. 常见误区是把学习项目说成生产项目。正确姿势是主动定位：这是我为了搞懂生产级 Agent 架构而完整实现的一套系统，规模是自测级，但每个决定都对着真实约束做过取舍——面试官对诚实的自评远比对夸大的描述宽容。
    6. 可以预期的追问：这个项目最难的地方是什么？提前准备一个具体的、有过程的答案（比如流式接口推流之后没法用状态码报错，最后改成流内 error 事件加上把校验全部前置），比任何形容词都有说服力。

    Key points

    • Keep a fixed structure: positioning, the business problem, three or four traded-off decisions, and one verifiable result
    • Do not recite a stack — interviewers retain problems, trade-offs and numbers, not tool lists
    • Frame decisions as trade-offs, naming the alternative and why it lost
    • Attach numbers even from self-testing, but label their source and never invent them
    • Scope the project honestly as a complete build at self-test scale; honest framing survives follow-ups better than inflation

    答题要点

    • 结构固定：定位一句、业务问题一句、三到四个有取舍的技术决定、一句可验证的结果
    • 不要报菜名：面试官记不住工具清单，记得住问题、取舍和数字
    • 技术决定要讲取舍而不是讲实现，说清楚备选方案是什么、为什么没选它
    • 结果尽量带数字，自测数据也可以，但必须标明来源，绝不编造
    • 主动定位项目规模：为搞懂生产架构而完整实现、自测级规模，诚实自评比夸大更容易通过

Comments