Packaging It as a Service: Fastify + SSE + Docker (dg P07); Week One Retrospective
Package the agent built over the previous six days into a Fastify service other frontends and services can call, streaming output over SSE and packaged with Docker, wrapping up week one.
Today's Goals
- Expose a chat endpoint that supports SSE streaming with Fastify
- Write the agent service as a Dockerfile, and build and run it successfully locally
- Summarize in one sentence the key concepts learned in week one, from the LLM API to the Pi SDK
Everything you wrote over the past six days ran the same way: type a command in your own terminal, watch the output, hit Ctrl+C, and serve exactly one person. Today it becomes something other people can call — with an address, an interface contract, a frontend able to connect to it, and an image you can carry to any machine. This is the last step of W1 and the starting point for W2's architecture split.
Plain-Language Walkthrough
From script to service: publish an interface and every assumption changes
Cooking in your own kitchen, you never think about opening hours, five tables arriving at once, or a guest ordering something that is not on any menu. The moment you open for business none of that is avoidable — your cooking did not change, the way it is used did.
A CLI script becoming a service is the same. Three unchallenged assumptions hide in the past six days of code. One user, so that history is yours by nature and nobody has to ask whose message this is. Serial execution, so two things never modify the same state at once. Trusted input, because you typed the arguments yourself and they are never a ten-megabyte string or JSON missing a field. All three fail in a service, and the first fails fastest and is hardest to find — because it behaves perfectly in single-person local testing.
There is a subtler one, and you solved a version of it only yesterday: on D6 you wrote sessions into .sessions/*.jsonl and a separate process could read them back. That road closes the moment you become a service. To handle concurrency you will run two instances, each writing its own local disk, and when the load balancer sends a user's second request to the other machine that disk has none of their history. D6 solved process restart; becoming a service introduces multiple instances, and those are not the same problem. So today we take a step back, put the history in process memory keyed by session, and get the service running. Moving it somewhere independent of any single instance is D8's job, and it is exactly the motivation for splitting gateway and worker next week.
// The script era: there was one user in the world, so a module-level array was enough
// const history = []
// With two people chatting in a service, both write into that same array and A sees B's conversation
// The right way: isolate by session
const histories = new Map()
function historyOf(sessionId) {
if (!histories.has(sessionId)) histories.set(sessionId, [])
return histories.get(sessionId)
}
// Note: the Map is only a step-back stopgap. D6's local file fails across instances,
// and D8 replaces this with Postgresfrom collections import defaultdict
# defaultdict removes the "create it if it does not exist" boilerplate
histories: defaultdict[str, list[dict]] = defaultdict(list)
def history_of(session_id: str) -> list[dict]:
return histories[session_id]
# Reminder: with a multi-worker deployment each process holds its own copy of this dict,
# so the same user's second request landing on another worker finds no history// A server is multi-threaded by nature: a plain HashMap loses data under concurrent writes,
// so use ConcurrentHashMap plus computeIfAbsent to get-or-create atomically in one step
static final Map<String, List<Message>> HISTORIES = new ConcurrentHashMap<>();
static List<Message> historyOf(String sessionId) {
return HISTORIES.computeIfAbsent(sessionId,
key -> Collections.synchronizedList(new ArrayList<>()));
}// The idiomatic Swift answer is an actor: the compiler guarantees only one task mutates
// this state at a time, so you neither add a lock nor can forget to
actor SessionStore {
private var histories: [String: [Message]] = [:]
func history(for sessionID: String) -> [Message] {
histories[sessionID] ?? []
}
func append(_ message: Message, to sessionID: String) {
histories[sessionID, default: []].append(message)
}
}Four versions solve one problem, and the languages differ in how dangerous their defaults are: JavaScript's single thread lets you luck out of part of the concurrency problem, Java forces you to choose a concurrent container explicitly, and Swift's actor forbids the mistake at the type level. When you build a service in a new language, the first question to ask is who owns concurrency safety.
Session isolation is only the entry ticket. Interface design still has four decisions: the shape of the endpoint (return one complete JSON, or push as you generate), how a session is identified (the client carries a sessionId, or the server issues a cookie), auth and rate limiting, and how errors are expressed.
The last one is the most overlooked and the most damaging: a streaming endpoint cannot express an error with an HTTP status code. Once the 200 and the first byte are out, the status code is already on the wire; a subsequent model timeout, insufficient balance, or upstream 500 can only be reported as an agreed error event inside the stream. So the interface documentation has to name the three kinds of event — normal, finished, failed — and you must validate everything validatable before you start streaming, because that is your last chance to speak in status codes.
Once the contract is settled, the genuinely hard part is that long-lived connection itself: get the response headers wrong and the browser refuses it outright, sit idle for two minutes and a gateway silently cuts it, and if the user closes the page you keep burning money generating tokens for them. Let us work through it, starting with bringing the service up.
Interview questions
For streaming LLM responses, would you pick SSE or WebSocket, and why?流式返回大模型回复,你会选 SSE 还是 WebSocket?为什么?
Common in ChinaCommon overseasBasic#sse#streaming#api-design🔒 Unlock this chapter to see the answer keys.
When turning a local agent script into a production service, what does the interface layer have to get right?把一个本地跑的 Agent 脚本改造成生产服务,接口层要重点考虑哪些事?
Common in ChinaCommon overseasIntermediate#api-design#service-architecture#streaming🔒 Unlock this chapter to see the answer keys.
What are the key decisions in a Dockerfile that packages a Node service?把一个 Node 服务打包成 Docker 镜像,Dockerfile 里有哪些关键决定?
Common in ChinaCommon overseasIntermediate#docker#deployment#nodejs🔒 Unlock this chapter to see the answer keys.
For a long-lived SSE service in production, what problems do heartbeats, disconnect handling and graceful shutdown each solve?一个 SSE 长连接服务上线,心跳、连接断开处理和优雅退出分别在解决什么问题?
Common in ChinaCommon overseasDeep dive#sse#reliability#deployment🔒 Unlock this chapter to see the answer keys.
In a two-minute self-introduction, how do you convey the value of an agent project?自我介绍时,怎么在两分钟里讲清楚一个 Agent 项目的价值?
Common in ChinaCommon overseasBasic#interview-prep#communication🔒 Unlock this chapter to see the answer keys.
