Why Split Gateway and Worker; Postgres Table Design (sessions/runs/messages) + Drizzle
Understand why a production-grade agent splits its intake layer from its execution layer, and use Drizzle to create mini-koda's first batch of tables.
Today's Goals
- Draw a three-part gateway / message-bus / worker architecture and state each part's responsibility
- Define the sessions, runs, and messages tables with Drizzle and generate a migration
- Explain why a stateless gateway and idempotent writes are prerequisites for horizontal scaling
Yesterday closed on a promise: the in-memory session Map becomes three Postgres tables today — sessions, runs, messages. But today is not only a change of storage. First we split D7's carry-everything service down the middle, and then we decide where the state goes. This is the foundation for everything else this week.
Plain-Language Walkthrough
A check-in desk does not stop because one aircraft loads slowly
An airport check-in desk does remarkably little: verify the document, weigh the bag, print the boarding pass, push the bag onto the belt. Thirty seconds per passenger. The agent does not care what time your case reaches the hold, or whether that aircraft is still waiting for a tug. The heavy work happens on the apron, where the loading crew carries cartloads of bags into the hold and can grind on for half an hour when things go badly. The two sides are joined by one baggage belt.
The entire value of that division is one sentence: the desk's throughput is unaffected by the apron's pace. Even if one aircraft's loading goes badly wrong, the queue still advances one passenger every thirty seconds.
D7's service is precisely the opposite: one process is both the desk and the loading crew. It runs beautifully on your machine and has three ways to die in production.
First, a restart loses every in-flight execution. D6 already wrote sessions to a local file, which persisted what was said; but that file holds nothing about how far the currently running execution got. Restart the service and there is not even a body to find, and the user waits for a reply that will never come. D6 solved history; today solves execution itself, and they are not the same problem.
Second, two instances each store their own. You run two instances to handle concurrency, the load balancer sends the user's second request to the other one, its Map is empty, and the agent asks blankly what they were just saying.
Third, and subtlest: one slow request drags down the whole machine. An agent loop with tool calls in it taking thirty seconds is unremarkable, and for those thirty seconds it holds a connection, memory, and a buffer for the tens of thousands of tokens in its context. Run a dozen of those on one machine at once and incoming health checks start timing out; the orchestrator declares the machine dead, pulls it out and restarts it — taking those dozen in-flight executions down with it.
So the first test for a production-grade agent service is: anything that can be done in the worker should not sit in the gateway. The gateway does four things only: authenticate, rate-limit, persist, deliver. What those four share is a bounded cost measured in milliseconds, which does not stretch because the model is slow today.
// The gateway's POST /chat: do the four things and return, never run the agent loop here
app.post('/chat', async (request, reply) => {
const { sessionId, message, clientMessageId } = request.body
if (!sessionId || !message) {
return reply.code(400).send({ error: 'sessionId and message are required' })
}
// 1. auth and 2. rate limiting are handled by upstream plugins, leaving 3. persist
const key = idempotencyKeyFor(sessionId, clientMessageId, message)
const run = await store.createRun({ sessionId, input: message, idempotencyKey: key })
// 4. deliver to the message bus - empty today; D9 wires the bus up
reply.code(202) // 202 Accepted: I took it and I am working on it, not 200 "here is the result"
return { runId: run.id, status: run.status }
})# The gateway's POST /chat: do the four things and return, never run the agent loop here
@app.post("/chat", status_code=202) # 202 Accepted: I took it and I am working on it
async def chat(body: ChatRequest) -> ChatAccepted:
# FastAPI validates the body with a pydantic model, so a missing field is a 422
# without an if statement of your own
# 1. auth and 2. rate limiting go to dependencies and middleware, leaving 3. persist
key = idempotency_key_for(body.session_id, body.client_message_id, body.message)
run = await store.create_run(
session_id=body.session_id, input=body.message, idempotency_key=key
)
# 4. deliver to the message bus - empty today; D9 wires the bus up
return ChatAccepted(run_id=run.id, status=run.status)// Dependencies: spring-boot-starter-web. @Valid hands body validation to Bean Validation
@PostMapping("/chat")
@ResponseStatus(HttpStatus.ACCEPTED) // 202: I took it and I am working on it, not 200 "done"
public ChatAccepted chat(@Valid @RequestBody ChatRequest body) {
// 1. auth and 2. rate limiting are done by a Filter and Spring Security, leaving 3. persist
var key = Runs.idempotencyKey(body.sessionId(), body.clientMessageId(), body.message());
var run = store.createRun(body.sessionId(), body.message(), key);
// 4. deliver to the message bus - empty today; D9 wires the bus up
return new ChatAccepted(run.getId(), run.getStatus());
}// Dependencies: Vapor 4. Content plus Validatable leaves body validation to the framework
app.post("chat") { req async throws -> Response in
try ChatRequest.validate(content: req)
let body = try req.content.decode(ChatRequest.self)
// 1. auth and 2. rate limiting are done by middleware, leaving 3. persist
let key = idempotencyKey(for: body.sessionId, clientMessageId: body.clientMessageId,
message: body.message)
let run = try await store.createRun(sessionId: body.sessionId, input: body.message,
idempotencyKey: key)
// 4. deliver to the message bus - empty today; D9 wires the bus up
// 202 Accepted: I took it and I am working on it, not 200 "done"
return try await ChatAccepted(runId: run.requireID(), status: run.status)
.encodeResponse(status: .accepted, for: req)
}Notice the status code. 200 means the work is finished and here is the result; 202 means I took it and I am working on it — after the split the gateway holds no result, so 202 is all it can honestly say. That is not pedantry, it decides how the frontend is written: having received a 202 it must take the runId and subscribe for the result, so one request becomes submit plus subscribe.
That is the split's bill: you pay one extra round trip and one subscription endpoint, and the desk never queues. Whether it is worth it depends on whether one execution of your agent takes three hundred milliseconds or thirty seconds. Do not split a three-hundred-millisecond scenario; and the moment you use tool calls, thirty seconds is normal.
Interview questions
Why do production agent services usually split a gateway from workers, and when should you not split?为什么生产级 Agent 服务通常要把 Gateway 和 Worker 拆开?什么情况下不该拆?
Common in ChinaCommon overseasBasic#architecture#scalability🔒 Unlock this chapter to see the answer keys.
What makes a service stateless, what does that mean for horizontal scaling, and are workers stateful?什么是无状态服务?它对水平扩展意味着什么?Worker 算不算有状态?
Common in ChinaCommon overseasIntermediate#stateless#scalability🔒 Unlock this chapter to see the answer keys.
With at-least-once delivery, how do you guarantee a redelivered message does not create two runs?消息总线是至少一次投递,同一条消息被重复投递时,怎么保证不会产生两条 run?
Common in ChinaCommon overseasDeep dive#idempotency#database#reliability🔒 Unlock this chapter to see the answer keys.
How would you design primary keys and indexes for sessions, runs and messages, and why avoid auto-increment ids?sessions / runs / messages 这三张表你会怎么设计主键与索引?为什么不用自增主键?
Common in ChinaCommon overseasIntermediate#database#schema-design#idempotency🔒 Unlock this chapter to see the answer keys.
