Dayward AIWeChat Mini Program
From Frontend Engineer to Agent Engineer in 30 Days
Week 2 · D8🔒 LockedAbout 6 hours

Why Split Gateway and Worker; Postgres Table Design (sessions/runs/messages) + Drizzle

Understand why a production-grade agent splits its intake layer from its execution layer, and use Drizzle to create mini-koda's first batch of tables.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Draw a three-part gateway / message-bus / worker architecture and state each part's responsibility
  2. Define the sessions, runs, and messages tables with Drizzle and generate a migration
  3. Explain why a stateless gateway and idempotent writes are prerequisites for horizontal scaling

Yesterday closed on a promise: the in-memory session Map becomes three Postgres tables today — sessions, runs, messages. But today is not only a change of storage. First we split D7's carry-everything service down the middle, and then we decide where the state goes. This is the foundation for everything else this week.

Plain-Language Walkthrough

A check-in desk does not stop because one aircraft loads slowly

An airport check-in desk does remarkably little: verify the document, weigh the bag, print the boarding pass, push the bag onto the belt. Thirty seconds per passenger. The agent does not care what time your case reaches the hold, or whether that aircraft is still waiting for a tug. The heavy work happens on the apron, where the loading crew carries cartloads of bags into the hold and can grind on for half an hour when things go badly. The two sides are joined by one baggage belt.

The entire value of that division is one sentence: the desk's throughput is unaffected by the apron's pace. Even if one aircraft's loading goes badly wrong, the queue still advances one passenger every thirty seconds.

D7's service is precisely the opposite: one process is both the desk and the loading crew. It runs beautifully on your machine and has three ways to die in production.

First, a restart loses every in-flight execution. D6 already wrote sessions to a local file, which persisted what was said; but that file holds nothing about how far the currently running execution got. Restart the service and there is not even a body to find, and the user waits for a reply that will never come. D6 solved history; today solves execution itself, and they are not the same problem.

Second, two instances each store their own. You run two instances to handle concurrency, the load balancer sends the user's second request to the other one, its Map is empty, and the agent asks blankly what they were just saying.

Third, and subtlest: one slow request drags down the whole machine. An agent loop with tool calls in it taking thirty seconds is unremarkable, and for those thirty seconds it holds a connection, memory, and a buffer for the tens of thousands of tokens in its context. Run a dozen of those on one machine at once and incoming health checks start timing out; the orchestrator declares the machine dead, pulls it out and restarts it — taking those dozen in-flight executions down with it.

So the first test for a production-grade agent service is: anything that can be done in the worker should not sit in the gateway. The gateway does four things only: authenticate, rate-limit, persist, deliver. What those four share is a bounded cost measured in milliseconds, which does not stretch because the model is slow today.

gateway.js
// The gateway's POST /chat: do the four things and return, never run the agent loop here
app.post('/chat', async (request, reply) => {
  const { sessionId, message, clientMessageId } = request.body
  if (!sessionId || !message) {
    return reply.code(400).send({ error: 'sessionId and message are required' })
  }
 
  // 1. auth and 2. rate limiting are handled by upstream plugins, leaving 3. persist
  const key = idempotencyKeyFor(sessionId, clientMessageId, message)
  const run = await store.createRun({ sessionId, input: message, idempotencyKey: key })
 
  // 4. deliver to the message bus - empty today; D9 wires the bus up
  reply.code(202) // 202 Accepted: I took it and I am working on it, not 200 "here is the result"
  return { runId: run.id, status: run.status }
})

Notice the status code. 200 means the work is finished and here is the result; 202 means I took it and I am working on it — after the split the gateway holds no result, so 202 is all it can honestly say. That is not pedantry, it decides how the frontend is written: having received a 202 it must take the runId and subscribe for the result, so one request becomes submit plus subscribe.

That is the split's bill: you pay one extra round trip and one subscription endpoint, and the desk never queues. Whether it is worth it depends on whether one execution of your agent takes three hundred milliseconds or thirty seconds. Do not split a three-hundred-millisecond scenario; and the moment you use tool calls, thirty seconds is normal.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • Why do production agent services usually split a gateway from workers, and when should you not split?为什么生产级 Agent 服务通常要把 Gateway 和 Worker 拆开?什么情况下不该拆?
    Common in ChinaCommon overseasBasic#architecture#scalability

    🔒 Unlock this chapter to see the answer keys.

  • What makes a service stateless, what does that mean for horizontal scaling, and are workers stateful?什么是无状态服务?它对水平扩展意味着什么?Worker 算不算有状态?
    Common in ChinaCommon overseasIntermediate#stateless#scalability

    🔒 Unlock this chapter to see the answer keys.

  • With at-least-once delivery, how do you guarantee a redelivered message does not create two runs?消息总线是至少一次投递,同一条消息被重复投递时,怎么保证不会产生两条 run?
    Common in ChinaCommon overseasDeep dive#idempotency#database#reliability

    🔒 Unlock this chapter to see the answer keys.

  • How would you design primary keys and indexes for sessions, runs and messages, and why avoid auto-increment ids?sessions / runs / messages 这三张表你会怎么设计主键与索引?为什么不用自增主键?
    Common in ChinaCommon overseasIntermediate#database#schema-design#idempotency

    🔒 Unlock this chapter to see the answer keys.

Comments