Dayward AIWeChat Mini Program
From Frontend Engineer to Agent Engineer in 30 Days
Week 2 · D10🔒 LockedAbout 6 hours

Sharding and Leases: Hashing userId → shard, SET NX + TTL + Lua Renewal, Per-User Ordering, Handoff

Use hash-based sharding to spread user traffic across multiple workers, then implement lease renewal with Redis's SET NX + TTL + a Lua script, guaranteeing a single user's messages are processed in strict order.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Implement a function that hashes a userId to a fixed number of shards
  2. Acquire a shard's lease with SET NX + TTL, and safely renew it with a Lua script
  3. Explain how a shard is handed off to another worker once its lease expires, without disturbing one user's message order

Yesterday's message bus works, and it has one glaring side effect: a consumer group hands a message to any idle worker, so two sentences a user sends in a row may be processed by two processes at once. Today we tackle that head on.

Plain-Language Walkthrough

Hand out parcels at random and one user's two sentences land on two people

Put yesterday's leftover problem on the table. One stream, one consumer group, three workers taking from the same shelf — that sharing is fine for unrelated tasks, and our tasks happen to be related.

The scenario is e-commerce support. The user sends "I want a refund," then two seconds later "order number A1024." Both sit in the stream almost simultaneously, the group hands the first to worker-A and the second to worker-B, and which finishes first depends on which machine was less loaded. So the user quite likely sees "please provide your order number" first and "your refund has been logged" second — having already given the number.

The root is not in the bus, it is in the unit of assignment: a consumer group assigns one message at a time, while the business's minimum serial unit is one user.

Everyday life has a ready-made solution: precinct-based city services. The city does not send whichever inspector is free to the next repair report; it divides the city into a fixed number of precincts, each held by one inspector carrying a badge — so anything in one block always reaches the same person.

A precinct is a shard, and this course fixes it at 256. A user id hashes and takes a modulus to land on a shard, the shard belongs to a worker, and therefore all of one user's messages always queue onto the same worker's same queue, where the order is naturally the enqueue order.

TextText
user u-1001 --shardOf(userId)--) shard 68 --lease--) worker-A
user u-2077 --shardOf(userId)--) shard 12 --lease--) worker-B
user u-3310 --shardOf(userId)--) shard 68 --lease--) worker-A (same block, same person)

Why not simply take userId modulo the worker count? Because the worker count changes, and a change of divisor moves nearly every user's assignment, migrating whole sessions. Interposing a fixed 256 shards nails down user-to-shard and lets only shard-to-worker float with scaling.

One architectural change has to be declared, or you will get stuck writing this on top of yesterday's code: sharding also changes the shape of the stream. A consumer group's job is precisely to spread messages across any idle consumer, and there is no such thing as "read only messages belonging to my shards." So from today koda:runs splits into 256 substreams, koda:runs:s0 through koda:runs:s255: the gateway computes the shard and XADDs to the matching substream, a worker reads only the ones it holds, and the consumer group degrades into one consumer per substream, with the sharing taken over by leases. The price is 1 stream becoming 256, so backlog alerts have to aggregate across substreams and XAUTOCLAIM runs once per held substream. Yesterday was not wrong, the requirement changed — D9's sharing was designed for unrelated tasks.

The hash function holds no mystery, but one requirement cannot be relaxed: the same userId must compute the same shard in any process, any language, and after any restart. So use a digest function, not the language's built-in string hash.

shard.js
import { createHash } from 'node:crypto'
 
export const SHARD_COUNT = 256 // a constant; changing it is a data migration
 
export function shardOf(userId) {
  // A digest function is stable by nature: the same input gives the same result across
  // processes, machines, and languages
  const digest = createHash('sha1').update(userId, 'utf8').digest()
  return digest.readUInt32BE(0) % SHARD_COUNT
}

At this point "one user always belongs to one block" holds. But blocks are fixed and people are not — where is "who owns which block" recorded, and what guarantees only one person owns it at a time? That is today's genuinely hard part.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • Why hash user ids into shards instead of letting the consumer group dispatch freely, and how do you pick the shard count?为什么要对 userId 做哈希分片,而不是让消费组随机派发?分片数应该怎么选?
    Common in ChinaCommon overseasBasic#sharding#consistent-hashing#scalability

    🔒 Unlock this chapter to see the answer keys.

  • Why must a lease carry a TTL, and why renew it with a Lua script instead of GET followed by PEXPIRE?租约为什么必须配合 TTL?续约为什么要用 Lua 脚本,而不是先 GET 再 PEXPIRE?
    Common in ChinaCommon overseasIntermediate#lease#redis#atomicity

    🔒 Unlock this chapter to see the answer keys.

  • What happens when two workers both believe they hold the same shard lease (split brain), and how do you mitigate it?两个 worker 同时认为自己持有同一个 shard 的租约(脑裂)会造成什么后果,怎么规避?
    Common in ChinaCommon overseasDeep dive#split-brain#fencing-token#reliability

    🔒 Unlock this chapter to see the answer keys.

  • In a multi-worker agent service, how do you guarantee that one user's messages are processed in strict order?在一个多 worker 的 Agent 服务里,怎么保证同一个用户的消息严格按顺序被处理?
    Common in ChinaCommon overseasIntermediate#ordering#sharding#distributed-systems

    🔒 Unlock this chapter to see the answer keys.

Comments