Dynamic Routing With a Supervisor: Structured-Output Routing, Override, routingReason
Implement a supervisor node that uses structured output to dynamically decide which subagent to route to, and support human override plus recording the reason behind each routing decision.
Today's Goals
- Implement a supervisor node that outputs a routing decision as structured output
- Add a routingReason to the routing result to make debugging and retrospectives easier
- Implement an override mechanism for a human to correct the model when it routes incorrectly
Yesterday's graph contained one obvious fiction: all three edges hard-wired, with the same handler always following the entrance. Today one of those edges gets its destination decided by the model at runtime, and we think through "what if it decides wrong" once and for all.
Plain-Language Walkthrough
A dispatch desk does not work the case, it decides whose case it is
When you call an emergency line, the person answering does not chase the thief or fight the fire. They do one thing: hear your first two sentences, decide whether this is police, traffic, or fire, and pass the ticket to the matching department. The whole call may last twenty seconds, and those twenty seconds decide everybody else's workload โ send it to the wrong department and the responders discover the mistake on arrival, it gets transferred again, and all the time went into travel.
That is the Supervisor pattern: one node dedicated to deciding who should take this case, handing it off and doing no work itself. In our e-commerce support scenario there are three takers, and their names do not change this week: order_lookup for order status and shipping progress, refund_draft to draft a refund proposal by the rules without executing a refund, and smalltalk for chat and as the catch-all.
Why make the decision its own node rather than letting the order-lookup agent judge for itself? Because deciding and executing are two different kinds of work. Deciding needs the whole picture, speed, and cheapness, usually in a few dozen tokens; executing needs detail, tools, and a pile of rules. Mixed together you can neither swap a smaller model in for the decision alone nor measure triage accuracy separately โ and that metric happens to be the most worthwhile one to watch in a multi-agent system.
Technically the Supervisor rests on a conditional edge: this edge's destination is unknown at compile time and is decided at runtime by a selector function. Yesterday's three unconditional edges hard-wired the order; today's edge hands "where next" to a field in the state.
import { StateGraph, START, END } from '@langchain/langgraph'
// The supervisor writes two fields only: who takes it (route) and why (routingReason).
// It does not answer the user's question
const supervisor = async (state) => decide(lastText(state))
// The selector reads state and calls no model: the decision was already made inside
// supervisor, and this only translates it into a node name
const selectRoute = (state) => state.route ?? 'smalltalk'
export const graph = new StateGraph(AgentAnnotation)
.addNode('supervisor', supervisor)
.addNode('order_lookup', orderLookup)
.addNode('refund_draft', refundDraft)
.addNode('smalltalk', smalltalk)
.addEdge(START, 'supervisor')
// The third argument maps the selector's return value to a node name. Write it out in
// full: returning an unregistered name is a compile-time error, and a registered node
// nobody points at makes compile() throw UnreachableNodeError
.addConditionalEdges('supervisor', selectRoute, {
order_lookup: 'order_lookup',
refund_draft: 'refund_draft',
smalltalk: 'smalltalk',
})
.addEdge('order_lookup', END)
.addEdge('refund_draft', END)
.addEdge('smalltalk', END)
.compile()from langgraph.graph import StateGraph, START, END
# The supervisor writes two fields only: who takes it (route) and why (routing_reason)
async def supervisor(state: AgentState) -> dict:
return decide(last_text(state))
# The selector reads state and calls no model: it only translates into a node name
def select_route(state: AgentState) -> str:
return state.get("route") or "smalltalk"
graph = (
StateGraph(AgentState)
.add_node("supervisor", supervisor)
.add_node("order_lookup", order_lookup)
.add_node("refund_draft", refund_draft)
.add_node("smalltalk", smalltalk)
.add_edge(START, "supervisor")
# The third argument maps the selector's return value to a node name; write it in full
.add_conditional_edges(
"supervisor",
select_route,
{"order_lookup": "order_lookup", "refund_draft": "refund_draft", "smalltalk": "smalltalk"},
)
.add_edge("order_lookup", END)
.add_edge("refund_draft", END)
.add_edge("smalltalk", END)
.compile()
)// Java has no LangGraph, so this hand-writes the same graph idiomatically for JDK 17+:
// a node is a function and a conditional edge is one table lookup.
// Dependencies: JDK 17+, no third-party library
enum Route { ORDER_LOOKUP, REFUND_DRAFT, SMALLTALK }
// An EnumMap's key space is the enum itself, and with the self-check below a missing
// branch prevents startup rather than crashing halfway through
static final Map<Route, UnaryOperator<AgentState>> AGENTS = new EnumMap<>(Map.of(
Route.ORDER_LOOKUP, Graph::orderLookup,
Route.REFUND_DRAFT, Graph::refundDraft,
Route.SMALLTALK, Graph::smalltalk));
static {
for (Route r : Route.values())
if (!AGENTS.containsKey(r)) throw new IllegalStateException("route " + r + " has no subagent");
}
static AgentState run(AgentState input) {
AgentState routed = supervisor(input); // decides who takes it, answers nothing
return AGENTS.get(routed.route()).apply(routed); // this one lookup is the conditional edge
}// Swift has no LangGraph either. This hand-writes the same graph with an enum and a switch:
// a switch over an enum is exhaustive, so a missing branch **does not compile** - the
// biggest benefit of the Swift version of a conditional edge
enum Route: String {
case orderLookup = "order_lookup"
case refundDraft = "refund_draft"
case smalltalk
}
func nextAgent(for route: Route) -> (AgentState) async throws -> AgentState {
switch route {
case .orderLookup: return orderLookup
case .refundDraft: return refundDraft
case .smalltalk: return smalltalk
}
}
func run(_ input: AgentState) async throws -> AgentState {
let routed = try await supervisor(input) // decides who takes it, answers nothing
return try await nextAgent(for: routed.route)(routed)
}All four teach one thing: turn "where next" from a hard-wired order in the code into one runtime table lookup. The mechanics are now covered, and everything remaining is the genuinely hard question โ what goes inside decide. Can you just have the model say "I think the order-lookup colleague should see this" and parse that sentence?
Interview questions
How is the routing decision usually implemented in a supervisor pattern? What should that node do, and what should it not do?Supervisor ๆจกๅผ้็่ทฏ็ฑๅณ็ญไธ่ฌๆไนๅฎ็ฐ๏ผ่ฏท่ฏด่ฏด่ฟไธช่็น่ฏฅๅไปไนใไธ่ฏฅๅไปไนใ
Common in ChinaCommon overseasBasic#multi-agent#routing#langgraph๐ Unlock this chapter to see the answer keys.
Why use structured output rather than natural language for routing? What exactly goes wrong with free text?ไธบไปไน่ฆ่ฎฉๆจกๅ่พๅบ structured output ่ไธๆฏ่ช็ถ่ฏญ่จๆฅๅ่ทฏ็ฑ๏ผ่ช็ถ่ฏญ่จๅฐๅบๅทฎๅจๅช๏ผ
Common in ChinaCommon overseasIntermediate#structured-output#routing#reliability๐ Unlock this chapter to see the answer keys.
How should the system handle an uncertain or wrong routing decision, and how do you pick the threshold?่ทฏ็ฑไธ็กฎๅฎๆ่ ่ทฏ็ฑ้่ฏฏๆถ๏ผ็ณป็ปๅบ่ฏฅๆไนๅ ๅบ๏ผ้ๅผ่ฏฅๆไนๅฎ๏ผ
Common in ChinaCommon overseasDeep dive#routing#fallback#reliability๐ Unlock this chapter to see the answer keys.
What is a field like routingReason actually worth in production? Is it just logging?routingReason ่ฟ็ฑป่ฐ่ฏไฟกๆฏๅจ็ไบง็ณป็ป้ๆไปไนไปทๅผ๏ผๅชๆฏๆๆฅๅฟ่ๅทฒๅ๏ผ
Common in ChinaCommon overseasIntermediate#observability#routing#debugging๐ Unlock this chapter to see the answer keys.
