Getting Started With the Pi SDK: the Three-Layer Architecture, Comparing It to the Agent Loop (dg P01/P02/M02/M03)

Today's Goals

1. State what each layer of the Pi SDK's three-layer architecture is responsible for
2. Rewrite day two's tool-calling agent with the Pi SDK, with noticeably less code
3. Compare the hand-written agent loop against the Pi SDK's built-in agent loop

After hand-writing that agent loop yesterday you probably came away with a feeling: this code has almost nothing to do with any particular business. Swap the tools, swap the task, and it still looks like that. So today we hand the code that is identical inside every agent over to a framework, and then reconcile it line by line: which of your lines went where, who does what on your behalf, and what you gave up in exchange. When you are done, come back to the top and tick off the three goals.

Plain-Language Walkthrough

Bare shell or move-in ready: what a framework is actually selling

There are two ways to take delivery of an apartment. Bare shell: wiring runs, plumbing, walls, and floors are all yours, three months of work, and you deal with seven or eight tradespeople along the way. The upside is that you watched every pipe go in, so when you later want to drill into a wall you know what is behind it. Move-in ready: sign the papers, carry your bags in, sleep there within a week. The price is that you never saw what is inside the walls, so moving one socket means first working out how the developer laid it out — and you may find you cannot move it at all.

That is exactly the relationship between a framework and hand-written code. Yesterday you built the bare shell: model calls, the loop, tool dispatch, feeding results back, the round guard — you laid every brick. Today you move into the finished apartment, but because you built one yourself once, you get to be the owner who understands what is behind the plaster.

So what is in the finished package? Pull up yesterday's code and it separates cleanly into two piles:

• The part tied to your business: how the two tool manuals are worded, what execute looks up or computes, how the sentence shown to the user is assembled. Move to another project and all of it gets rewritten.
• The part with nothing to do with your business: issuing the HTTP request, appending the reply to the history, reading the stop reason to decide whether to continue, mapping a tool name to a function, wrapping a result into a message and pushing it back, counting rounds to prevent a runaway. Move to another project and not one word of it changes.

The second pile is what the framework sells. It gathers that generic logic into a kernel and exposes only a handful of functions: register a tool, open a session, send a message, subscribe to events. Your code stops driving a machine and starts fitting parts into one.

State the engineering cost right now: the part you did not write did not disappear, it turned into defaults somebody else chose for you. Which model, what the system prompt says, how many rounds the loop may run, what happens when a tool errors — yesterday each of those was a visible line in your file, and today all of them live inside the kernel. They do not cease to exist because you cannot see them; they simply change value quietly during some framework upgrade, production behavior shifts with them, and your code never moved. That is the wiring behind the plaster, and it is why this chapter insists on a line-by-line reconciliation instead of copying an API out of the docs.

Pi's three layers: model, kernel, application

Pi is an Agent SDK written in TypeScript, and it cuts that kernel into three layers, each an independent npm package, with dependencies strictly one-directional downward:

@earendil-works/pi-coding-agent   application: sessions, resource loading, built-in tools, four run modes
        |  depends on
@earendil-works/pi-agent-core     kernel: the agent loop, tool execution, state management, the event stream
        |  depends on
@earendil-works/pi-ai             model: unified model calls, auth, model discovery, cost accounting

The bottom layer is the model-calling layer. It does exactly what your hand-written fetch did on D1, except it absorbs every vendor difference: request body shape, auth scheme, stream framing, the field names for tool calls all converge on one interface, and it tallies tokens and spend while it is at it. This layer also carries one hard constraint worth remembering — it only admits models that support tool calling, because a model that cannot call a tool is unusable in an agent setting.

The middle is the agent kernel. Built on the model layer, it owns precisely the loop you hand-wrote yesterday: take the model's reply, read the stop reason, run the tool, feed the result back, decide whether to go around again — and broadcast that whole process live as events. This is the layer this course actually wants you to read, because it is what your code from yesterday looks like once abstracted.

The top is the application layer. It addresses one concrete scenario, a coding agent in a terminal: how a session is stored and resumed, which directories extensions and skills load from, which tools ship built in, and four run modes — interactive, print, inter-process, and embedded SDK. Today we use only the last one, treating it as a library embedded in our own program.

Why cut it this way? Because each layer can be swapped out on its own and tested on its own. If all you want is a unified model-calling layer and you will write the loop yourself, install only the bottom. If you want the full agent loop but not the terminal interaction, stop at the middle. That is also a general test of whether a framework is any good: can its layering let you take half? Conversely, a framework you must swallow whole will eventually charge you for the half you never wanted.

The layering has one very practical benefit too: when something breaks, your first job is deciding which layer it broke in. A stack trace naming the model layer's package usually means auth, a model id, or a request-format problem; one naming the kernel means the loop or tool execution. The two investigations go in completely different directions, and that judgment comes from the three-layer picture you are memorizing right now in five minutes.

Line by line: where did every line you wrote yesterday go

First shrink yesterday's skeleton into its smallest form. It is the left half of the comparison table:

// D2's hand-written version: the loop, the dispatch, the refill, every line in your own file
const MAX_STEPS = 6 // same ceiling as the one in D2's text
 
async function runAgent(messages) {
  for (let step = 0; step < MAX_STEPS; step++) {
    const res = await fetch('https://openrouter.ai/api/v1/chat/completions', {
      method: 'POST',
      headers: { Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}` },
      body: JSON.stringify({ model: 'openai/gpt-4o-mini', messages, tools: TOOL_SCHEMAS }),
    })
    const choice = (await res.json()).choices[0]
    messages.push(choice.message) // the history is yours to maintain
 
    // the anchor D2 left behind: the stop reason decides whether the loop continues
    const stopReason = choice.finish_reason
    if (stopReason !== 'tool_calls') return choice.message.content
 
    for (const call of choice.message.tool_calls) {
      const args = JSON.parse(call.function.arguments) // argument validation is yours too
      let result
      try {
        result = await TOOL_IMPLS[call.function.name](args) // manual dispatch
      } catch (err) {
        result = `Tool failed: ${err.message}` // you wrap the error into a message yourself
      }
      messages.push({ role: 'tool', tool_call_id: call.id, content: String(result) })
    }
  }
  throw new Error('Exceeded the maximum number of steps; it may be stuck in a loop')
}

# D2's hand-written version: the loop, the dispatch, the refill, every line in your own file
MAX_STEPS = 6  # same ceiling as the one in D2's text
 
 
def run_agent(messages: list[dict]) -> str:
    for _ in range(MAX_STEPS):
        res = httpx.post(
            "https://openrouter.ai/api/v1/chat/completions",
            headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
            json={"model": "openai/gpt-4o-mini", "messages": messages, "tools": TOOL_SCHEMAS},
        )
        choice = res.json()["choices"][0]
        messages.append(choice["message"])  # the history is yours to maintain
 
        # the anchor D2 left behind: the stop reason decides whether the loop continues
        stop_reason = choice["finish_reason"]
        if stop_reason != "tool_calls":
            return choice["message"]["content"]
 
        for call in choice["message"]["tool_calls"]:
            args = json.loads(call["function"]["arguments"])  # argument validation is yours too
            try:
                result = TOOL_IMPLS[call["function"]["name"]](**args)  # manual dispatch
            except Exception as err:
                result = f"Tool failed: {err}"  # you wrap the error into a message yourself
            messages.append(
                {"role": "tool", "tool_call_id": call["id"], "content": str(result)}
            )
    raise RuntimeError("Exceeded the maximum number of steps; it may be stuck in a loop")

// Dependencies: java.net.http (JDK 11+) plus Jackson. D2's hand-written version,
// every line in your own file
static final int MAX_STEPS = 6; // same ceiling as the one in D2's text
 
static String runAgent(ArrayNode messages) throws Exception {
    for (int step = 0; step < MAX_STEPS; step++) {
        JsonNode choice = postChat(messages).at("/choices/0");
        messages.add(choice.get("message")); // the history is yours to maintain
 
        // the anchor D2 left behind: the stop reason decides whether the loop continues
        var stopReason = choice.at("/finish_reason").asText();
        if (!"tool_calls".equals(stopReason)) {
            return choice.at("/message/content").asText();
        }
 
        for (JsonNode call : choice.at("/message/tool_calls")) {
            var name = call.at("/function/name").asText();
            var args = MAPPER.readTree(call.at("/function/arguments").asText()); // validate yourself
            String result;
            try {
                result = TOOL_IMPLS.get(name).apply(args); // manual dispatch
            } catch (Exception err) {
                result = "Tool failed: " + err.getMessage(); // wrap the error yourself
            }
            var toolMessage = MAPPER.createObjectNode();
            toolMessage.put("role", "tool");
            toolMessage.put("tool_call_id", call.at("/id").asText());
            toolMessage.put("content", result);
            messages.add(toolMessage);
        }
    }
    throw new IllegalStateException("Exceeded the maximum number of steps; it may be stuck in a loop");
}

// D2's hand-written version: the loop, the dispatch, the refill, every line in your own file
let maxSteps = 6 // same ceiling as the one in D2's text
 
func runAgent(messages: inout [ChatMessage]) async throws -> String {
    for _ in 0..<maxSteps {
        let choice = try await postChat(messages).choices[0]
        messages.append(choice.message) // the history is yours to maintain
 
        // the anchor D2 left behind: the stop reason decides whether the loop continues
        guard choice.finishReason == "tool_calls" else {
            return choice.message.content ?? ""
        }
 
        for call in choice.message.toolCalls ?? [] {
            var result: String
            do {
                // decoding the arguments and dispatching by hand are both on you
                let args = Data(call.function.arguments.utf8)
                result = try await toolImpls[call.function.name]?(args) ?? "Unknown tool"
            } catch {
                result = "Tool failed: \(error)" // wrap the error yourself
            }
            messages.append(ChatMessage(role: "tool", toolCallId: call.id, content: result))
        }
    }
    throw AgentError.tooManySteps
}

Hand the same job to a session-based kernel and the calling side is down to four steps: register the tools, open a session, subscribe to events, send a message.

// In a real project these four names are imported from '@earendil-works/pi-coding-agent';
// this course's lab reproduces the same API surface in a roughly 400-line pi-lite,
// which has the advantage of running offline with a readable kernel.
import { createAgentSession, ModelRuntime, SessionManager } from './pi-lite.js'
 
const modelRuntime = await ModelRuntime.create()
const model = (await modelRuntime.getAvailable())[0]
 
const { session } = await createAgentSession({
  model,
  modelRuntime,
  customTools: [getTimeTool, calcTool], // hand over the manual and the implementation only
  sessionManager: SessionManager.inMemory(), // the kernel maintains the history
})
 
session.subscribe((event) => {
  if (event.type === 'tool_execution_start') console.log(`calling tool: ${event.toolName}`)
})
 
// the loop, the dispatch, the refill, the stop check all happen inside the line below
await session.prompt('What time is it? Also work out 12 plus 30, then times 2')
session.dispose()

# The Pi SDK itself is TypeScript. What follows is the idiomatic shape of the same
# session-based kernel in Python, so you can see the four-step skeleton on the calling
# side rather than learning four SDKs in four languages.
model_runtime = await ModelRuntime.create()
model = (await model_runtime.get_available())[0]
 
session = await create_agent_session(
    model=model,
    model_runtime=model_runtime,
    custom_tools=[get_time_tool, calc_tool],  # hand over the manual and the implementation only
    session_manager=SessionManager.in_memory(),  # the kernel maintains the history
)
 
 
def on_event(event: AgentEvent) -> None:
    if event.type == "tool_execution_start":
        print(f"calling tool: {event.tool_name}")
 
 
session.subscribe(on_event)
 
# the loop, the dispatch, the refill, the stop check all happen inside the line below
await session.prompt("What time is it? Also work out 12 plus 30, then times 2")
session.dispose()

// The Pi SDK is TypeScript; this is the idiomatic shape of the same session-based kernel in Java
var modelRuntime = ModelRuntime.create();
var model = modelRuntime.getAvailable().get(0);
 
var config = new SessionConfig(model, modelRuntime,
    List.of(GET_TIME_TOOL, CALC_TOOL),      // hand over the manual and the implementation only
    SessionManager.inMemory());             // the kernel maintains the history
 
// AgentSession implements AutoCloseable, so try-with-resources replaces a hand-written dispose()
try (var session = AgentSession.create(config)) {
    session.subscribe(event -> {
        if (event instanceof ToolExecutionStart start) {
            System.out.println("calling tool: " + start.toolName());
        }
    });
 
    // the loop, the dispatch, the refill, the stop check all happen inside the line below
    session.prompt("What time is it? Also work out 12 plus 30, then times 2");
}

// The Pi SDK is TypeScript; this is the idiomatic shape of the same session-based kernel in Swift
let modelRuntime = try await ModelRuntime.create()
guard let model = try await modelRuntime.available().first else { throw AgentError.noModel }
 
let session = try await AgentSession(
    model: model,
    modelRuntime: modelRuntime,
    customTools: [getTimeTool, calcTool],  // hand over the manual and the implementation only
    sessionManager: .inMemory()            // the kernel maintains the history
)
defer { session.dispose() } // defer guarantees cleanup even if something throws early
 
let unsubscribe = session.subscribe { event in
    if case .toolExecutionStart(let name) = event { print("calling tool: \(name)") }
}
defer { unsubscribe() }
 
// the loop, the dispatch, the refill, the stop check all happen inside the line below
try await session.prompt("What time is it? Also work out 12 plus 30, then times 2")

Now set the two side by side. This table is the most valuable thing here today:

One row in that table deserves a long stare: yesterday's stop-reason check did not disappear. Plenty of people assume that adopting a framework means "there is no loop any more." The loop is still there and so is the stop reason; they simply moved out of your if and into the kernel's if. Pi calls the field stopReason, with values covering a normal finish, hitting the length ceiling, wanting a tool, erroring, and being cancelled — and the tool branch's value is toolUse, which corresponds precisely to the line where you decided yesterday whether to go around again. The reason you cannot feel it from outside is that you switched observation methods: instead of reading a return value, you subscribe to events, which is the next section.

One number in passing: yesterday's agent ran about 120 lines, of which fewer than 30 were genuinely business logic. Moved onto a session-based kernel, the same functionality lands around 40 lines and those 30 business lines are untouched. The 80 lines you saved are exactly the cells marked "framework" in the table.

Framework or hand-written: one test you can actually apply

Saving 80 lines sounds like a sure thing, and engineering has no sure things. Here is a test you can apply: ask yourself whether you need to see and change every step inside this loop.

If you do, hand-write it. Three common cases. One, learning and debugging, where visible steps are the whole point — that is what yesterday was for. Two, behavior that must be controllable to the letter, such as a compliance requirement that every model call lands in an audit log and every tool call passes an approval gate, where the framework's hooks may not sit where you need them. Three, a genuinely tiny scenario: one or two tools, at most two rounds, where saving 40 lines does not cover the cost of pulling in a whole dependency tree (a complete agent framework often installs to hundreds of megabytes, so anything cold-start sensitive should price that first).

If you do not, use the framework. The test is whether you will eventually have to do all of these: more than five tools, pushing every agent step to a frontend live, sessions that survive a restart, automatic compression when the context fills, swapping models at will. None of those is hard alone; together they are a small framework — writing it yourself means reinventing one, and the version nobody else tests.

What is actually worth guarding against is not using a framework, it is using one without knowing what it does for you. Three concrete consequences.

First, the stack gets deeper when you debug. A tool that never ran might have a description the model could not parse, might have failed schema validation, or might have been stopped by one of the kernel's hooks. Somebody who knows the layering checks the event stream first for a tool-execution event: if it fired, the problem is in execution; if it never fired, the model never intended to call it.

Second, you inherit a pile of defaults you never wrote. It chose the model for you, it inserted the system prompt for you, it registered the built-in tools for you.

Third, an upgrade changes behavior you never tested. A default shifts, your code moves not one line, and production behaves differently — which is exceptionally hard to trace, because your first instinct will always be that you did not change anything.

One sentence to close: hand-write it once before adopting a framework, and you spend a day to save half a day on every future investigation. That is the entire reason this course puts D2 before D3.

Registering tools and subscribing to events: the two things you actually write

Once a framework is in play, your code is mostly down to two jobs: hand the tools in, take the events back.

Tools first. A tool has three parts: a manual written for the model (name, description, parameter schema), an implementation that does the work, and one registration. The most underrated part of the manual is description — it is not a comment for a colleague, it is the only basis on which the model decides whether and when to call this tool, and vagueness makes it skip a tool it should have used. D5 expands on exactly this.

import { defineTool } from './pi-lite.js' // a real project imports from '@earendil-works/pi-coding-agent'
 
export const calcTool = defineTool({
  name: 'calc',
  label: 'Arithmetic', // display only, the model never sees it
  // description is the part written for the model: it decides whether and when to call
  description: 'Evaluate an arithmetic expression containing only digits, plus, minus, times, divide and parentheses, and return the result.',
  // TypeScript types are gone after compilation and unreadable at runtime, so the schema
  // has to restate the type; the three languages below keep type information at runtime
  // and can use the type declaration directly as the schema
  parameters: {
    type: 'object',
    properties: {
      expression: { type: 'string', description: 'The expression to evaluate, for example 12 plus 30 then times 2' },
    },
    required: ['expression'],
  },
  // params has already been validated against the schema above, so use it directly;
  // throw on failure and the kernel feeds the error back to the model
  async execute(_toolCallId, params) {
    return { content: [{ type: 'text', text: String(evaluate(params.expression)) }], details: {} }
  },
})

from dataclasses import dataclass
 
 
@dataclass
class CalcParams:
    expression: str
 
 
async def run_calc(tool_call_id: str, params: CalcParams) -> ToolResult:
    # params has already been validated against the schema; raise on failure and the
    # kernel feeds the error back to the model
    return ToolResult(content=[TextPart(text=str(evaluate(params.expression)))])
 
 
calc_tool = define_tool(
    name="calc",
    label="Arithmetic",  # display only, the model never sees it
    # description is the part written for the model: it decides whether and when to call
    description="Evaluate an arithmetic expression containing only digits, plus, minus, times, divide and parentheses, and return the result.",
    # the dataclass doubles as the schema; the kernel converts it to JSON Schema and validates
    parameters=CalcParams,
    field_descriptions={"expression": "The expression to evaluate, for example 12 plus 30 then times 2"},
    execute=run_calc,
)

// The record is the parameter schema: the kernel turns its field names and types into
// JSON Schema and takes care of validation
record CalcParams(
    @JsonPropertyDescription("The expression to evaluate, for example 12 plus 30 then times 2") String expression) {}
 
static final AgentTool<CalcParams> CALC_TOOL = AgentTool.<CalcParams>builder()
    .name("calc")
    .label("Arithmetic") // display only, the model never sees it
    // description is the part written for the model: it decides whether and when to call
    .description("Evaluate an arithmetic expression containing only digits, plus, minus, times, divide and parentheses, and return the result.")
    .parameters(CalcParams.class)
    // params is already validated; throw on failure and the kernel feeds the error back to the model
    .execute((toolCallId, params) -> ToolResult.text(String.valueOf(evaluate(params.expression()))))
    .build();

// A Codable struct is the parameter schema; the kernel turns it into JSON Schema and validates
struct CalcParams: Codable {
    let expression: String
    static let fieldDescriptions = ["expression": "The expression to evaluate, for example 12 plus 30 then times 2"]
}
 
let calcTool = defineTool(
    name: "calc",
    label: "Arithmetic", // display only, the model never sees it
    // description is the part written for the model: it decides whether and when to call
    description: "Evaluate an arithmetic expression containing only digits, plus, minus, times, divide and parentheses, and return the result.",
    parameters: CalcParams.self
) { _, params in
    // params is already validated; throw on failure and the kernel feeds the error back to the model
    ToolResult.text(String(evaluate(params.expression)))
}

Notice the "throw on failure" line inside execute. It runs against most people's instinct: writing business code, we habitually swallow the exception and return an error object. Inside an agent, though, a tool's exception should be thrown at the kernel, which turns it into an error-flagged tool result for the model — and a model that sees something like "no such file" will often fix its argument and retry on the next turn. Swallow the exception and return "operation failed" as a normal result and the model concludes the tool succeeded. You implemented this rule by hand with try/catch yesterday; today it is a framework convention.

Now events. Yesterday, finding out what the agent was doing meant sticking a console.log in the loop. The loop is no longer yours, so observation comes from subscribing. One complete prompt roughly emits these:

agent_start              one prompt begins
- turn_start             one round begins (one model call plus that round's tool execution)
  - message_start        one message begins (assistant or toolResult)
  - message_update       a text delta, pushed per token, extremely frequent
  - message_end          one message ends
  - tool_execution_start a tool starts running
  - tool_execution_update a progress fragment while the tool runs
  - tool_execution_end   the tool finishes
- turn_end               one round ends, carrying that round's tool results
- agent_end              the loop is over; no further events follow

Lay this list over the comparison table from the previous section and a neat correspondence appears: turn_start and turn_end are one iteration of yesterday's for loop, and agent_end is the moment you hit return. The character-by-character text stream you got by hand-parsing SSE on D1 is here the text delta carried by message_update — the same thing, observed from a different seat.

Two traps in advance. First, message_update fires per token at very high frequency, so never do real work in that callback — writing to a database, issuing a request, or running a regular expression there drags the whole streaming path down, and the right move is to accumulate a small batch and process it together. Second, event subscription is your entry point for observability: latency stats, tool success rates, and cost attribution all tap off here, which is what D5 expands on.

Source Reading

Hands-On Lab

Confirm one thing before you start: the kernel in this lab is a roughly 400-line pi-lite.ts, nearly a quarter of it comments, whose API shape corresponds one to one with Pi's public interface. That arrangement lets you run it offline and genuinely open the kernel to look at that loop — having the plaster taken off the wall once beats a hundred pages of documentation. The README gives the handful of edits that switch it to the real SDK.

1. Run MOCK=1 pnpm start unmodified first: the model reaches only one tool and outright says it has no calculator to hand. Remember that incomplete output; it is your baseline for comparison.
2. Write the tool manual properly: both description and the field descriptions are written for the model, and vagueness makes it hesitate. Rerun and watch the fake model go from unable to judge whether the tool applies to actually calling it.
3. Register the second tool with the session, then rerun and confirm both tools get called.
4. Add the round-event subscription, count how many rounds this conversation took, and compare it against the iteration count of yesterday's for loop.
5. Delete the exception-swallowing try/catch inside execute, rerun with --tool-error to see how the model reacts on the next round, then open pi-lite.ts and find the implementation for every cell of the comparison table above.

Interview Questions

Today's four questions are in the bank below, weighted toward agent-loop design decisions and the framework-versus-hand-written trade-off. Expand a question and read the analysis before the key points — the follow-up on question 2, about the upgrade risk that framework defaults create, is the spot in this chapter interviewers press hardest, so do not skip it. Each is tagged for the China-domestic or overseas market so you can prioritize by where you are applying.

Checklist and Tomorrow

• State what each layer of the Pi SDK's three-layer architecture is responsible for
• Rewrite day two's tool-calling agent with the Pi SDK, with noticeably less code
• Compare the hand-written agent loop against the Pi SDK's built-in agent loop
• Point at every line of yesterday's hand-written loop and say who owns it inside the framework, especially the line checking the stop reason
• All 5 acceptance criteria of the lab pass
• Answer at least 3 of the 4 interview questions without looking at the key points

Tomorrow (D4) we defuse exactly those two time bombs. The first is the model: the framework picked one for you, so we pull the model-calling layer out, wire three providers into it ourselves, and make it fail over to the next one when a provider misbehaves. The second is the persona: that default system prompt you never wrote, which nonetheless decides how your agent talks, gets found, overridden with one you wrote, and verified as taking effect. Framework first and dismantling second is a deliberate order — you have to see which blanks the framework filled before you know which controls to take back.