Messages, Context Engineering and Compression, Session Storage/Recovery/Forking (dg M06/M08/M09/M10)
Understand how messages are organized and passed around inside an agent, how to compress context once the window fills up, and how a session is persisted, recovered, and forked.
Today's Goals
- Draw a message's complete path from user input to being written into history
- Implement a simple context-compression strategy that fires automatically as the window nears full
- Implement persisting a session, resuming it to continue the conversation, and forking a new session from a given step
Tools learned to correct themselves yesterday, but D5 left a bill open: every tool call pushes two messages into messages, and the tool-result one is often a thousand-token blob of JSON. The better your tools, the faster the history grows, until it bursts the context window. Today we face that head on.
Plain-Language Walkthrough
The life of a message: an agent's memory is this array
Some classrooms keep a relay diary: one notebook circulates, and when your turn comes you have to page back through it to know what to write next. The thicker it gets the longer the paging takes, and you have no alternative. An agent's memory is that notebook โ D1 established that the model is stateless and that a multi-turn conversation is entirely the messages array you maintain client-side. Today we fill in its internal structure. Every message carries a role; D1 covered three (system, user, assistant), and D2's tools added a fourth, tool, which holds a tool's return value.
So a message's path runs: user input arrives and appends a user; the array goes to the model; the model decides to call a tool and the returned assistant carries tool_calls, appended as-is; the tool runs and its result is wrapped into a tool and appended; the array is resent, and only now does the model emit prose for the user, which is another assistant to append. One round with a tool in it writes four messages into the history, not one.
// The agent's "memory" is this array: the whole of it is resent to the model every round
const history = [
{ role: 'system', content: 'You are an order assistant.' },
{ role: 'user', content: 'When will the order I placed three days ago arrive?' },
]
// One tool call pushes two messages: the model's call request plus the tool's result
function appendToolRound(history, call, result) {
history.push({ role: 'assistant', content: '', tool_calls: [call] })
history.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(result) })
}
// A rough token estimate: one character counts as one token, deliberately on the high side.
// Underestimating means the compression threshold never fires, and that is the one
// mistake you cannot make here
const estimateTokens = (msgs) =>
msgs.reduce((sum, m) => sum + (m.content ?? '').length + 4, 0)import json
import math
history: list[dict] = [
{"role": "system", "content": "You are an order assistant."},
{"role": "user", "content": "When will the order I placed three days ago arrive?"},
]
def append_tool_round(history: list[dict], call: dict, result: dict) -> None:
"""One tool call pushes two messages: the model's call request plus the tool's result."""
history.append({"role": "assistant", "content": "", "tool_calls": [call]})
history.append(
{
"role": "tool",
"tool_call_id": call["id"],
"content": json.dumps(result, ensure_ascii=False),
}
)
def estimate_tokens(msgs: list[dict]) -> int:
# One character per token is above the true figure, and that is deliberate:
# underestimating means the threshold never fires. A threshold check does not need
# an exact number, so there is no reason to pull in a tokenizer
return sum(len(m.get("content") or "") + 4 for m in msgs)// Dependencies: Jackson (com.fasterxml.jackson.databind)
// A record is immutable by construction, which matches "history is append-only" exactly.
// The toolCalls field cannot be dropped: the assistant's call request pairs with the tool
// result below it by id, and both the compression cut point and the completeness check on
// recovery rest on that pairing
record ToolCall(String id, String name, String arguments) {}
record Message(String role, String content, String toolCallId, List<ToolCall> toolCalls) {
static Message user(String text) {
return new Message("user", text, null, List.of());
}
static Message tool(String id, String json) {
return new Message("tool", json, id, List.of());
}
static Message assistantCalling(ToolCall call) {
return new Message("assistant", "", null, List.of(call));
}
}
static final ObjectMapper MAPPER = new ObjectMapper();
static final List<Message> HISTORY = new ArrayList<>(List.of(
new Message("system", "You are an order assistant.", null, List.of()),
Message.user("When will the order I placed three days ago arrive?")));
// One tool call pushes two messages: the model's call request plus the tool's result
static void appendToolRound(List<Message> history, ToolCall call, Object result) throws Exception {
history.add(Message.assistantCalling(call));
history.add(Message.tool(call.id(), MAPPER.writeValueAsString(result)));
}
// One character per token is above the true figure, deliberately: underestimating means
// the compression threshold never fires
static int estimateTokens(List<Message> msgs) {
return msgs.stream().mapToInt(m -> m.content().length() + 4).sum();
}// Making Message Codable directly means one type both goes to the API and lands on disk,
// sparing you two parallel models
struct ToolCall: Codable {
let id: String
let name: String
let arguments: String
}
struct Message: Codable {
let role: String
let content: String
var toolCallId: String? = nil
// toolCalls cannot be dropped: the assistant's call request pairs with the tool result
// below it by id, and both the compression cut point and the completeness check on
// recovery rest on that pairing
var toolCalls: [ToolCall]? = nil
enum CodingKeys: String, CodingKey {
case role, content
case toolCallId = "tool_call_id"
case toolCalls = "tool_calls"
}
}
var history: [Message] = [
Message(role: "system", content: "You are an order assistant."),
Message(role: "user", content: "When will the order I placed three days ago arrive?"),
]
// One tool call pushes two messages: the model's call request plus the tool's result
func appendToolRound(_ history: inout [Message], call: ToolCall, resultJSON: String) {
history.append(Message(role: "assistant", content: "", toolCalls: [call]))
history.append(Message(role: "tool", content: resultJSON, toolCallId: call.id))
}
// One character per token is above the true figure, deliberately: underestimating means
// the compression threshold never fires
func estimateTokens(_ msgs: [Message]) -> Int {
msgs.reduce(0) { $0 + $1.content.count + 4 }
}That estimateTokens deserves a note. D1 put an English word at roughly 1.3 tokens, so counting characters directly lands above the true figure โ and the direction is a deliberate choice. Underestimate and real usage hits 100% while your number has only just passed 70%, the threshold never fires, and you find out via a hard error thrown back by the model.
Run the arithmetic and D5's warning gets concrete. One order-table lookup returns two or three thousand tokens; assume one tool call per round with an 800-token result, and with the prose on both sides a round books about 1,000 tokens. Thirty rounds is 30,000, and all of it is resent in full on round 31. What bursts a history is never what the user typed, it is what tools hand back to the model โ precisely the part that grows faster the better you make the agent. So the question is not whether it fills up, it is what to do when it does.
Interview questions
When a long conversation outgrows the context window, how do you compress it โ when do you trigger, what do you drop, and what do you keep?้ฟๅฏน่ฏ้ไธไธๆๆพไธไธไบ๏ผไฝ ไผๆไนๅ็ผฉ๏ผไปไนๆถๅ่งฆๅใๅๆไปไนใไฟ็ไปไน๏ผ
Common in ChinaCommon overseasIntermediate#context-engineering#compression#cost๐ Unlock this chapter to see the answer keys.
What problems do session persistence, restore, and forking each solve, and what goes wrong in each?ไผ่ฏ็ๆไน ๅใๆขๅคๅๅๅๅๅซ่งฃๅณไปไน้ฎ้ข๏ผๅฎ็ฐๆถๅๆไปไนๅ๏ผ
Common in ChinaCommon overseasIntermediate#session-management#persistence#forking๐ Unlock this chapter to see the answer keys.
Where do you draw the line between short-term context and long-term memory, and how do you decide where a given fact belongs?็ญๆไธไธๆๅ้ฟๆ่ฎฐๅฟ็่พน็ๆไนๅ๏ผไธๆกไฟกๆฏ่ฏฅๅพๅชๆพ๏ผไฝ ็ๅคๆญไพๆฎๆฏไปไน๏ผ
Common in ChinaCommon overseasBasic#memory#context-engineering๐ Unlock this chapter to see the answer keys.
How do context engineering and RAG relate, and what breaks if you do RAG without context management?ไธไธๆๅทฅ็จๅ RAG ๆฃ็ดขๆฏไปไนๅ ณ็ณป๏ผๅชๅ RAG ไธๅไธไธๆ็ฎก็ไผๅบไปไน้ฎ้ข๏ผ
Common in ChinaCommon overseasDeep dive#context-engineering#rag#retrieval๐ Unlock this chapter to see the answer keys.
