Dayward AIWeChat Mini Program
Context Engineering in 5 Days
Week 1 · D5🔒 LockedAbout 3 hours

Measuring and Tuning: the Token Bill, Context Utilization, Failure-Mode Triage, and a Comprehensive Interview Deep Dive

Fold the previous four days' techniques into one measurable loop: work out the token bill and cache hits, watch two utilization metrics, locate the problem against four failure modes, then work through a comprehensive context-engineering interview deep dive.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Work out a task's token bill, and split it into cache writes, cache hits, and new input
  2. Use the window-occupancy rate and the useful-information ratio to locate which part of the context is the problem
  3. Match a symptom against four typical failure modes, and give a triage path from symptom to fix

The first four days taught techniques: trimming, pruning, compaction, isolation. Today teaches which technique when — and when to stop. Come back when you have read it and tick off the three goals.

Plain-Language Walkthrough

Weighing the bag on the way home: the bill is not the last weigh-in

Back at the airport you put your bag on the scale, and the number is how heavy the case is at this moment. But if the airline charged by weighing separately on each leg, what you actually pay is the sum of four legs' weights — and you kept adding things along the way, so the later legs are certainly heavier than the first.

A model's bill works exactly that way. Many people quote cost as "how many tokens per request," which is a weigh-in, not a bill. The bill is the running sum of every turn's input across the whole session, because the model has no memory and every turn resends everything before it.

How big is that difference? Work it out on the 20-turn e-commerce support task from today's lab: the stable prefix per turn is 1,612 tokens (system prompt 881 plus tool definitions 731), history grows about 55 per turn, and a tool is called roughly every 3 turns at about 1,424 each time. That adds up to this table:

ComponentGrowth patternTotal over 20 turns
Stable prefixResent verbatim every turn32240
Conversation historyLinear growth11550
Tool resultsStep growth89712
Total133502

The single input on the last turn is 11,256 tokens while the session's bill is 133,502 — nearly 12 times apart. Anyone reporting only the single input understates the cost by an order of magnitude.

That table also answers where to change first: tool results take 67%, so do D3's pruning first; the prefix takes 24%, so D2's trimming comes next; history is only 9%, so D4's compaction comes last. The four days' order is not sorted by difficulty but by this table.

But the table left one thing out, and it turns the conclusion inside out — caching. That is where we start.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • How do you compute the token bill for one agent task, and which parts can be cached away?怎么给一个 Agent 算一次任务的 token 账单?哪些部分是可以被缓存掉的?
    Common in ChinaCommon overseasDeep dive#token-accounting#prompt-caching

    🔒 Unlock this chapter to see the answer keys.

  • When context is the problem, how do you localize which of the four buckets is at fault?上下文出问题的时候,你怎么定位是四块里的哪一块?
    Common in ChinaCommon overseasIntermediate#diagnostics#metrics

    🔒 Unlock this chapter to see the answer keys.

  • How much context engineering is enough, and how do you know when to stop?上下文工程做到什么程度算够?你怎么知道该停手了?
    Common in ChinaCommon overseasIntermediate#tuning#stopping-criteria

    🔒 Unlock this chapter to see the answer keys.

Comments