Context Compression: How to Count Tokens, What to Compress and What to Keep, and How to Verify Nothing Was Lost
Context always fills up eventually: first calibrate a local token estimator against real usage, implement segment-summary compression, pin down which message categories must be kept verbatim, then verify that compression didn't lose key information, using a set of probes that ask about early facts.
Today's Goals
- Implement a calibratable token estimator, and explain when estimation versus exact counting each applies
- Design a compression strategy, and explain which message categories must be kept verbatim and why
- Check with a verifiable method whether compression lost key information
For four days we kept adding to the context: references, instructions, checklists, memory. Today we take something out for the first time. When you finish, scroll back up and tick the three goals off.
Plain-Language Walkthrough
Only today's files on the desk
The new hire's desk is crowded now: the material you handed over, the team handbook, the checklist, the handover notes, and the dozen files opened in the last two hours, all spread out. The desk is finite, and one more sheet means removing one.
The question is which one. The lazy approach pulls the bottom sheets — older is lower, which sounds reasonable. But the oldest sheet may be exactly what you said this morning: this repository's build command is not pnpm build. Remove it and they spend the afternoon running the wrong command before asking you why it fails.
The correct approach is archiving: put that stack of read files away, but first write one line on a sticky note — "the facts confirmed in those files are..." The desk is clear and the conclusions remain.
That is context compaction: replacing a growing history with a short summary rather than discarding it. Day six's usage limit stops and tells you to compact — today builds the other half: not only stopping but making room.
Boundaries first: today is implementation only. What belongs in the context is a strategy question, and this platform's five-day context engineering course covers it end to end. Today answers three engineering questions: how to count, what to compress and what to keep, and how to prove nothing was lost.
The third is the most skipped, and it is the only one that lets you enable compaction in production with confidence: get the first two wrong and you see "the answers are a bit off"; skip the third and you cannot even tell that the wrongness came from compaction.
Interview questions
Without pulling in a tokenizer, how would you estimate the token count of a context? How large is the error, and when must you not rely on an estimate?不引入分词器,你怎么估算一段上下文有多少 token?误差有多大,什么时候不能用估算?
Common in ChinaCommon overseasBasic#token-counting#calibration🔒 Unlock this chapter to see the answer keys.
When compacting context, which kinds of messages must be kept verbatim, and which are the safest to compress away?做上下文压缩时,哪几类消息必须原样保留?压掉哪些最安全?
Common in ChinaCommon overseasIntermediate#context-compaction#keep-list🔒 Unlock this chapter to see the answer keys.
How do you prove that a context compaction did not lose critical information?怎么证明一次上下文压缩没有丢掉关键信息?
Common in ChinaCommon overseasDeep dive#compaction-verification#probes🔒 Unlock this chapter to see the answer keys.
