Why an Agent Starts Degrading Once It Runs Past the Second Context Window
Start with what really separates a long-horizon task from an interactive conversation, then reproduce the degradation with your own hands in a run that crosses two context windows: the second window redoes everything the first one already did. The model did not get dumber. There is simply no evidence left in its context that any of that work happened.
Start with what really separates a long-horizon task from an interactive conversation, then reproduce the degradation with your own hands in a run that crosses two context windows: the second window redoes everything the first one already did. The model did not get dumber. There is simply no evidence left in its context that any of that work happened.
Interview questions
What is the difference between the context window and state? Why can a long-horizon agent's authoritative progress not live only in the window?上下文窗口和状态有什么区别?为什么长时程 Agent 的权威进度不能只存在窗口里?
Common in ChinaCommon overseasIntermediate#long-horizon#state-management#context-window🔒 Unlock this chapter to see the answer keys.
Why should the model-facing interface not know which harness features are switched on? What happens if it does?为什么给模型的接口不该知道当前开了哪些 harness 功能?如果知道了会发生什么?
Common in ChinaCommon overseasDeep dive#experiment-design#interface-boundary#false-green🔒 Unlock this chapter to see the answer keys.
An agent that ran unattended overnight tells you in the morning that every task is complete. How do you verify that claim?一个无人值守跑了一夜的 Agent 早上告诉你全部任务已完成。你会怎么验证这句话?
Common in ChinaCommon overseasIntermediate#verification#long-horizon#reporting🔒 Unlock this chapter to see the answer keys.
Same model, same prompts, but swapping the harness lifts the benchmark score by more than ten points. What does that tell you?同一个模型、同一套提示词,换一个 harness 成绩提升十几个点。这说明了什么?
Common in ChinaCommon overseasIntermediate#harness-engineering#benchmarks#model-capability🔒 Unlock this chapter to see the answer keys.
