Red teaming: keeping an attack suite alive, wiring it into CI, and what to do after an incident
Turn four days of defenses into a loop you can keep running: maintain an attack suite that grows, write a red-team script that produces a report, gate CI on both numbers, and add rogue-behavior detection, a kill switch and a minimal incident response plan.
Today's Goals
- Design an attack suite that can grow, and say which fields every case has to carry
- Wire the red-team script into CI and set two thresholds that will not fire false alarms every day
- Write a security checklist covering rogue detection, the kill switch and incident response steps
The past four days were about making deskmate hard to attack. Today is a different job: keeping it hard to attack six months from now. When you are done, come back to the top of the page and check off the three goals.
Plain-Language Walkthrough
A fire drill is not practice for the extinguisher
Most offices run a fire drill once a year and treat it as theater — the extinguisher is not complicated. But the extinguisher was never what the drill was for. The drill practices what happens after the alarm: who moves first, which stairwell, who counts heads. Nobody rehearses those, and on the real day everyone stands still and looks at each other.
Security engineering works the same way. Four days in, you have the equipment: D1 showed where the fire starts, D2 measured how far input-side filtering gets you, D3 restructured the agent so untrusted input cannot reach a consequential action, D4 locked the blast radius down at runtime. Today adds the procedure: who reruns the drill, who reads the result, who can block a release when it fails, and what the first three steps are on the bad day.
The reason is plain: your defenses expire and the attacks do not. Tomorrow someone adds a tool to deskmate; the day after, someone rewrites a prompt; the day after that, the model gets swapped. Any one of those can quietly dismantle D3's isolation, and not one of them looks like a security change in the diff — exactly what D1's point about risk being a property of the combination guarantees. Code review will not see it, and neither will unit tests.
So today's artifact has one shape: a loop that can rerun itself, made of five things — the suite, the runner, the criterion, the gate, and regressions. Today's lab writes all five once and ends with a report and an exit code.
And the moment the loop runs, it produces a result most people do not expect: across the three configurations, the row worth remembering is not the one that reaches zero.
Interview questions
Once red-teaming runs in CI, how do you set thresholds that are both meaningful and not a daily false alarm?红队测试进 CI 之后,阈值该怎么定才既有效又不会天天误报?
Common in ChinaCommon overseasDeep dive#red-teaming#ci-gating🔒 Unlock this chapter to see the answer keys.
How do you tell an agent has gone rogue, and who gets to pull the kill switch?怎么判断一个 Agent 已经失控?断路开关该由谁来拉?
Common in ChinaCommon overseasIntermediate#rogue-agent#kill-switch🔒 Unlock this chapter to see the answer keys.
After an agent data exfiltration incident, what are your first three steps?一次 Agent 数据外泄事故发生后,你的前三步分别做什么?
Common in ChinaCommon overseasIntermediate#incident-response#forensics🔒 Unlock this chapter to see the answer keys.
