Error Handling and Self-Correction: Feeding Failures Back, Backoff Retries, Loop Detection, and Cancellation

Six failure classes and three handlings: feed back, retry, or surface

There are far more than six failures, but grouped by what you do about them they fall into three buckets. The lab injects each of six once, filling all three:

The criterion is worth memorizing: can the model fix this itself? If so, feed it back; if not but a different moment would work, retry; if neither, surface it.

Surfacing is the most expensive tier, because it spends the user's attention. So reserve it for two cases: a broken environment (no key, a gateway that never responds) and a hard limit being hit.

export type Handling = 'feedback' | 'retry' | 'surface'
 
/**
 * emitted is "how many deltas this attempt already emitted," the single most important
 * parameter in the taxonomy: retrying requires that this attempt left no trace the user saw.
 */
export function diagnoseStreamError(error: unknown, emitted: number): Diagnosis {
  if (error instanceof RateLimitError) {
    return { handling: 'retry', message: 'gateway rate limited', retryAfterMs: error.retryAfterMs }
  }
  if (error instanceof StreamTruncatedError) {
    // The same error, two opposite handlings, differing only in whether characters were emitted
    if (emitted === 0) return { handling: 'retry', message: 'the stream died at the start', retryAfterMs: 500 }
    return { handling: 'surface', message: 'the stream was cut mid-way; a retry would say the same passage twice' }
  }
  if (emitted === 0 && looksTransient(error)) {
    return { handling: 'retry', message: 'network blip', retryAfterMs: 500 }
  }
  return { handling: 'surface', message: describeError(error) }
}

from dataclasses import dataclass
from typing import Literal
 
Handling = Literal["feedback", "retry", "surface"]
 
 
@dataclass(frozen=True)
class Diagnosis:
    handling: Handling
    message: str
    retry_after_ms: int | None = None
 
 
def diagnose_stream_error(error: Exception, emitted: int) -> Diagnosis:
    """Retry only when emitted is 0: retrying after output means the user hears it twice."""
    match error:
        case RateLimitError():
            return Diagnosis("retry", "gateway rate limited", error.retry_after_ms)
        case StreamTruncatedError() if emitted == 0:
            return Diagnosis("retry", "the stream died at the start", 500)
        case StreamTruncatedError():
            return Diagnosis("surface", "cut mid-way; a retry would repeat the passage")
        case _ if emitted == 0 and looks_transient(error):
            return Diagnosis("retry", "network blip", 500)
        case _:
            return Diagnosis("surface", str(error))

Both versions put rate limiting and truncation — both nominally retryable — into different buckets, differing only by the emitted parameter. It is today's most overlooked criterion, and it appears twice more below.

How to word feedback: specific enough to act on, never just "it failed"

Since a failure message is a specification, write it like one. Three requirements; miss one and the model's reaction drifts:

1. Say whether there was a side effect. "The write failed and the file is unchanged" and "the write failed" are entirely different sentences: the first says a retry is safe, the second makes it afraid to retry, or makes it write twice.
2. Give the next action, not a root-cause analysis. "That content appears three times" is a cause; "write more context so it becomes unique" is an action.
3. Do not leak technical detail. Stack traces, errno values and internal paths are noise, and it will try to "fix" them.

INJECT=tool_error is the experiment for this: the first file write fails, the feedback carries "the file is unchanged, retry as is," and the model resends the identical call next turn, succeeds, and the tests go green. Had the feedback said only "write failed," it would go off to read the file, check the directory, try creating it — each step reasonable, each step useless.

Feedback has limits too. The INJECT=bad_args segment is designed to make self-correction fail: the gateway truncates every argument string into invalid JSON, the model tries four different tools, breaks all four, and finally explains to the user. That demonstrates the point — self-correction is the first line, a hard limit is the last, and you need both.

Retry only the retryable: rate limits and network belong to the gateway, tool failures to the model

The first thing to settle is not "how many retries" but which layer the retry wraps. There is one correct answer: wrap the gateway call, never the whole turn.

The reason is short: retrying presumes the step is idempotent, and write tools inherently are not. Retrying a whole turn means a half-edited file gets edited again; and if the first edit actually succeeded and only the response was lost, the second hits "cannot find that original."

So the division is clear: the gateway layer swallows and retries its own errors, and tool-layer failures always go to the model. In code that is a thin wrapper:

export async function* streamWithRetry(
  provider: ChatProvider,
  req: ChatRequest,
  options: RetryOptions = {}
): AsyncIterable<StreamDelta> {
  const maxAttempts = options.maxAttempts ?? 3
 
  for (let attempt = 1; ; attempt += 1) {
    let emitted = 0
    try {
      for await (const delta of provider.stream(req)) {
        emitted += 1
        yield delta
      }
      return
    } catch (error) {
      // The user cancelled: not a failure; rethrow and let the loop wrap up
      if (req.signal?.aborted) throw error
      const diagnosis = diagnoseStreamError(error, emitted)
      if (diagnosis.handling !== 'retry' || attempt >= maxAttempts) throw error
      // If the gateway sent retry-after, use it as a floor: it knows better than our guess
      const waitMs = Math.max(backoffDelay(attempt, options), diagnosis.retryAfterMs ?? 0)
      options.onRetry?.({ attempt, waitMs, message: diagnosis.message })
      await sleep(waitMs, req.signal)
    }
  }
}

async def stream_with_retry(provider, req, options=RetryOptions()):
    """Wrap only the gateway layer: tool failures do not come here, they feed back."""
    for attempt in itertools.count(1):
        emitted = 0
        try:
            async for delta in provider.stream(req):
                emitted += 1
                yield delta
            return
        except Exception as error:
            if req.signal.is_set():          # the user cancelled; not a failure
                raise
            diagnosis = diagnose_stream_error(error, emitted)
            if diagnosis.handling != "retry" or attempt >= options.max_attempts:
                raise
            wait_ms = max(backoff_delay(attempt, options), diagnosis.retry_after_ms or 0)
            options.on_retry(attempt, wait_ms, diagnosis.message)
            await sleep_cancellable(wait_ms / 1000, req.signal)

emitted appears a second time here, and it is this code's only state. It guards a visible floor: characters already printed must not be said again. The starter deliberately gets this wrong (retrying truncation too), and running INJECT=truncated prints the same sentence three times — that is not recovery, that is corruption.

One more easily missed point: a failure caused by the user cancelling is not a fault and must not be retried. Without that check you meet the maddening behavior of "I pressed cancel and it sent another request."

Backoff needs jitter: why a fixed interval turns rate limiting into an avalanche

The plainest retry interval is "wait one second." That is fine while you debug alone and causes incidents in production.

Suppose your service runs fifty sessions at once and they all hit the rate limit at one moment. If everyone waits one second, then a second later fifty requests fire simultaneously, are refused simultaneously, and wait a second simultaneously — the rate limit is not eased by retries, it is dragged by you into a periodic avalanche.

The fix adds randomness to the wait, called jitter. It does one thing: spread those requests across a window so the rhythm disappears.

/** How long to wait after failure number `attempt` (attempt starts at 1) */
export function backoffDelay(attempt: number, options: BackoffOptions = {}): number {
  const baseMs = options.baseMs ?? 400
  const capMs = options.capMs ?? 8000
  const random = options.random ?? Math.random
  const window = Math.min(capMs, baseMs * 2 ** (attempt - 1))
  // Equal jitter: half fixed, half random. Not "random between 0 and the window",
  // which can draw a few milliseconds and amount to no backoff at all
  return Math.round(window / 2 + random() * (window / 2))
}

import random as _random
 
 
def backoff_delay(attempt: int, base_ms: int = 400, cap_ms: int = 8000, rng=_random) -> int:
    """Equal jitter: half the window is fixed, half random. attempt starts at 1."""
    window = min(cap_ms, base_ms * 2 ** (attempt - 1))
    return round(rng.uniform(window / 2, window))

Fix the random source at 0.5 and three waits are 300, 600 and 1200 milliseconds; with the source varying between 0 and 1, the second falls between 400 and 800; once it reaches the cap it stays at 8000. Those numbers are asserted individually in the self-test and are reproducible.

Three details that get probed:

• Why not full jitter (random between 0 and the window)? It can draw a few milliseconds, making that retry effectively unbacked-off; equal jitter guarantees a floor.
• There must be a cap. Exponential growth reaches minutes in four or five steps, and at that point the user should decide.
• If the gateway sent retry-after, use it as a floor. That is the other side telling you the truth, more accurate than a guess.

As for each gateway's actual rate-limit thresholds — do not write them into code and do not memorize them. They vary by plan, by model and over time; the reliable approach reads the response headers and falls back to this scheme when there are none.

The cancellation chain: keypress, AbortSignal, fetch, child process — one broken link and nothing dies

Shout "hold on a second" and the new hire must stop typing and also stop the build they just started. Do only the first and you see them look up while the machine keeps running.

An Agent's cancellation chain has four links, and missing any one produces "pressing it did nothing":

flowchart LR
  A["1. keypress<br/>Esc / Ctrl+C"] --> B["2. AbortController<br/>abort()"]
  B --> C["3. fetch's signal<br/>actually close the connection"]
  B --> D["4. child process<br/>kill the whole process group"]
  C --> E["loop wraps up<br/>keeping what was received"]
  D --> E

Links three and four are the easiest to miss. Checking only "has the signal aborted" inside the loop makes the word "interrupted" appear instantly and feels like stopping — while that node --test still runs and that download continues. Cancellation only counts when it reaches the outermost system resource.

Link four is the dirtiest, handled once when writing the command tool on day four: spawn the child in its own process group and kill the whole group (a negative PID), sending the terminate signal first, allowing two seconds to clean up, and force-killing if it has not exited. The reason is that node --test spawns children of its own, so killing only the outer shell orphans the grandchildren, which keep running and keep holding the pipes — so your promise never settles, and that is where "killed but not stopped" comes from.

Link one has a platform detail: key events exist only in a real terminal, and pipes and CI have no keypress event at all. So the lab extracts "which keys count as cancel" into a function so that judgment can be asserted in a pipe too — Esc counts, Ctrl+C counts, a lone c does not, and Alt combinations do not.

The model is spinning: same tool, same arguments, three times running, so interrupt and change strategy

One failure class does not look like a failure: every step succeeds and the whole thing circles in place. It reads a file, gets the content, and next turn reads the same file.

It is not broken; it cannot see its own repetition. Both calls and both results are in the context, but "I already did the same thing" requires comparing across turns, while its way of working is continuing the context — and the smoothest continuation is often what it just said.

So the loop must do this, plainly: key on "tool name plus argument text," and interrupt when consecutive hits reach a threshold.

export const REPEAT_THRESHOLD = 3
 
export class RepeatDetector {
  private lastKey = ''
  private streak = 0
 
  /** Feed one call, get how many times in a row it has occurred (including this one) */
  push(call: ToolCall): number {
    const key = `${call.name}(${call.args.trim()})`
    // Consecutive only: any other call in between resets the count
    if (key === this.lastKey) this.streak += 1
    else {
      this.lastKey = key
      this.streak = 1
    }
    return this.streak
  }
 
  tripped(call: ToolCall): boolean {
    return this.push(call) >= REPEAT_THRESHOLD
  }
}

REPEAT_THRESHOLD = 3
 
 
class RepeatDetector:
    """Compare argument text, consecutive only. Semantics cost too much; ignoring
    consecutiveness mistakes a normal verify rhythm for spinning."""
 
    def __init__(self) -> None:
        self._last_key = ""
        self._streak = 0
 
    def push(self, call: ToolCall) -> int:
        key = f"{call.name}({call.args.strip()})"
        self._streak = self._streak + 1 if key == self._last_key else 1
        self._last_key = key
        return self._streak
 
    def tripped(self, call: ToolCall) -> bool:
        return self.push(call) >= REPEAT_THRESHOLD

Three rules must be read together, or you get false positives:

• Compare the argument text, not the semantics. Arguments differing by one space count as a different call — at least it is trying something new; real spinning is character-identical repetition.
• Consecutive only. Any other call in between resets the count: "read file, run tests, read the same file" is a normal verification rhythm.
• Interrupt by feeding back, not by surfacing. Tell it "you have called this with identical arguments three times and the results are above; take another approach or give the conclusion," and in this lab it changes course that same turn.

The threshold is three rather than two, because two is common — re-reading a file after editing it is a good habit. Also, this detection sits before the approval gate: a spinning call should not disturb the user for a third approval.

The loop's hard limits: turns, duration and spend, all three

Day five's loop had one lonely constant: at most eight turns. It blocks one kind of runaway and not two others, so today it grows into three:

All three are needed, and when one trips it must say which — a user who sees "stopped" without a reason will only retry the same sentence.

Spend here is counted in tokens only, never converted to money: unit prices vary by gateway and model, and writing them into code binds the project to one vendor's price list; what you actually need to limit is input size, and tokens are its unit. To see money, multiply by a unit price configured as an environment variable at the deployment layer.

One implementation decision: make the three limits one accounting object rather than three scattered variables. Day ten's task checklist, day twelve's compaction and day twenty's cost statistics all read the same ledger. And one more: when a limit trips, besides telling the user, leave a record in the message array, or when the user says the next thing the model will not know why the previous turn stopped.

Source Reading

Hands-On Lab

Today leaves five exercises, all of the "wrong code raises no error and surfaces on the bad day" kind: retrying truncation too, backoff without jitter, checking only the turn limit, loop detection not wired into the loop, and cancel recognizing only Ctrl+C. The starter passes five of eleven unmodified.

One thing first: INJECT is read once at module load, so the self-test items that switch injections all spawn a child process rather than mutating the environment in one process — the latter verifies the injection you imagine rather than the one in effect.

1. Implement the classification function mapping the six injected failures to feed back, retry and surface; first run INJECT=truncated to see the starter's "same sentence printed three times," then fix it.
2. Add jittered backoff retry to the gateway layer for rate limits and network errors only; with INJECT=rate_limit the terminal should print a line saying it will wait some milliseconds before attempt 2, and the task then completes.
3. Wire cancellation from the keypress to the child process: type "run a long command, do not stop" and approve, then press Esc, and the fifteen-second command should stop within a second.
4. Implement repeat detection and the three hard limits; with INJECT=loop the third identical call is blocked and the model then gives its conclusion.
5. Run the self-test: MOCK=1 SELFTEST=1 pnpm start should print 11/11 passed. Delta counts, retry counts, blocked-call counts and backoff window bounds are all reproducible; only the actual milliseconds in the cancellation item vary, and there you read magnitude rather than value.

Acceptance is six ticks: the self-test prints 11/11 passed; a rate limit backs off, retries successfully and the task completes; truncation is not retried and everything received is kept; the third identical call is blocked with a "change approach" feedback; a hung command is killed on timeout with a next action in the feedback; and pressing Esc really stops a long-running command.

Interview Questions

Today's three questions test implementation judgment in error handling, not slogans like "retries need exponential backoff":

1. How do you classify an Agent's runtime errors? Which are fed back to the model and which are reported to the user?
2. How do you implement retry and backoff? Why add jitter? Which errors must never be retried?
3. When a user presses cancel, how many layers must the cancellation signal cross before it really stops?

Full bilingual prompts, analyses and key points are in this course's day-six question bank. Question three exposes experience best — someone who has not implemented it stops at "set a flag," while someone who has brings up child processes and process groups unprompted.

Checklist and Tomorrow

• I can place the six failure classes into feed back, retry and surface, and state the criterion
• I know why rate limiting and truncation are both nominally retryable yet handled oppositely
• I can explain why a retry may wrap only the gateway layer and not the whole turn
• I can explain why a fixed interval turns rate limiting into a periodic avalanche, and why equal jitter beats full jitter
• I can name the cancellation chain's four links and the symptom of missing links three and four
• I know repeat detection's three rules: compare argument text, consecutive only, interrupt by feeding back
• I can say what each of the three hard limits blocks, and why spend is counted in tokens only

Tomorrow is D7, "Session Persistence and Recovery: an Append-Only Event Log, resume and forking, plus a week-one review." Today it can get up after falling, but the moment the process exits the whole conversation is gone — including the facts it worked hard to establish. Tomorrow puts the conversation on disk with an append-only event log so it can restart and continue, and fork a new session from any point in the middle. The order is deliberate: clean up within-turn failures before discussing cross-process recovery, or "this turn errored" and "last time did not finish" end up tangled in the same code. Week one's seven layers stack up tomorrow, and tomorrow ends with a review.