Capstone and Retrospective: Turning a Team's Conventions Into a Skill Pack and Driving a Subagent Through a Real Task

From a conventions document to three skills: split by trigger situation, not by chapter

Given a team conventions document, the most natural move is to split along its table of contents: chapter one one skill, chapter two another. That is the most common mistake.

Chapter structure serves a human's reading order, usually running from concept to detail; a skill's boundary must serve the trigger situation, because the model decides whether to open it at the moment a user has just said something. Those two structures almost never coincide.

Splitting by trigger situation takes three steps in practice.

Step one, read the document once and note only when somebody would need this passage. Note situations, not content. A thirty-page engineering handbook usually compresses to a dozen situations.

Step two, cluster the situations by "the same moment." The format of a commit message, the allowed type values, and what the body should say may be scattered across three chapters, but all three are used at the single moment of "I am about to commit" — they are one skill. Conversely, "how to write a commit message" and "how to split a commit," discussed in the same chapter, are two moments and must be separated.

Step three, write one description per cluster and check for mutual exclusion. Use day three's positive-and-negative apparatus here: write three sentences that would trigger each skill and two similar-looking ones that should not, then run them and see whether they steal from each other. Stealing means the clustering was not clean, so go back to step two.

There is one further class of content to handle separately: the deterministic rules in the document — the type may only be one of these six, the version number must match this pattern. Those are better distilled into a validation script per day four's criteria, leaving the skill's body one line saying to run the validation script when finished.

Driving a subagent: why isolate a clean session

The three skills are installed, so how do you try them? Not in the session you have been debugging in for two hours.

That session's context is already scattered with the convention fragments you typed by hand, the wording you corrected, the examples you pasted. The model performs well under those conditions and you conclude the skill did it — when in fact you recited the answer yourself in the session. That is the most common self-deception in evaluating a skill.

The right approach hands it to a subagent: an executor with its own separate context window that starts from a clean slate, carries only what you specify, and hands the result back when done.

It brings three things, each serving today's acceptance directly.

First, the context is clean. A subagent does not inherit the main session's conversation history. Whatever you said or corrected in the main session, it knows nothing of. Only results from that setup genuinely reflect the skill's effect.

Second, capabilities can be bounded precisely. A subagent's definition can declare which tools it may use and which model, and can declare which skills to preload — which is especially useful for a controlled comparison: the with-skill arm states them explicitly and the without arm leaves it empty, with every other condition identical.

Third, noise stays out of the main session. It keeps the file digging, the trial and error, and the script runs in its own context and hands back only the conclusion.

A subagent definition is a Markdown file with frontmatter in a conventional directory:

---
name: convention-runner
description: Complete commit, review, and release tasks per the team conventions
tools: Read, Grep, Bash
model: sonnet
skills:
  - team-conventions:commit-message
---
 
You are an executor working strictly to the team conventions. Do only the one thing asked,
and change no other files. Before handing back the result, self-check it against the
conventions line by line.

In a controlled experiment the two definitions differ only by that skills line. Any other difference — a different model, a different tool set, one extra sentence of prompt — makes your conclusion unable to say whose credit it was.

Evals: how to write assertions

With a clean execution environment, the next step is today's most technical: turning better into a number.

An eval case has three parts: one prompt, a description of the expected output, and two to four judgeable assertions. The assertions are the whole point; write them badly and the eval is just sighing in a new location.

Judgeable means: not whether the output is good, but whether a fact holds. A comparison makes it clear:

• Not judgeable: "the commit message is clearly written," "the review comment is valuable" — still feelings.
• Judgeable: "the type field is one of those six values," "the scope field equals a directory that genuinely exists in the repository," "the body does not repeat the list of file names already in the diff," "the first line is at most 50 characters."

An assertion is best checking one thing only. Three assertions each checking one thing are far more useful than one checking three, because on failure you immediately know which one broke.

How should an eval set be balanced? Positive, boundary, and negative at roughly five to three to two. Positives are the most common usage; boundaries are inputs that nearly should trigger a different skill; negatives are inputs that definitely should not trigger, with the assertion written as "this skill was not activated." Without negatives you cannot detect an over-broad trigger surface — and over-broad is a skill's most common failure.

Running it is plain: the same set of cases, run once by the with-skill subagent and once by the without, judged case by case, with a pass rate for each side.

type Assertion = { id: string; describe: string; check: (output: string) => boolean }
type Case = { id: string; prompt: string; kind: 'positive' | 'boundary' | 'negative'; assertions: Assertion[] }
type Run = { caseId: string; output: string }
 
export type Report = { total: number; passed: number; rate: number; failures: string[] }
 
/** A case passes only when every assertion holds. Record failures individually, since the detail is what guides edits. */
export function score(cases: Case[], runs: Run[]): Report {
  const byCase = new Map(runs.map((r) => [r.caseId, r.output]))
  const failures: string[] = []
  let passed = 0
 
  for (const c of cases) {
    const output = byCase.get(c.id)
    if (output === undefined) {
      failures.push(`${c.id}: no run record for this case`)
      continue
    }
    const failed = c.assertions.filter((a) => !a.check(output))
    if (failed.length === 0) passed += 1
    else failures.push(...failed.map((a) => `${c.id} / ${a.id}: ${a.describe}`))
  }
  return { total: cases.length, passed, rate: passed / cases.length, failures }
}
 
/** The comparison's conclusion is one number: the difference in pass rate. Both arms must run the same cases. */
export function compare(withSkill: Report, without: Report): string {
  const delta = (withSkill.rate - without.rate) * 100
  return `without ${(without.rate * 100).toFixed(0)}% -> with ${(withSkill.rate * 100).toFixed(0)}%, up ${delta.toFixed(0)} percentage points`
}

from dataclasses import dataclass
from typing import Callable
 
 
@dataclass
class Assertion:
    id: str
    describe: str
    check: Callable[[str], bool]
 
 
@dataclass
class Case:
    id: str
    prompt: str
    kind: str  # positive | boundary | negative
    assertions: list[Assertion]
 
 
@dataclass
class Report:
    total: int
    passed: int
    rate: float
    failures: list[str]
 
 
def score(cases: list[Case], runs: dict[str, str]) -> Report:
    """A case passes only when every assertion holds. Failure detail is what guides edits."""
    failures: list[str] = []
    passed = 0
 
    for case in cases:
        output = runs.get(case.id)
        if output is None:
            failures.append(f"{case.id}: no run record for this case")
            continue
        failed = [a for a in case.assertions if not a.check(output)]
        if not failed:
            passed += 1
        else:
            failures.extend(f"{case.id} / {a.id}: {a.describe}" for a in failed)
 
    total = len(cases)
    return Report(total, passed, passed / total if total else 0.0, failures)
 
 
def compare(with_skill: Report, without: Report) -> str:
    """The comparison's conclusion is one number: the difference in pass rate."""
    delta = (with_skill.rate - without.rate) * 100
    return f"without {without.rate:.0%} -> with {with_skill.rate:.0%}, up {delta:.0f} percentage points"

Writing it up as a portfolio entry

This delivery deserves a place on your resume, but only if written correctly. An entry that survives questioning has three things.

A number, and the conditions it was measured under. "Improved code convention consistency with skills" is empty; "across 20 review tasks, convention compliance rose from 55% to 90%, with the same model, the same prompt, and two subagent arms differing only in whether the skills were preloaded" is questionable in the good sense. Giving the final value without the starting point is giving nothing.

What you gave up. That is the one part of the entry nobody can copy, and where the follow-up questions concentrate. For example: "the three skills total just over three hundred lines, with all the background and history from a thirty-page handbook left out — the criterion being whether the model would get it wrong without a line," or "deterministic format validation did not go into the body but was distilled into a script."

Why not the alternative. An interviewer will certainly ask why these rules were not simply written into the system prompt. The answer came on day one, and today you should be able to give it in one breath: the system prompt is resent every turn, so twenty conventions are billed continuously; and dearer still is attention, where a system prompt stuffed with unrelated rules lets rule seventeen interfere while working on the third thing. A skill's value is not how much it holds but that it stays closed until needed.

The seven-day retrospective and what to learn next

Look back at the through-line of these seven days: from one folder to one conclusion that survives questioning. Day one established what a skill is, day two wrote the first one that triggers, day three turned authoring into method, day four added scripts, day five saw from the runtime's side how it gets loaded, day six made it handable to others, and today proved it genuinely helps.

If you take away only three sentences:

First, the description is the only trigger surface. It is the one stretch of text in this mechanism that you pay for every turn and the one that decides success or failure.

Second, capability gaps are filled by tools and experience gaps by skills. Get that cut wrong and every later selection decision is wrong.

Third, determinism goes to code and judgment stays with the model. It explains at once when to write a script and when to write the body.

Two roads from here, both sister courses released in the same batch, and which to take depends on what you lack.

Missing the wiring, go learn MCP. You can now hand experience to a model, but the model still has none of your company's tools and data in hand. MCP in 7 Days: wiring tools into any agent runs from the protocol triangle to remote deployment and security governance, and day six's prompt injection and least privilege is a lesson you cannot skip when this package actually rolls out across a team.

Missing the trade-offs, go learn context engineering. Skills helped you close away what should not be resident, but the context still has the system prompt, tool results, retrieved passages, and history messages competing for room. Context Engineering in 5 Days covers exactly how that bill is computed, how long sessions are compacted, and how failure modes are triaged — and day five's token bill and utilization metrics can be taken straight over to add another dimension to today's eval.

Source Reading

Hands-On Lab

The easiest one to fudge today is criterion 3. The moment the two arms have a second difference, your number cannot say whose credit it was — a swapped model, an extra tool, half a sentence of changed prompt, all count. Before starting, put the two subagent definitions side by side and diff them, confirming only that one line differs.

1. Read the solution's delivery document and see by what trigger situations its three skills were split.
2. Fill in the conventions document's split table in the starter, writing each skill's boundary and what it does not handle.
3. Fill in the eval case table, writing each case's prompt, expected output, and two to four judgeable assertions.
4. Record the results of running with and without skills, judge case by case, and compute the pass rate difference.
5. Compress the whole delivery into one portfolio entry, noting the measurement conditions of every number.

Interview Questions

Today's 3 questions are in the question bank below, weighted toward splitting and accepting a skill pack, eval design and baseline comparison, and presenting results. Expand each one and read the analysis before the key points — practicing the derivation beats memorizing bullets. The high-frequency labels for the domestic and overseas markets are there so you can pick by target market.

Checklist and Tomorrow

• Split a team's conventions document into three skills with clear, non-overlapping boundaries
• Design a set of eval cases that prove the difference in output with and without skills
• Write up this delivery as a portfolio entry that holds up under follow-up questions
• Explain why a controlled experiment needs a clean subagent rather than a session you have been debugging in
• All 5 acceptance criteria of the lab pass
• Answer at least 2 of the 3 interview questions without looking at the key points

The seven days end here. Look back at day one's "a skill is a folder containing a SKILL.md" and you should now hear how much it left out — how to write a description that triggers accurately, how many layers the body takes, when a script belongs, how a runtime loads it, how a set of skills is handed to a team, and how to prove it genuinely helps. The format really is simple; writing experience down clearly never was. Next, go fill in the half you lack: missing the wiring, take MCP in 7 Days; missing the trade-offs, take Context Engineering in 5 Days.