Cost and Model Routing: Choosing a Model per Stage, Caching, Degradation, and a Budget Circuit Breaker
Get a clear tally of the whole line's costs, tier models per stage, bring cost down with caching and degradation, and fit every run with a budget circuit breaker that actually pulls the brake.
Today's Goals
- Break cost down per stage, and point out where most of the money goes and why
- Configure a model-routing policy per stage that makes an explainable trade-off between quality and cost
- Implement a budget circuit breaker that safely stops a run when it goes over budget instead of burning through it all
For eleven days you made this line steadier: resumable, concurrent, reviewable, quality-checked. Today changes dimension — making it affordable to run. When you finish, scroll back up and tick the three goals off.
Plain-Language Walkthrough
The line producer's first day: put the books on the table
Productions have a role called the line producer, responsible for one thing: where the money went. The classic rookie error is to start by squeezing the catering budget — catering is a few hundred a day, and that crane is a few thousand. Squeeze the wrong thing and you exhaust yourself saving nothing.
Your production line is the same. For eleven days run.json has been quietly keeping the books (D7 printed a bill, D8 persisted ledger entries), and nobody has read the table carefully. Today we open it, because the four kinds of call on this line differ in price by three orders of magnitude.
| Stage | Billing unit | Published rate | Usage per episode | Subtotal |
|---|---|---|---|---|
| Shot video (768P) | per second | CNY 0.50 | 3 shots, 18 seconds | CNY 9.00 |
| Look tests and per-shot first frames | per image | CNY 0.025 | 5 images | CNY 0.125 |
| Line dubbing | per billed character | about CNY 0.00035 (derived) | 27 characters | CNY 0.0095 |
| Script and review text | per token | per the official console | — | not priced |
Three caveats must come first, or everything after them is wrong.
First, the speech tier is not an official rate. The vendor publishes only package pricing (an HD package at CNY 630 for 2 million characters), and the per-unit rate is derived from it, a bit over three yuan per ten thousand characters. So every speech-related amount in this course is marked as derived — reporting a derived figure to your boss as an official price will end badly.
Second, the text stage is simply not priced. Its per-unit rate is whatever the official console says; if we cannot get it, we write no number, and the ledger still counts tokens with the amount column blank. An uncertain number written into a table is far more dangerous than a blank, because nobody will ever check it again.
Third — the most counterintuitive one — the video row's rate cannot be attached to any specific model id. The official pay-as-you-go page gives per-second tiers (CNY 0.50 per second at 768P, 0.80 at 2K, 0.33 at 480P), while the Hailuo line used in this course's code is deducted as credits from a package, with no published per-second rate; the per-second tiers belong to a separately billed set of models. The two billing regimes are simply not interchangeable. So the lab's table prices by "tier plus spec," and the video rows are derived constants for ledger demonstration, marked as such in the code comments and on the dashboard.
That is itself a good teaching point: the first task in building a cost ledger is not writing the table, it is establishing what each item is actually billed by. Different product lines from one vendor, pay-as-you-go versus subscription, credits versus cash — the regimes are frequently inconsistent. Either verify them or honestly mark them derived. The worst outcome is mixing regimes in one table and then budgeting from it.
Add the four rows: just over CNY 9.13 per episode, of which video is 98.5%. That is the origin of this chapter's — and this whole course's — central judgment:
The most expensive, slowest and most failure-prone thing on this line is video generation, so the entire engineering design orbits the question of how to call the video API one fewer time.
Looking back at the first eleven days feels different now: idempotency keys (D8) exist to avoid repeat video calls, reference-image reuse (D3) exists to need fewer video attempts, the concurrency gate (D9) exists to avoid blowing the video quota, and human review before assembly (D10) exists to avoid shooting a whole episode for nothing. They are not unrelated best practices; they are corollaries of one judgment.
How does the table become code? The key is that the ledger records at the moment of the call, not by guessing afterwards. Each record carries the stage, the type, the model, the usage, whether it hit the cache and whether it was really billed; aggregate by stage at the end and where the money went is obvious.
// Rate table: published rates only. Where we cannot get one, leave it blank rather than invent it.
const RATES = {
'video-768p': { unit: 'second', cny: 0.5, approx: false },
'video-2k': { unit: 'second', cny: 0.8, approx: false },
'video-480p': { unit: 'second', cny: 0.33, approx: false },
image: { unit: 'item', cny: 0.025, approx: false },
// Only package pricing is published, so this tier is derived and must be flagged on the dashboard
tts: { unit: 'character', cny: 0.00035, approx: true },
}
const estimate = (rateId, units) =>
rateId ? Number((RATES[rateId].cny * units).toFixed(4)) : 0
function summarize(rows) {
const byStage = new Map()
for (const r of rows) {
// Cache hits, and failed calls that were not billed, are recorded but cost nothing
const cny = r.cached || !r.billed ? 0 : estimate(r.rateId, r.units)
byStage.set(r.stage, (byStage.get(r.stage) ?? 0) + cny)
}
const total = [...byStage.values()].reduce((a, b) => a + b, 0)
// Tie-break by stage name so two runs produce line-for-line identical reports
return [...byStage].sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]))
.map(([stage, cny]) => ({ stage, cny, share: total ? cny / total : 0 }))
}from collections import defaultdict
# Rate table: published rates only. Where we cannot get one, leave it blank rather than invent it.
RATES = {
"video-768p": {"unit": "second", "cny": 0.5, "approx": False},
"video-2k": {"unit": "second", "cny": 0.8, "approx": False},
"video-480p": {"unit": "second", "cny": 0.33, "approx": False},
"image": {"unit": "item", "cny": 0.025, "approx": False},
# Only package pricing is published, so this tier is derived and must be flagged
"tts": {"unit": "character", "cny": 0.00035, "approx": True},
}
def estimate(rate_id, units):
return round(RATES[rate_id]["cny"] * units, 4) if rate_id else 0
def summarize(rows):
by_stage = defaultdict(float)
for r in rows:
# Cache hits, and failed calls that were not billed, are recorded but cost nothing
cny = 0 if r["cached"] or not r["billed"] else estimate(r["rate_id"], r["units"])
by_stage[r["stage"]] += cny
total = sum(by_stage.values())
# Tie-break by stage name so two runs produce line-for-line identical reports
ordered = sorted(by_stage.items(), key=lambda kv: (-kv[1], kv[0]))
return [
{"stage": s, "cny": c, "share": c / total if total else 0} for s, c in ordered
]Two details deserve mention. One is the cached and billed booleans: cache hits are recorded with zero cost, and a failed or safety-blocked video is not billed, likewise recorded with zero cost. Without those two flags the report either overstates the bill or hides how much the cache saved. The other is the sort tie-breaker — order by name when amounts match, or line order drifts between runs and you will think something is broken.
With the books open the question gets concrete: video is over ninety percent, can that ninety percent be reduced? It can, along four non-overlapping routes — tiering, caching, degradation, circuit breaking — each covered below.
Interview questions
How do you break down the cost of a content-generation pipeline, and which stage would you optimize first?一条内容生成流水线的成本要怎么拆?拆完你会先优化哪一环,为什么?
Common in ChinaCommon overseasBasic#cost-analysis#observability🔒 Unlock this chapter to see the answer keys.
When should you degrade instead of retry, and which dimensions can you degrade first?什么情况下该降级而不是重试?如果决定降级,你有哪些维度可以降,怎么排先后?
Common in ChinaCommon overseasIntermediate#degradation#retry-strategy🔒 Unlock this chapter to see the answer keys.
How would you design a budget circuit breaker for a pipeline that calls paid APIs, and what makes the stop safe?给一条会调用付费接口的流水线加预算熔断,你会怎么设计?做到什么程度才算安全停机?
Common in ChinaCommon overseasDeep dive#budget-control#circuit-breaker🔒 Unlock this chapter to see the answer keys.
