From Shot to Footage: Image-to-Video, Polling Async Tasks, and Retrying Failures
Video generation is the slowest, most expensive, and most failure-prone step in the whole pipeline; today turn it into a safely retryable async task, and learn to write prompts in shot language the model actually understands.
Today's Goals
- Generate a single shot from a first-frame image plus a prompt, and explain the trade-off between text-to-video and image-to-video
- Write a correct async task-polling loop that handles queuing, timeouts, backoff, and cancellation
- Classify video API failures, distinguishing which are worth retrying from which retrying won't fix
Yesterday you locked down characters and sets, so you hold a batch of stable stills. Today you make them move — but the hard part is not the picture. It is every engineering consequence of one call taking several minutes. When you finish, scroll back up and tick the three goals off.
Plain-Language Walkthrough
How long a shot runs and how it is shot: putting camera language into the prompt
Start with the thing beginners skip: what exactly are you asking the model to shoot?
On a soundstage a director never just says "shoot the lead downstairs." They say: medium shot, slow push in, six seconds, the lead crouched beside the scooter, a lit phone screen in his hand. Those four things — shot size, camera movement, duration, and the picture — are all required. Leave one out and the operator guesses, and the guess is never what you wanted.
A video API is the same, except it cannot ask a follow-up question. So the storyboard record gives each of the four its own field: shotSize is the framing (only wide, medium, close; do not invent a fourth), camera is the movement, durationSec is the length, visual is the picture. Assemble the prompt as what we shoot, then how, then the style. That order is not arbitrary: prompts have a 2000-character ceiling, and truncation always eats the tail — so the least important piece goes last, where losing it costs least.
Duration carries a counterintuitive trade-off. The API defaults to six seconds, and the first instinct is "then generate thirty seconds in one go." Don't. Longer clips fall apart in the middle — a character walks along and their face changes halfway. Failure cost scales linearly: one failed thirty-second clip burns what five six-second clips would. And a long clip cannot be partially redone — a problem at second eighteen means re-running everything, whereas six-second shots means re-running one. Short-form drama already cuts every three to eight seconds, so the engineering optimum and the editorial optimum coincide.
Now text-to-video versus image-to-video. Text-to-video skips an image call and looks cheaper, but it hands total control of the frame to the model, and the face you locked down yesterday goes unused. Image-to-video starts from a first frame and acts forward from there: one extra image buys you composition, lighting and appearance pinned to a still you already approved.
Whether that trade is worth it depends on the gap in magnitude. Images are listed at CNY 0.025 apiece. For video, the official pay-as-you-go page lists CNY 0.50 per second at 768P, CNY 0.80 at 2K, and CNY 0.33 at 480P. So one six-second shot costs a hundred-odd times what one still costs — spending two cents to lower its failure rate is a guaranteed win. (Those per-second tiers are the pay-as-you-go path; today's model is billed against a package credit balance instead. The two ledgers are not interchangeable, and D12 unpacks that.)
Which gives today's first ruling: use the approved still you picked on D3 as the video's first frame. How to pass it, which field it goes in, and why the real trouble only starts after that step — because this API does not hand you the video on the spot.
Interview questions
You are asked to implement a client for an asynchronous generation task. Which failure cases would you cover?让你实现一个异步生成任务的客户端,你会考虑哪些失败情况?
Common in ChinaCommon overseasIntermediate#async-task#error-handling🔒 Unlock this chapter to see the answer keys.
How do you choose a polling interval, and why is a fixed interval a bad default?轮询间隔怎么定?为什么不能一直用固定间隔死等?
Common in ChinaCommon overseasBasic#async-task#backoff🔒 Unlock this chapter to see the answer keys.
When a generation API returns a failure, how do you decide whether to retry, and what happens after the retries run out?生成类接口返回失败,你怎么判断该不该重试?重试几次之后该做什么?
Common in ChinaCommon overseasDeep dive#error-handling#retry#cost🔒 Unlock this chapter to see the answer keys.
