Dayward AIWeChat Mini Program
Build Your Own Coding Agent in 21 Days
Week 3 · D19🔒 LockedAbout 4 hours

Multimodal Input: Pasting Screenshots, Image Validation, and Fixing Code From a Screenshot

Give the agent eyes: wire images into the message structure, handle both paste and file sources, size and dimension floors, and how to degrade gracefully for a model that doesn't support images, then drive a real code change from one screenshot of a UI.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Wire images into the existing message structure without breaking the plain-text path's compatibility
  2. Implement image source parsing and validity checks, giving human-readable failure messages
  3. Give a usable degradation path when the model doesn't support images

For eighteen days mca has worked from words alone. Today we open one eye. When you finish, scroll back up and tick the three goals off.

Plain-Language Walkthrough

Let them look at the drawing, not just hear the description

There are two ways to hand a new hire a UI change.

The first is to describe it: "the blue primary button on the left of the toolbar — reword it so it says something about saving." That takes three rounds of clarification. Which one is the primary button? Is the toolbar across the top or down the side? What wording counts as "something about saving"?

The second is to put the drawing on their desk: "make it match this."

For eighteen days we have used the first way. Everything mca knows about a screen came from the few lines you typed. Yet for many tasks the information already exists as a picture: an error screenshot, a design mockup, a visual-diff image from a test framework. Describe it and half the information is gone — and the half you lost is exactly the half you did not notice.

Today's job is to let that picture into the conversation. It sounds like adding a field. It is not. It pulls four real problems in behind it: how to extend the message structure without breaking the plain-text path eighteen days are built on; what "pasting an image" can even mean inside a terminal; which images should be stopped before they are sent; and the nastiest one — how do you know whether the model on the other end can see the picture at all.

That last one is not a rhetorical question. Measured on 2026-09-07, with a real key against an OpenAI-compatible gateway: one pure-red image went to four models, and all four returned normally with no error of any kind. One of them described that pure-red image as blue-green. It did not refuse; it made an answer up. That single measurement rewrites the whole second half of today.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • How would you add images to an existing text-only message schema without breaking compatibility?在已有的纯文本消息结构上加图片,你会怎么改才不破坏兼容?
    Common in ChinaCommon overseasBasic#multimodal#message-schema

    🔒 Unlock this chapter to see the answer keys.

  • What validation does image input need, and why would a very small image be rejected?图片输入要做哪些校验?为什么小图片会被拒绝?
    Common in ChinaCommon overseasIntermediate#input-validation#multimodal

    🔒 Unlock this chapter to see the answer keys.

  • How do you design the fallback when a model cannot see images, and how do you find out that it cannot?模型不支持图片时的降级策略怎么设计?你怎么先知道它不支持?
    Common in ChinaCommon overseasDeep dive#capability-detection#graceful-degradation

    🔒 Unlock this chapter to see the answer keys.

Comments