Dayward AIWeChat Mini Program
From Frontend Engineer to Agent Engineer in 30 Days
Week 1 · D4🔒 LockedAbout 6 hours

Model Integration and System Prompts: a Multi-Provider Abstraction With Fallback, Overriding the Default Persona (dg P03/P04/M04)

Learn to abstract multiple model providers behind one layer with automatic failover, and understand why you must override the framework's default system prompt.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Implement a model-calling layer that supports at least 3 providers, switchable by configuration
  2. Write a custom system prompt for an agent that overrides the framework's default persona
  3. Name the factors a fallback strategy must weigh when a model call fails (latency, cost, availability)

For three days your code has had exactly one provider in it: the address hard-coded into a fetch, and the model name coming, at best, from a MODEL environment variable (step 4 of D1's lab did precisely that). Changing the model id is fine; changing vendors means changing code. Today we take that layer apart properly. When you are done, come back and tick off the three goals.

Plain-Language Walkthrough

Wiring up one model means handing your uptime to somebody else's operations team

Picture a restaurant with a single supplier. Dishes, prices, and how fast food arrives all depend on them. Everything is fine, right up until the morning their truck breaks down — and then you are not having a slightly worse day, you cannot open.

Wiring up a model is the same. Hard-coding openai/gpt-4o-mini binds your service's availability entirely to one vendor's operational quality, and LLM API availability is lower than you think: regional outages, an account tripping a rate limit, a model retired or renamed, a call inexplicably hanging for 60 seconds without returning. These are not freak events; they happen a few times a month.

Do the arithmetic and it becomes obvious why this is not a bet worth taking. Suppose a vendor's monthly availability is 99.5% — which sounds high, and means roughly 3.6 hours a month during which your agent is dead and every user sees a spinner and an error. Wire up three vendors whose failures are uncorrelated and all three going down at once is 0.5% cubed, about 1.25 in ten million: unavailable time drops from 3.6 hours (12,960 seconds) to 0.33 seconds, less than one second. The same order of magnitude of code, four-plus orders of magnitude of availability, roughly forty thousand times better.

That number carries one premise that has to be stated out loud: the failures are uncorrelated. So that you can run today's example and today's lab with a single key, all three providers here go through one aggregation gateway, OpenRouter — which means three model ids, not three independent paths. The gateway going down, or the key being suspended, takes all three out at once, and the aggregator has become the new single point of failure. Real redundancy means direct endpoints at different vendors, with independent credentials, independent data centers, and independent billing; only then does that forty-thousand-times figure count. Interviewers love to enter through this crack, so volunteer it: the order of magnitude assumes independence, and a shared gateway does not satisfy that assumption.

The question is how far the last three days' code sits from wiring up three of them. Look at what you have written:

hardcoded.js
// The past three days: the address hard-coded at the call site, and the model name
// coming from an environment variable at best
const res = await fetch('https://openrouter.ai/api/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: process.env.MODEL ?? 'openai/gpt-4o-mini',
    messages,
  }),
})
const json = await res.json()
// What happens when this vendor goes down? There is no other road, and the whole
// service goes down with it.

Not one line of that is wrong. The problem is that it has only one road. The MODEL variable looks like flexibility, but it swaps a model id, not a vendor: the address, the auth header, and the shape of the response fields are all welded into this one call. When something breaks you can only change a variable, restart, and pray the other vendor's response format happens to match.

There is a more mundane reason too: prices and capabilities change every month. The best-value model today may be beaten two months from now by somebody else's new version at half the price. If switching models means rewriting the calling code and retesting every prompt, you will keep paying for the expensive slow one because switching is too much hassle. A high integration cost is, at bottom, robbing you of the freedom to choose later.

So the first rule of a production-grade agent is: a model is a replaceable part, not a hard-coded foundation.

Which raises the obvious objection — three vendors have three request formats, three response structures, and three ways of reporting errors, so does wiring up three mean writing the calling code three times? The answer is of course to add a layer of abstraction, but the layer is not the hard part. The hard parts are the three things behind it: which classes of error make a retry pure waste (almost everyone puts 404 in the wrong bucket), how fallback triples your bill without you noticing, and a tiered routing scheme that saves over sixty thousand a year. We take them one at a time.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • Why do production agents usually integrate more than one model provider?为什么生产级 Agent 通常要接入多个模型 provider?
    Common in ChinaCommon overseasBasic#model-routing#reliability

    🔒 Unlock this chapter to see the answer keys.

  • What trade-offs shape a model fallback strategy?设计模型 fallback 策略时要权衡哪些因素?
    Common in ChinaCommon overseasIntermediate#model-routing#reliability#cost

    🔒 Unlock this chapter to see the answer keys.

  • What does the system prompt do in an agent, and why not rely on the framework default?系统提示词(system prompt)在 Agent 里起什么作用?为什么不能用框架默认的?
    Common in ChinaCommon overseasIntermediate#prompt-engineering#system-prompt

    🔒 Unlock this chapter to see the answer keys.

  • How do you pick the right model per task, balancing cost against latency?如何在成本和延迟之间给不同任务选择合适的模型?
    Common in ChinaCommon overseasIntermediate#model-routing#cost#latency

    🔒 Unlock this chapter to see the answer keys.

Comments