Dayward AIWeChat Mini Program
MCP in 7 Days: Wire Tools Into Any Agent
Week 1 · D7🔒 LockedAbout 6 hours

Productionizing and Retrospective: Writing Evals for Tools, Versioning, Publishing to npm and a Registry, Observability, and a Capstone Project

Turn a server that runs into a server you can maintain long-term: write an eval set for your tools, manage description changes as breaking changes, add tracing and metrics, and turn the week into one portfolio entry.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Write a re-runnable eval set for a tool, and name the three categories of cases it should cover
  2. Judge whether a change is breaking, and explain why a description change counts as one too
  3. Add tracing and key metrics to a server, and say which one to check first when something goes wrong

The last day. After six days you hold a server that runs, works remotely, can be connected by your own client, and has been through a security checklist. Today answers the final question: how does it survive the next six months. Come back and tick off the three goals above.

Plain-Language Walkthrough

Tools need evals too

Start with an incident so real it is almost boring.

A server had been live three months with not a line of tool code changed when somebody edited search_docs's description from "search the internal documentation library by keyword" to "search documents" — on the grounds that it was too wordy. Nothing alerted after the deploy: unit tests green, zero server errors, clean logs. Two weeks later somebody reported that the assistant had got dumber lately and kept saying it did not know when asked about documentation.

The cause was that the model stopped selecting the tool. It never once failed a call, because it was never called.

That is MCP tools' most counter-intuitive property: whether the model selects it goes wrong earlier than whether the tool itself is correct, and no traditional test covers the former. A unit test checks that given arguments produce the right output; break the description and the arguments never reach your function at all.

So a tool needs another layer of testing, generally called an eval. Its shape is simple: give one user sentence and a tool table, see which tool the model picked (or that it picked none), and compare against the expectation.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • How do you evaluate whether an MCP tool is any good, and what categories of test cases would you design?怎么评估一个 MCP 工具做得好不好?你会设计哪几类测试用例?
    Common in ChinaCommon overseasIntermediate#eval#tooling

    🔒 Unlock this chapter to see the answer keys.

  • When versioning an MCP server, what counts as a breaking change, and why is editing a tool description more dangerous than renaming a field?给一个 MCP 服务端做版本管理时,什么样的改动算破坏性变更?为什么说改一句工具描述比改一个字段名更危险?
    Common in ChinaCommon overseasIntermediate#versioning#tooling

    🔒 Unlock this chapter to see the answer keys.

  • A user reports that one of your MCP tools is 'always getting it wrong' in production. In what order do you investigate?线上有人反馈某个 MCP 工具「总是调不对」。你按什么顺序排查?
    Common in ChinaCommon overseasIntermediate#observability#debugging

    🔒 Unlock this chapter to see the answer keys.

Comments