Dayward AIWeChat Mini Program
Prompt Engineering From Scratch in 5 Days
Week 1 · D4🔒 LockedAbout 4 hours

Iteration and Evaluation: Small Test Sets, A/B Testing, Version Control, Common Anti-Patterns

Turn "this version feels better" into "this version got three more of ten test cases right": build a small test set, run an A/B comparison, version your prompts, and recognize the most common anti-patterns.

Today's goals 0/3

Sign in to tick these off and save your progress.

Turn "this version feels better" into "this version got three more of ten test cases right": build a small test set, run an A/B comparison, version your prompts, and recognize the most common anti-patterns.

Sign in and link WeChat to read

Sign in, then link your WeChat mini program account on the account page. Linked accounts share one free article per day. Unlock additional articles with rewarded videos in the mini program.

Mini program QR code

Scan in WeChat to open Dayward AI. Open the same course and chapter, choose the rewarded video, then refresh this page after completion.

Interview questions

  • How do you build a test set for a prompt? How would you choose ten samples, and where do the expected answers come from?怎么给一个提示词建测试集?十条样本该怎么挑,标准答案从哪来?
    Common in ChinaCommon overseasBasic#evaluation#test-set

    🔒 Unlock this chapter to see the answer keys.

  • How do you version prompts? How does it differ from versioning code, and what is your first move when production misbehaves?提示词版本化怎么做?它和代码版本管理有什么不同?线上出问题你先做什么?
    Common in ChinaCommon overseasIntermediate#prompt-versioning#evaluation

    🔒 Unlock this chapter to see the answer keys.

  • Is using a model to grade another model's output reliable? When is it acceptable, and when must a human look?用模型给模型打分靠谱吗?什么时候可以用,什么时候必须人工看?
    Common in ChinaCommon overseasIntermediate#evaluation#llm-as-judge

    🔒 Unlock this chapter to see the answer keys.

Comments