LLM API Basics: messages/roles, Tokens, Streaming, Temperature; What an Agent Actually Is
Starting from messages/roles, tokens, streaming, and temperature, get clear on what an LLM API actually does, and what an agent has that a plain chat doesn't.
Starting from messages/roles, tokens, streaming, and temperature, get clear on what an LLM API actually does, and what an agent has that a plain chat doesn't.
Interview questions
What are tokens and the context window, and how do they shape agent design?什么是 token 和上下文窗口?它们如何影响 Agent 的设计?
Common in ChinaCommon overseasBasic#llm-basics#context🔒 Unlock this chapter to see the answer keys.
What do the system / user / assistant roles do, and why does system exist?messages 里的 system / user / assistant 三种角色各起什么作用?为什么要有 system?
Common in ChinaCommon overseasBasic#llm-basics#prompt🔒 Unlock this chapter to see the answer keys.
Why do LLM apps stream responses, and how do you choose between SSE and WebSockets?为什么 LLM 应用几乎都用流式输出?SSE 和 WebSocket 该怎么选?
Common in ChinaCommon overseasIntermediate#streaming#protocol🔒 Unlock this chapter to see the answer keys.
What fundamentally separates a chatbot from an agent?聊天机器人和 Agent 的本质区别是什么?
Common in ChinaCommon overseasBasic#agent-basics🔒 Unlock this chapter to see the answer keys.
What do temperature and top_p control, and when would you use 0 versus 0.7?temperature 和 top_p 分别控制什么?什么场景用 0,什么场景用 0.7?
Common overseasBasic#llm-basics#sampling🔒 Unlock this chapter to see the answer keys.
A streaming reply is cut off mid-way. What do the client and server each do, and can EventSource auto-reconnect help?流式回复到一半网络断了,前端和后端各要做什么?EventSource 的自动重连能用上吗?
Common in ChinaCommon overseasIntermediate#streaming#reliability#sse🔒 Unlock this chapter to see the answer keys.
The user backgrounds the app or closes the tab. How do you restore a reply that was still being generated?用户切到后台或者直接关掉网页,回来后怎么恢复那条还在生成的回复?
Common in ChinaCommon overseasDeep dive#streaming#reliability#architecture🔒 Unlock this chapter to see the answer keys.
After a retry, how do you avoid double billing and re-executing tool calls that already ran?断线重试之后,怎么保证不重复计费、也不重复执行已经做过的工具调用?
Common in ChinaCommon overseasDeep dive#reliability#tools#idempotency🔒 Unlock this chapter to see the answer keys.
On mobile, connectivity is flaky. How would you design the reconnection strategy for a chat feature?移动端 App 里的对话,网络频繁抖动,你会怎么设计重连策略?
Common in ChinaCommon overseasIntermediate#reliability#mobile#streaming🔒 Unlock this chapter to see the answer keys.
A user pressing stop and a dropped connection both look like a closed connection server-side. How do you tell them apart?用户主动点「停止生成」和网络意外断开,在服务端看起来都是连接没了,怎么区分处理?
Common in ChinaCommon overseasDeep dive#streaming#reliability#ux🔒 Unlock this chapter to see the answer keys.
