The Responses API and Built-in Tools: Function Calling, Web Search / File Search / Computer Use, Structured Output

Today's Goals

1. Explain the three differences between the Responses API and Chat Completions in input shape, output shape, and state management
2. Write a minimal script with function calling and web search, and correctly feed function_call_output back in
3. Get structured output with strict json_schema mode, and handle the refusal branch

For two days you used the finished product. Today we open a layer: what the interface products like Codex use to talk to the model looks like, how tools get wired in, and where multi-turn state lives. Once you have read this and finished the work, scroll back to the top and tick off the three goals.

Plain-Language Walkthrough

From messages to items: more than a field rename

Still the two contractors. You sent a job to each firm, and the two firms use different work-order formats. The old format (Chat Completions) is a chat transcript: messages one after another, each labeled with who said it. The new format (the Responses API) is more like a work log: it holds not only what was said but what was looked up, which tool was called, and what the tool returned, each as a typed entry. OpenAI calls the latter items — the input is an array of items, the output is an array of items, and a message is only one of the types.

Three differences are worth getting exactly right, because they decide how you write code:

1. Input shape: Chat Completions takes a messages array with the system prompt as its first entry; Responses takes input (either a string or an array of items), with system-level instruction in a separate top-level instructions field.
2. Output shape: Chat Completions returns choices and you read choices[0].message.content; Responses returns an output array holding, in order, everything the model did this turn (tool calls, searches, messages), plus an output_text helper in the SDK to get the final text directly.
3. State management: Chat Completions is stateless and you resend the history every turn; Responses defaults store to true, saving this response so the next turn only needs previous_response_id and no resent history. You can also turn storage off and maintain the history yourself — the last section covers that trade-off.

The minimal call looks like this:

import OpenAI from 'openai'
 
const client = new OpenAI() // reads the OPENAI_API_KEY environment variable
 
const response = await client.responses.create({
  model: process.env.OPENAI_MODEL ?? '',
  instructions: 'You are an assistant for a TODO API project. Keep answers short.',
  input: 'Explain what input validation is in one sentence',
})
 
console.log(response.output_text)

import os
from openai import OpenAI
 
client = OpenAI()  # reads the OPENAI_API_KEY environment variable
 
response = client.responses.create(
    model=os.environ["OPENAI_MODEL"],
    instructions="You are an assistant for a TODO API project. Keep answers short.",
    input="Explain what input validation is in one sentence",
)
 
print(response.output_text)

Read that alongside the Chat Completions request from D1 of the 30-day course: messages became input plus instructions, and choices[0].message.content became output_text. This course always reads the model id from the OPENAI_MODEL environment variable — OpenAI's model names change quickly, hard-coding one in the prose expires within months, and the current recommendation in the official documentation is the authority.

Why change the format at all? Because the chat-transcript model of Chat Completions cannot hold tools. Once the model is not merely replying but calling tools, searching the web, and reading files, forcing those actions into "messages" gets awkward. Items give every kind of action its own type, which is the precondition for the built-in tools of the next few sections to slot in cleanly.

Function calling: declare, request, feed back

When the contractor needs something from your internal systems, they cannot fetch it themselves — you fetch it and tell them the result. Function calling is that round trip: you declare which functions can be called and what their parameters look like; the model, when it judges it necessary, requests a call with arguments; your code actually executes it and feeds the result back, and the model continues from there.

The three steps correspond to three kinds of item. The declaration is an entry in the request's tools array with type function, carrying name, description, parameters as a JSON Schema, and strict: true to require the arguments match the schema exactly. The model's request is a function_call item in the output, carrying call_id (the id of this call) and arguments (a JSON string). Feeding back means placing a function_call_output item in the next turn's input, with a matching call_id and the result as output.

import OpenAI from 'openai'
 
const client = new OpenAI()
const model = process.env.OPENAI_MODEL ?? ''
 
const todos = [{ id: 1, title: 'Add validation to POST /todos', done: false }]
 
const tools: OpenAI.Responses.Tool[] = [
  {
    type: 'function',
    name: 'list_todos',
    description: 'List all current TODOs',
    parameters: { type: 'object', properties: {}, additionalProperties: false },
    strict: true,
  },
]
 
const first = await client.responses.create({
  model,
  tools,
  input: 'What is still unfinished?',
})
 
// Find the function calls the model requested, actually execute them, then feed the results back
const calls = first.output.filter((item) => item.type === 'function_call')
const outputs = calls.map((call) => ({
  type: 'function_call_output' as const,
  call_id: call.call_id,
  output: JSON.stringify(todos.filter((t) => !t.done)),
}))
 
const second = await client.responses.create({
  model,
  tools,
  previous_response_id: first.id, // continue from the last turn; no history to resend
  input: outputs,
})
 
console.log(second.output_text)

import json
import os
from openai import OpenAI
 
client = OpenAI()
model = os.environ["OPENAI_MODEL"]
 
todos = [{"id": 1, "title": "Add validation to POST /todos", "done": False}]
 
tools = [
    {
        "type": "function",
        "name": "list_todos",
        "description": "List all current TODOs",
        "parameters": {"type": "object", "properties": {}, "additionalProperties": False},
        "strict": True,
    }
]
 
first = client.responses.create(model=model, tools=tools, input="What is still unfinished?")
 
# Find the function calls the model requested, actually execute them, then feed the results back
outputs = [
    {
        "type": "function_call_output",
        "call_id": item.call_id,
        "output": json.dumps([t for t in todos if not t["done"]]),
    }
    for item in first.output
    if item.type == "function_call"
]
 
second = client.responses.create(
    model=model,
    tools=tools,
    previous_response_id=first.id,  # continue from the last turn; no history to resend
    input=outputs,
)
 
print(second.output_text)

Note two engineering details. First, call_id must match verbatim, since that is how the model pairs a result with a request; three functions requested in one turn means three results fed back. Second, strict: true and additionalProperties: false are a pair — the former makes the model's arguments conform to the schema, the latter forbids extra fields; drop either and you have to handle malformed arguments defensively in your code. What those two lines buy you is an argument-parsing layer that needs essentially no try/catch.

Look back at D1: Codex reading files and running commands is this same round trip underneath, except its tools are the shell and the file system, and it turns the loop itself. The twenty-odd-line script you now have in hand is one iteration of a minimal agent loop.

Built-in tools: web search and file search are run for you by the platform

The functions of the last section are executed by you. Some tools are so universal that OpenAI simply executes them server-side, and you only declare them in tools — those are the built-in tools. They are declared in the same array as function tools, differing in type, and they need nothing fed back.

web search: { type: 'web_search' }. The model searches on its own when it judges it necessary, the output gains a web_search_call item recording what it searched for, and the final message item carries annotations in which each url_citation points at a source. search_context_size (low / medium / high) controls how much page content is pulled into the context, and filters.allowed_domains restricts it to certain sites — very practical for question answering over an enterprise's own knowledge.

file search: upload files into a vector store first, then declare { type: 'file_search', vector_store_ids: ['...'], max_num_results: 3 }. The model retrieves from that store when it needs to, the output gains a file_search_call item, and the annotations in the message are file_citation. Adding include: ['file_search_call.results'] to the request returns the retrieved passages as well, which helps when debugging what it actually saw. This is essentially a hosted RAG: the pipeline hand-built in D12 of the 30-day course compressed into one field.

import OpenAI from 'openai'
 
const client = new OpenAI()
 
const response = await client.responses.create({
  model: process.env.OPENAI_MODEL ?? '',
  tools: [{ type: 'web_search', search_context_size: 'low' }],
  input: 'Which libraries does the community commonly use for request body validation in Express? Cite sources.',
})
 
console.log(response.output_text)
 
// List the cited sources separately: annotations live in the content of the message item
for (const item of response.output) {
  if (item.type !== 'message') continue
  for (const part of item.content) {
    if (part.type !== 'output_text') continue
    for (const a of part.annotations) {
      if (a.type === 'url_citation') console.log('source:', a.url)
    }
  }
}

import os
from openai import OpenAI
 
client = OpenAI()
 
response = client.responses.create(
    model=os.environ["OPENAI_MODEL"],
    tools=[{"type": "web_search", "search_context_size": "low"}],
    input=(
        "Which libraries does the community commonly use for request body "
        "validation in Express? Cite sources."
    ),
)
 
print(response.output_text)
 
# List the cited sources separately: annotations live in the content of the message item
for item in response.output:
    if item.type != "message":
        continue
    for part in item.content:
        if part.type != "output_text":
            continue
        for a in part.annotations:
            if a.type == "url_citation":
                print("source:", a.url)

How do you choose between a built-in tool and a function of your own? The rule: data that lives outside and is generic (the public web, documents you uploaded) goes to a built-in tool, and data that lives inside your system (databases, internal services, business logic) gets a function. Built-in tools spare you the execution and the feeding back, but you do not control how they search or what they find; function tools require you to write everything, but every step is in your hands. Production systems almost always mix the two.

Computer use: letting the model operate an interface, and why it belongs in isolation

One step further: what if the task is "open this admin console and change the status of these tickets," there is no API to call, and all that exists is a web interface? That is what computer use addresses — letting the model see screenshots, decide where to click, and decide what to type. The documentation currently offers two routes: have the model write code to drive the interface (generating a Playwright script, say, executed in an isolated environment that persists across calls), or use the { type: 'computer' } computer tool, where the model returns structured actions (click, type, scroll, keypress, screenshot, and so on), your code executes them, and you send the new screenshot back, round and round.

No code for this today, because it does not belong in a minimal script. What the documentation stresses repeatedly is this: confine it to an isolated browser or virtual machine, and give it an allowlist of sites and actions. A model that can click any button and type any text cannot run on your everyday desktop — it might click Delete All, and it might be induced by a stretch of text on a page to do something else entirely (which is prompt injection in its interface-operating form). So the right starting point for computer use is not code but the environment: a throwaway container, a restricted account, an allowlist.

It is covered here so you can see the full spectrum of tools: from the safest web search (read-only public web), through file search (read-only over the files you supplied), through function calling (your code, your control of the side effects), to computer use (the model producing side effects directly). The further right, the more capable, and the heavier the isolation required. The Codex sandbox from D1 is the isolation solution for the "let the model run a shell" cell of that spectrum.

Structured output: strict mode, the parse helper, and refusal

Back to the contractor. You ask for a progress report and get a page of prose from which you still have to dig the numbers out — better to hand them a form to fill in. Structured output is that form: you supply a JSON Schema and the model's output is guaranteed to match it.

In the Responses API it goes in the text.format field: { type: 'json_schema', name, schema, strict: true }. strict is the key — with it on, the output is guaranteed to validate against the schema rather than being merely very likely valid JSON. Both SDKs offer helpers so you write less schema: in TypeScript, zodTextFormat converts a zod schema; in Python, you pass a pydantic model directly. Then use responses.parse instead of create and the result arrives already parsed in output_parsed.

import OpenAI from 'openai'
import { z } from 'zod'
import { zodTextFormat } from 'openai/helpers/zod'
 
const client = new OpenAI()
 
const Review = z.object({
  verdict: z.enum(['approve', 'request_changes']),
  issues: z.array(z.object({ file: z.string(), line: z.number(), note: z.string() })),
})
 
const response = await client.responses.parse({
  model: process.env.OPENAI_MODEL ?? '',
  input: 'Review this diff: … (elided)',
  text: { format: zodTextFormat(Review, 'review') },
})
 
// Check for a refusal first: refusal is its own content type, not a parse failure
const refused = response.output
  .filter((item) => item.type === 'message')
  .flatMap((item) => item.content)
  .find((part) => part.type === 'refusal')
 
if (refused) {
  console.log('the model refused:', refused.refusal)
} else {
  console.log(response.output_parsed?.verdict, response.output_parsed?.issues.length)
}

import os
from openai import OpenAI
from pydantic import BaseModel
 
client = OpenAI()
 
 
class Issue(BaseModel):
    file: str
    line: int
    note: str
 
 
class Review(BaseModel):
    verdict: str  # approve / request_changes
    issues: list[Issue]
 
 
response = client.responses.parse(
    model=os.environ["OPENAI_MODEL"],
    input="Review this diff: … (elided)",
    text_format=Review,
)
 
# Check for a refusal first: refusal is its own content type, not a parse failure
refused = next(
    (
        part
        for item in response.output
        if item.type == "message"
        for part in item.content
        if part.type == "refusal"
    ),
    None,
)
 
if refused:
    print("the model refused:", refused.refusal)
else:
    review = response.output_parsed
    print(review.verdict, len(review.issues))

That refusal branch is easy to overlook. When the model declines for safety reasons it does not force invalid JSON on you but returns a content block of type refusal. Ignore it and your code receives a response with an empty output_parsed and explodes downstream; handle it and you can distinguish "the model was unwilling" from "the model got it wrong" — which is mandatory in automated review or classification pipelines.

Strict mode solves whether the format is right; it does not solve whether the content is right. The schema guarantees verdict is one of two enum values but guarantees nothing about the judgment being correct. Interviewers often press on this: structured output removes the defensive code in your parsing layer, and removes not one line of validation from your business layer.

Multi-turn state: previous_response_id or carrying the history yourself

One last question: where does the history of a multi-turn conversation live? Responses offers two routes.

Let the server store it: store: true by default saves every response, so the next turn continues with previous_response_id and the request body carries only what is new. The upside is small requests and simple code, which is how the function calling example above was chained. The cost is that the state lives elsewhere: rewinding to turn three means remembering each turn's id; auditing the full conversation means fetching it separately; and where compliance requires data not to leave a jurisdiction, this route is closed.

Maintain it yourself: with store: false, every turn resends the complete items array (including the earlier function_call and function_call_output) as input. That is the carry-your-own-history approach from D1 of the 30-day course: cost grows with turns, but every byte is in your hands, and you can compress it, truncate it, or replay it against a different model.

Which one? Prototypes and internal tools take the first for convenience; user-facing production systems mostly take the second, or a hybrid — chaining short runs with previous_response_id while persisting a copy for audit and recovery. Tomorrow's Agents SDK packages this choice as sessions, and you will see the same trade-off under a new shell.

Source Reading

Hands-On Lab

The code is in labs/codex-mastery/day-03-responses-tools, with four exercise points cut out of starter/ and complete answers in solution/. This is a one-shot script rather than a REPL, so pass arguments directly and do not pipe input into it.

1. Run MOCK=1 pnpm start "hello" and watch the simplest responses.create round trip print output_text.
2. Do exercises 1 and 2: declare the list_todos function tool, filter function_call out of the output, execute it and feed the result back as function_call_output, then run MOCK=1 pnpm start "what is still unfinished" and see the TODO title in the final answer.
3. Do exercise 3: with --search, add web_search to tools and print the url_citation entries separately out of the message annotations.
4. Do exercise 4: with --structured, switch to responses.parse and zodTextFormat, checking refusal before reading output_parsed.
5. Configure OPENAI_API_KEY and OPENAI_MODEL, run once without MOCK, and compare whether the shape of the mock output matches the real one.

Interview Questions

Today's 3 questions are in the question bank below, weighted toward the trade-off between the Responses API and Chat Completions, the boundary between built-in tools and custom functions, and the reliability of structured output. Expand each one and read the analysis before the key points — practicing the derivation beats memorizing bullets.

Checklist and Tomorrow

• Explain the three differences between the Responses API and Chat Completions in input shape, output shape, and state management
• Write a minimal script with function calling and web search, and correctly feed function_call_output back in
• Get structured output with strict json_schema mode, and handle the refusal branch
• Order web search, file search, function calling, and computer use by "more capability, heavier isolation" and justify it
• All 4 acceptance criteria of the lab pass
• Answer at least 2 of the 3 interview questions without looking at the key points

Tomorrow (D4) we fold today's hand-written loop into a few lines of API with the official Agents SDK: an Agent object holds the instructions and the tools, and a run function turns the loop for you. Then we take up three things today did not touch — how several agents hand off to each other (handoffs), how to put guardrails at the entry and the exit (guardrails), and how to hand multi-turn memory to the SDK to store (sessions). Hand-writing before using the SDK is a deliberate order: you already know what the function_call and function_call_output round trip is, so when the SDK hides them tomorrow you will know what got hidden and which layer to look in when something breaks.