Migrate Function Calling from Chat Completions to Responses API

Migrating function calling from Chat Completions to the Responses API is not an endpoint rename. Your application must still receive a tool request, validate and execute it, return the result under the matching call_id, and ask the model for the final answer. If any link is missing, a request can return HTTP 200 while the user’s task remains unfinished.

This guide starts with an application that already uses chat.completions.create and custom functions. It does not cover the older Completions API or an Assistants migration. The safest first move is to run one non-destructive function in staging, compare its trace with the existing path, and stop if the final answer or execution count differs.

The acceptance path is:

define tool → receive function_call → validate and execute in your app → return function_call_output with the same call_id → verify the final answer

OpenAI’s current migration guide recommends Responses for new projects, while Chat Completions remains supported. That makes a reversible, behavior-preserving migration more useful than a rushed rewrite.

The contract changes at eight points

Two boundaries matter before code changes:

1. A custom function runs in your application, not inside OpenAI. The model proposes a name and arguments; your allowlisted dispatcher decides whether anything runs.
2. response.output can contain more than text or one function call. Do not assume output[0] is the call, and do not use output_text to drive the tool loop.

The official function-calling guide owns the current item names, call_id contract, strict-schema rules, parallel-call behavior, and streaming event names.

Freeze a baseline before changing the endpoint

Record one successful Chat Completions trace using a harmless tool. For example, ask for order ORD-1042, let the application call get_order_status, then capture:

• normalized user input and system/developer instructions;
• exposed tool name and schema version;
• validated arguments;
• tool invocation count and business idempotency key;
• tool result;
• final user-facing answer;
• latency, token usage, and error classification.

Your old loop probably resembles this abbreviated JavaScript:

js
const completion = await client.chat.completions.create({
  model,
  messages,
  tools: [{
    type: "function",
    function: {
      name: "get_order_status",
      description: "Get the current status of an order",
      parameters: {
        type: "object",
        properties: {
          order_id: { type: "string" }
        },
        required: ["order_id"],
        additionalProperties: false
      },
      strict: true
    }
  }]
});

const calls = completion.choices[0].message.tool_calls ?? [];
// Execute each allowlisted call, then append:
// { role: "tool", tool_call_id: call.id, content: JSON.stringify(result) }

Keep this path available during staging. The comparison target is not “the new request succeeded”; it is “the same input produced the same authorized business result, one execution, and an acceptable final answer.”

A complete Responses function loop in JavaScript

The following example uses a local, non-destructive order lookup. It deliberately scans every typed output item, validates arguments, dispatches through an allowlist, caches results by call_id, preserves all returned output items for manual state, and repeats until it receives a final answer.

Install the current OpenAI JavaScript SDK, set OPENAI_API_KEY and OPENAI_MODEL, then save the code as responses-tool-loop.mjs.

js
import OpenAI from "openai";

const client = new OpenAI();
const model = process.env.OPENAI_MODEL;

if (!model) {
  throw new Error("Set OPENAI_MODEL to a model that supports function calling.");
}

const orders = new Map([
  ["ORD-1042", { status: "packed", eta: "2026-07-30" }]
]);

async function getOrderStatus({ order_id }) {
  return orders.get(order_id) ?? { status: "not_found" };
}

const handlers = Object.freeze({
  get_order_status: getOrderStatus
});

const tools = [{
  type: "function",
  name: "get_order_status",
  description: "Return the current status and ETA for an order.",
  strict: true,
  parameters: {
    type: "object",
    properties: {
      order_id: {
        type: "string",
        description: "Order ID in ORD-1234 format"
      }
    },
    required: ["order_id"],
    additionalProperties: false
  }
}];

function parseAndValidate(call) {
  if (!Object.hasOwn(handlers, call.name)) {
    throw new Error(`Tool is not allowlisted: ${call.name}`);
  }

  const args = JSON.parse(call.arguments);
  if (
    typeof args.order_id !== "string" ||
    !/^ORD-\d{4}$/.test(args.order_id)
  ) {
    throw new Error("order_id must match ORD-1234");
  }
  return args;
}

// Replace this in-memory cache with durable storage for production mutations.
const executionLedger = new Map();

async function executeOnce(call) {
  if (executionLedger.has(call.call_id)) {
    return executionLedger.get(call.call_id);
  }

  let result;
  try {
    const args = parseAndValidate(call);
    result = await handlers[call.name](args);
  } catch (error) {
    result = {
      error: "tool_execution_rejected",
      message: error instanceof Error ? error.message : "Unknown error"
    };
  }

  const outputItem = {
    type: "function_call_output",
    call_id: call.call_id,
    output: JSON.stringify(result)
  };
  executionLedger.set(call.call_id, outputItem);
  return outputItem;
}

const instructions = [
  "Use get_order_status when an order ID is provided.",
  "Do not invent an order status.",
  "Explain a not_found result without claiming the order exists."
].join(" ");

const input = [{
  role: "user",
  content: "What is the status of order ORD-1042?"
}];

let finalText = "";

for (let round = 0; round < 5; round += 1) {
  const response = await client.responses.create({
    model,
    instructions,              // resend on every manually managed round
    input,
    tools,
    parallel_tool_calls: false, // safer for the first migration
    store: false
  });

  // Preserve every item, including relevant reasoning items.
  input.push(...response.output);

  const calls = response.output.filter(
    (item) => item.type === "function_call"
  );

  if (calls.length === 0) {
    finalText = response.output_text;
    if (!finalText) {
      throw new Error("Response ended without a function call or final text.");
    }
    break;
  }

  const toolOutputs = await Promise.all(calls.map(executeOnce));
  input.push(...toolOutputs);
}

if (!finalText) {
  throw new Error("Tool loop exceeded the five-round safety limit.");
}

console.log(finalText);

Run it with:

bash
npm install openai
OPENAI_MODEL="your-function-capable-model" node responses-tool-loop.mjs

This is runnable application code, but no live OpenAI response is presented here as an observed test result. Before production, run it with your authorized account and current model, save the typed trace, and compare it with your baseline.

Why each guard exists

• strict: true, required, and additionalProperties: false make the intended schema explicit. The documented strict behavior and supported schema subset can change, so verify them against the current guide instead of relying on an old SDK example.
• Object.hasOwn(handlers, call.name) prevents the model-provided name from becoming arbitrary code execution.
• call.call_id is copied unchanged into function_call_output. It is the protocol join between the model’s request and your result.
• Promise.all(calls.map(...)) accounts for every call. The first rollout still disables parallel tool calls to reduce uncertainty.
• input.push(...response.output) preserves the typed items required by manual history, including relevant reasoning items.
• The round limit prevents an unbounded tool loop.
• Tool errors become structured outputs, allowing the model to explain a rejected or missing lookup without silently retrying a side effect.

For a write operation—refund, email, order update, payment—call_id alone is not sufficient business idempotency. Persist a domain key such as refund:{order_id}:{requested_amount}:{workflow_id} in durable storage, commit the result atomically, and return the saved result on retry. An in-memory Map only demonstrates control flow within one process.

Parallel calls: process all or disable them

Responses can produce multiple function_call items. Never bind results by array index, because the only stable correlation required by the protocol is the original call_id.

For the first rollout, parallel_tool_calls: false gives a simpler trace. If you later enable parallel calls:

• collect every item whose type is function_call;
• validate each call independently;
• apply concurrency limits around downstream systems;
• execute independent reads concurrently, but serialize conflicting writes;
• produce exactly one output item per accepted call;
• preserve each original call_id;
• wait for the complete batch before requesting the next Response.

A negative test should inject two calls plus a non-call output item. The test passes only if both calls execute once, both results return with the correct IDs, and the non-call item is not dispatched.

Choose one state strategy

The Responses API offers more than one way to carry context. The current conversation-state guide documents manual history, chained responses, and durable Conversation objects.

Do not combine previous_response_id with a Conversation in the same request. Also check your organization’s retention requirements before choosing stored state: Responses are stored by default unless you set store: false, and previous input in a response chain still contributes to billed input tokens. These are volatile operational facts—recheck the official state and data-control documentation at rollout time.

Manual state is the most observable first migration. Log item types and IDs, not sensitive content. After parity is proven, move to a server-managed option only if its recovery and retention tradeoffs fit your system.

Migrate the streaming reducer, not just the endpoint

A Chat Completions consumer often concatenates choices[].delta.content. That reducer does not understand typed Responses events.

For function arguments, handle the current events documented by OpenAI:

• response.output_item.added to register the new typed item;
• response.function_call_arguments.delta to accumulate partial argument JSON;
• response.function_call_arguments.done to finalize the arguments;
• response completion and error events to close the trace.

Do not execute on a delta. Buffer by the output item/call identity, parse only after the arguments are complete, then pass the finalized call through the same validator, allowlist, idempotency layer, and call_id correlation used by the non-streaming path.

Your streaming fixtures should split JSON at awkward boundaries—inside a quoted value, after a backslash, and between UTF-8 characters. A visually complete UI is not proof that the tool arguments were complete.

Diagnose failures by the broken contract

Keep transport, model, tool, and business outcomes separate in telemetry. A 200 OK with no matched tool result is an application failure, not a successful migration.

Staging acceptance checklist

Do not release on a code review or one happy-path HTTP response. Require the following evidence:

• The baseline Chat path and candidate Responses path receive the same normalized input, instructions, tool schema version, and safe test data.
• Every response.output item is recorded by type; no code assumes the first item is text or a call.
• Valid arguments execute only an allowlisted handler.
• Invalid JSON, an unknown tool name, and an invalid order ID are rejected without an unintended call.
• Each accepted function_call has exactly one returned function_call_output with the original call_id.
• A two-call fixture proves no call is lost; the initial production configuration explicitly enables or disables parallel calls.
• Retry tests prove a write operation cannot happen twice, including after a process restart.
• Manual, chained, or Conversation state is tested across at least two turns, including instruction persistence and recovery.
• Streaming tests use partial argument events and do not execute before the .done event.
• Mocks and snapshots contain typed output items rather than legacy choices[0] assumptions.
• The final answer is non-empty, grounded in the tool result, and equivalent to the baseline’s business outcome.
• Logs exclude secrets and sensitive tool payloads while retaining response ID, call ID, tool name, schema version, duration, and outcome class.
• A feature flag can return traffic to Chat Completions without replaying an already committed side effect.

Stop the rollout on any unmatched call_id, duplicate mutation, missing parallel call, state-policy regression, or material difference in the final business result. Fix the failing layer, rerun the negative tests, then widen traffic gradually.

Source and provider boundaries

Use OpenAI’s current first-party documentation as the contract owner:

• Migrate to the Responses API
• Function calling
• Conversation state

Azure OpenAI and “OpenAI-compatible” gateways can have different endpoints, authentication, deployment names, state support, retention, streaming, or tool behavior. Test their exact function loop independently; a compatible request shape or HTTP 200 does not prove identical execution semantics.

If your base request or authentication is not stable yet, fix that separately with the OpenAI API key setup guide. If you are still deciding whether the model should call an application function or return schema-constrained data, use the Structured Outputs vs. function calling guide before migrating the loop.

Your next action is concrete: run the harmless order lookup in staging, save both old and new typed traces, confirm one execution and one final answer, then enable the Responses path for a small traffic slice behind a reversible flag.