Skip to main content

Grok Voice Agent API: WebSocket Quickstart, Authentication, and Pricing

Use wss://api.x.ai/v1/realtime for a live Grok voice agent. Start with a server-side text or PCM turn, decode the audio reply, and use short-lived client secrets for browser connections. Current audio pricing is $0.08 per minute, with different duration meters for server VAD and push-to-talk.

LaoZhang AI TeamPublishedUpdated 12 min read
On this page
Conceptual Grok Voice API path from authenticated WebSocket input through response status and decoded WAV audio to speaker playback

The Grok Voice Agent API runs two-way conversations at wss://api.x.ai/v1/realtime. For a first integration, open a server-side connection, configure the session, send a text or audio turn, and collect the returned audio into a playable file. A successful WebSocket connection is only the beginning: your application also needs to handle the response status, audio decoding, and connection shutdown.

The current alias is grok-voice-latest, which points to grok-voice-think-fast-2.0. You can pin that version in the connection URL. Current audio pricing is $0.08 per minute, plus $0.004 per billable client text item. With default server VAD, the audio meter covers session duration, including idle time; push-to-talk uses audio sent and received. xAI model selection, voice pricing and duration meters.

This guide follows the public documentation checked on October 7, 2026. The example was checked offline for JavaScript syntax and WAV-header construction; it was not run against xAI, and no latency or actual billing result is claimed.

The contract in one screen

Integration decisionCurrent documented contract
Direct realtime endpointwss://api.x.ai/v1/realtime
Current modelgrok-voice-latest, or pin grok-voice-think-fast-2.0
Trusted server authenticationAuthorization: Bearer <API_KEY>
Browser authenticationBackend-issued client secret in xai-client-secret.<TOKEN> WebSocket subprotocol
Simple audio formatMono PCM16, little-endian, 24,000 Hz
Automatic voice turnsserver_vad; append audio and let the server trigger the response
Manual audio turnsSet turn_detection.type to null; append, commit, then request a response
Public capacity10 concurrent sessions per team; maximum session duration 120 minutes

The endpoint and event flow come from the Realtime reference; audio configuration comes from the speech-to-speech guide. Capacity figures are the public model-card limits, rather than a measurement of your account's available capacity.

The fastest safe path to a working session

Start with one response saved as a WAV file. This keeps microphone capture and speaker scheduling out of the initial integration while still completing the important path: input → model response → usable audio output.

The Node.js example accepts a built-in text question or a raw PCM file. It selects manual turns, configures JSON audio transport, waits for session.updated, and writes answer.wav only when response.done reports completed. It also prints the spoken transcript. A cancelled, incomplete, failed, or disconnected response produces a failure message and, if audio was received, an answer.partial.wav file.

Use a Node.js release that supports ES modules and install ws in your own project:

bash
npm install ws

Save this as voice-once.mjs. The API key belongs in the server environment; do not place it in this file or bundle it into browser code.

js
import WebSocket from "ws";
import { readFileSync, writeFileSync } from "node:fs";

const key = process.env.XAI_API_KEY;
if (!key) throw new Error("Set XAI_API_KEY in the server environment.");
const rate = 24000;
const inputPath = process.argv[2];
const input = inputPath ? readFileSync(inputPath) : null;
if (input && (!input.length || input.length % 2 !== 0)) {
  throw new Error("Input must contain nonempty PCM16 samples.");
}

function toWav(pcm) {
  if (pcm.length % 2 !== 0) throw new Error("Incomplete PCM16 sample.");
  const header = Buffer.alloc(44);
  header.write("RIFF", 0);
  header.writeUInt32LE(36 + pcm.length, 4);
  header.write("WAVE", 8);
  header.write("fmt ", 12);
  header.writeUInt32LE(16, 16);
  header.writeUInt16LE(1, 20); // PCM
  header.writeUInt16LE(1, 22); // mono
  header.writeUInt32LE(rate, 24);
  header.writeUInt32LE(rate * 2, 28);
  header.writeUInt16LE(2, 32);
  header.writeUInt16LE(16, 34);
  header.write("data", 36);
  header.writeUInt32LE(pcm.length, 40);
  return Buffer.concat([header, pcm]);
}

const ws = new WebSocket(
  "wss://api.x.ai/v1/realtime?model=grok-voice-think-fast-2.0",
  { headers: { Authorization: `Bearer ${key}` } },
);
const chunks = [];
let transcript = "";
let configured = false;
let submitted = false;
let finished = false;
const send = (event) => ws.send(JSON.stringify(event));

function finish(ok, detail) {
  if (finished) return;
  finished = true;
  clearTimeout(deadline);
  try {
    const pcm = Buffer.concat(chunks);
    if (pcm.length) {
      const file = ok ? "answer.wav" : "answer.partial.wav";
      writeFileSync(file, toWav(pcm));
      console.log(`Saved ${file}: ${(pcm.length / (rate * 2)).toFixed(2)}s`);
    } else if (ok) {
      ok = false;
      detail = "Completed response contained no audio.";
    }
    if (transcript) console.log("Assistant:", transcript);
  } catch (error) {
    ok = false;
    detail = `Could not save audio: ${error.message}`;
  }
  if (!ok) {
    console.error("Response failed:", detail);
    process.exitCode = 1;
  }
  // End this one-response session; a live app would keep its socket open.
  ws.terminate();
}

// Application watchdog, not a provider timeout or latency guarantee.
const deadline = setTimeout(() => finish(false, "Local 90s deadline."), 90000);

ws.on("message", (data, isBinary) => {
  if (finished) return;
  try {
    // Text frames can arrive as Node Buffers. We requested JSON output.
    if (isBinary) throw new Error("Unexpected binary frame in JSON output mode.");
    const event = JSON.parse(data.toString());
    if (event.type === "session.created" && !configured) {
      configured = true;
      send({
        type: "session.update",
        session: {
          voice: "eve",
          instructions:
            "Answer in English. Explain the requested idea in two short sentences.",
          turn_detection: { type: null },
          audio: {
            input: { format: { type: "audio/pcm", rate }, transport: "json" },
            output: { format: { type: "audio/pcm", rate }, transport: "json" },
          },
        },
      });
    } else if (event.type === "session.updated" && !submitted) {
      submitted = true;
      if (input) {
        // 20ms at 24kHz, mono, 16-bit: 960 bytes.
        for (let offset = 0; offset < input.length; offset += 960) {
          send({
            type: "input_audio_buffer.append",
            audio: input.subarray(offset, offset + 960).toString("base64"),
          });
        }
        send({ type: "input_audio_buffer.commit" });
      } else {
        send({
          type: "conversation.item.create",
          item: {
            type: "message", role: "user",
            content: [{ type: "input_text", text: "What is a cache miss?" }],
          },
        });
      }
      send({ type: "response.create" });
    } else if (event.type === "response.output_audio.delta") {
      chunks.push(Buffer.from(event.delta, "base64"));
    } else if (event.type === "response.output_audio_transcript.delta") {
      transcript += event.delta;
    } else if (
      event.type === "response.text.delta" ||
      event.type === "response.output_text.delta"
    ) {
      process.stdout.write(event.delta);
    } else if (event.type === "error") {
      const e = event.error || {};
      finish(false, JSON.stringify({
        type: e.type, code: e.code, message: e.message,
        param: e.param, event_id: e.event_id,
      }));
    } else if (event.type === "response.done") {
      const status = event.response?.status;
      finish(status === "completed", `response.done status: ${status}`);
    }
  } catch (error) {
    finish(false, error.message);
  }
});
ws.on("error", (error) => finish(false, error.message));
ws.on("close", (code) => {
  if (!finished) finish(false, `Socket closed before completion: ${code}`);
});

With your server credential already configured, use node voice-once.mjs for the text question, or node voice-once.mjs question.pcm for an audio turn. question.pcm must already be raw, mono, 24 kHz, signed 16-bit little-endian audio. Renaming an MP3, WAV container, or browser recording to .pcm does not convert it. The script only checks byte alignment; a headerless file does not tell it the original sample rate or channel count. Keep file input short for this first example; the loop is not a paced live microphone sender.

Open answer.wav in a local audio player. Success means a completed response with nonempty audio that decodes at the configured rate, accompanied by the spoken transcript when provided. The code uses session.created to configure the session without assuming whether conversation.created arrives before or after it. These events and the manual commit flow follow the Realtime reference.

The WAV writer stores the entire reply before playback. For a live assistant, queue decoded audio as it arrives instead. At this format, one second is 24,000 × 2 = 48,000 bytes; the player needs the same rate and channel count. response.done ends generation, while the player may still have queued audio to play.

Browser-safe authentication with ephemeral tokens

A native browser WebSocket cannot attach an arbitrary Authorization header. Use your backend to issue a short-lived client secret, then pass it through the documented WebSocket subprotocol. The long-lived xAI API key stays on the backend. xAI ephemeral-token guide.

The server-side issuance request is:

http
POST https://api.x.ai/v1/realtime/client_secrets
Authorization: Bearer <SERVER_API_KEY>
Content-Type: application/json

{"expires_after":{"seconds":300}}

The response contains value and expires_at, a Unix timestamp in seconds. Your application should expose an authenticated endpoint that performs this request after checking the caller's access and usage allowance. Return the client secret to that caller over HTTPS, disable response caching, and avoid logging either credential. A public token dispenser would let other people spend your API budget.

Once your application has that endpoint, a browser connection can use this function:

js
async function openVoiceSocket() {
  // Your app implements this route with authentication and a budget check.
  const response = await fetch("/api/voice/client-secret", {
    method: "POST",
    credentials: "same-origin",
    headers: { "Content-Type": "application/json" },
    body: "{}",
  });
  if (!response.ok) throw new Error(`Token request failed: ${response.status}`);
  const { value, expires_at } = await response.json();
  if (typeof value !== "string" || !Number.isFinite(expires_at) ||
      expires_at <= Date.now() / 1000) {
    throw new Error("Missing or expired client secret.");
  }
  return new WebSocket(
    "wss://api.x.ai/v1/realtime?model=grok-voice-think-fast-2.0",
    [`xai-client-secret.${value}`],
  );
}

This function opens a socket; the browser still needs session, input, output, and error handlers. Microphone capture also needs conversion into the selected codec. Browser recording chunks often contain WebM audio, which cannot be sent as raw PCM or raw Opus packets merely by base64-encoding them.

The client-secret reference documents a default lifetime of 600 seconds and a maximum of 3,600 seconds. That lifetime is separate from the maximum voice-session duration. There is a documentation discrepancy worth handling narrowly: the ephemeral-token guide says session and anchor are unsupported, while the reference lists an optional initial session. The minimal expires_after.seconds request above avoids that difference; neither description establishes a tested, universal token-binding policy.

Authentication diagram separating server API keys from short-lived browser client secrets

Audio turns, interruption, and playback

Choose turn detection according to who decides that the user has finished speaking. The audio guide documents these flows:

Input modeClient sendsWho requests the response?
Text messageconversation.item.create with input_textClient sends response.create
Manual audio / push-to-talkAudio appends, then input_audio_buffer.commitClient sends response.create
Automatic audio / server VADAudio appends as speech arrivesServer triggers the response after detecting the turn

For automatic voice turns, change the example's configuration to turn_detection: { type: "server_vad" }, send microphone chunks continuously, and remove the manual audio commit and response.create calls. Leaving those calls in the VAD path can create an extra response request. input_audio_buffer.clear discards pending, uncommitted input; it does not clear audio already queued in your speaker.

Keep the codec separate from the transport

The example uses JSON messages carrying base64 PCM. Binary mode sends raw codec bytes as WebSocket binary frames; configuration, transcripts, errors, and lifecycle events remain JSON. Input and output codecs can differ. Output follows the selected transport, and a mid-session output-transport change applies at the next response boundary. Audio transport documentation.

PCM is the simplest format for the file example. audio/pcmu and audio/pcma use 8 kHz telephony audio. audio/opus uses raw 24 kHz mono packets, one packet per payload, rather than a WebM or MP3 container. In Node's ws client, distinguish text and binary messages using isBinary: a text JSON message can itself be represented by a Buffer.

Stop unheard audio when the user interrupts

In VAD mode, input_audio_buffer.speech_started tells the application to stop its current speaker playback and clear queued assistant audio. Automatic interruption stops server generation, but cannot remove samples already buffered by your player. For manual interruption, send response.cancel as well as stopping local playback.

If the assistant generated more speech than the user heard, truncate the assistant item at the actual played duration, not the total bytes received. The current API supports:

json
{
  "type": "conversation.item.truncate",
  "item_id": "<assistant-item-id>",
  "content_index": 0,
  "audio_end_ms": 1250
}

Here, 1250 is an illustrative playback measurement. Track the current item and content index from response events, use your player's timing, and wait for conversation.item.truncated to acknowledge the change. The event removes audio and transcript content after that point so subsequent turns do not assume the user heard it. Older integrations that treated truncation as unsupported need updating. Current client and server events.

For user captions, enable audio.input.transcription.model: "grok-transcribe". conversation.item.input_audio_transcription.updated contains a cumulative transcript that may revise earlier words: replace the displayed caption rather than appending it. The assistant's response.output_audio_transcript.delta, by contrast, is incremental. An input language_hint biases recognition; session instructions control the language the assistant speaks. Transcription events, language hints.

Pricing, rate limits, and the hidden tool bill

Current rates are $0.08 per audio minute and $0.004 per billable client conversation.item.create text item. Function-result items are exempt from the text-item fee; audio items use the audio meter. response.create itself does not incur a text-item fee, although ensuing audio is billable. Voice model-card pricing, xAI pricing.

The duration definition changes the estimate substantially:

Hypothetical workloadMeter and calculationEstimated voice charges
10-minute server-VAD session, including silence, plus 2 billable text items10 × $0.08 + 2 × $0.004$0.808
Push-to-talk interaction lasting 10 wall-clock minutes, with 2 minutes sent, 3 minutes received, and 2 billable text items(2 + 3) × $0.08 + 2 × $0.004$0.408
One text question answered with 30 seconds of audio in a manual session0.5 × $0.08 + $0.004$0.044

These calculations use documented meters and assumed durations; they are not measured invoices. The hourly audio equivalent is $0.08 × 60 = $4.80. Counting only speaking time underestimates a default server-VAD session, while the entire wall-clock interaction does not match the documented push-to-talk meter.

Tools and outside services add another budgeting question. xAI's general tool schedule lists web search at $5 per 1,000 calls, while X search is priced by fetched items: $5 per 1,000 posts and $10 per 1,000 profiles. Thus 20 web searches correspond to $0.10 under that published search schedule. That arithmetic does not establish an all-inclusive Voice invoice: the general table covers broader API contracts, and your chosen tool route, external service, and telephone provider need applicable pricing. Do not apply a text-model token price to Voice merely because a response includes token-usage counters. Tool pricing and units.

The public model card currently lists 10 concurrent sessions per team, a 120-minute session maximum, and us-east-1 availability. Plan admission control around concurrency and end idle sessions deliberately. Token expiry, session duration, and concurrency describe three different limits.

Let tools finish before asking the agent to continue

Custom functions let your application supply information or perform an authorized action during a conversation. Register each function in session.tools, with its name, description, and argument schema. The model emits response.function_call_arguments.done; your application parses the JSON arguments, validates them, executes the allowed function, and sends its result using the matching call_id. Custom function tools.

For example, a read-only get_service_status tool might return your application's current health information. Once the application has actually obtained result, this helper constructs the response item; it does not execute the lookup:

js
function functionResultItem(callId, result) {
  return {
    type: "conversation.item.create",
    item: {
      type: "function_call_output",
      call_id: callId,
      output: JSON.stringify(result),
    },
  };
}

Use event.call_id, event.name, and parsed event.arguments from the completed function event. Do not let an arbitrary name select arbitrary application code. Return the actual success or failure result rather than fabricated success, and apply your application's authorization checks before side effects.

When a response requests several tools, collect the calls, finish all results, send one result item for each, and then send one response.create. Sending a continuation inside every callback can start the reply before other results arrive. Your controller also needs to wait until the current speaker queue finishes, or explicitly interrupt it, before starting overlapping playback. Parallel tool calling, avoiding audio overlap.

A forced line takes a different path. conversation.item.create with item.type: "force_message" synthesizes supplied text directly; do not send response.create afterward. Setting interruptible: false drops caller audio during playback. This can deliver a scripted greeting, but does not by itself establish consent or prove the caller heard it. Force-message contract.

Errors, reconnects, and OpenAI Realtime migration

Treat generation completion, playback completion, and connection closure as separate events. A long-lived application's next action should depend on the event it received:

Symptom or eventWhat to inspect or do next
Socket fails before openingInspect the connection error, endpoint, model, and server-key or client-secret authentication path. No voice turn has been proven successful.
error eventRecord type, code, message, param, and event_id; correct the identified request before retrying. The documentation says most errors are recoverable, rather than every error being fatal.
response.doneRead response.status: completed, cancelled, and incomplete require different handling. Let remaining speaker audio finish.
Transcript appears but audio is unusableCheck output codec and transport, received audio deltas or binary frames, decoded byte count, and player sample rate. A transcript alone does not prove playback works.
Socket closes mid-responseMark the turn incomplete and stop playback. Reconnect according to the history policy below; do not label partial audio a completed reply.

The one-response example deliberately ends on any API error. A production session may recover from a correctable error without closing, using details and the current event schemas to decide what is safe to retry.

For history across reconnects, opt into resumption.enabled: true on the original session and new connection, save conversation.created.conversation.id, and reconnect with that ID in the conversation_id query parameter. The documented cache expires after 30 minutes of inactivity. Without resumption, do not assume a new socket remembers the old conversation. Session resumption.

The resumption guide describes replayed conversation.item.created events; the reference lists conversation.item.added for new items. A client can recognize both while distinguishing replayed history from new work. Replayed tool calls are historical records, not instructions to repeat a purchase or booking. Keep application operation records and idempotency checks outside the conversation cache.

For an OpenAI Realtime migration, retain the session-and-event architecture but review the actual contract. Current xAI supports both response.text.delta and response.output_text.delta, and documents conversation.item.truncate. Conversely, conversation.item.retrieve remains listed as unsupported. Do not carry over an old compatibility table or infer that direct WebSocket provides every WebRTC or SIP event. Migration and compatibility documentation.

When Voice Agent API is the wrong route

Use the realtime agent when the product needs a conversation: speech input, generated replies, turn-taking, and optionally tools. If the application already decides what to say, a separate speech endpoint can complete the job with a smaller interaction loop:

Reader's taskSuitable xAI route
Live spoken agentwss://api.x.ai/v1/realtime
Transcribe an audio file or URLPOST https://api.x.ai/v1/stt
Stream audio into text without an agent replywss://api.x.ai/v1/stt
Turn predetermined text into speechPOST https://api.x.ai/v1/tts, or the separate TTS WebSocket streaming route

Standalone STT uses grok-voice-transcribe-2.0 as its current documented default; the embedded realtime caption setting uses grok-transcribe. Those IDs belong to different jobs. TTS supports REST and streaming, so choosing it does not necessarily mean waiting for a single complete file. See the current Voice API reference and audio overview for applicable request and response formats.

Route selection diagram contrasting live voice conversation, speech output, and framework-assisted integration

Media frameworks become useful when microphone capture, speaker queues, or telephone routing dominate the work. They do not eliminate the need to understand the chosen model, authentication, or billing meter. Start with the direct example to establish the response path, then add the media layer your product needs.

FAQ

What model should I use for Grok Voice Think Fast 2.0?

Pin grok-voice-think-fast-2.0 in the model query parameter for that version, or use grok-voice-latest to follow xAI's current alias. The alias currently points to 2.0; it is not a promise to remain pinned there indefinitely. Model selection.

How much does a Grok Voice Agent API session cost?

The current audio rate is $0.08 per minute, plus $0.004 per billable client text item. Default server VAD bills session duration; push-to-talk bills sent and received audio. A 10-minute VAD session therefore has a $0.80 audio charge before applicable text items and other services. Voice pricing.

Can I use my Grok subscription instead of an API key?

The documented developer connection requires an xAI API key on a trusted server, or a client secret issued using that server credential. A consumer login or subscription is not the authentication credential shown for this endpoint. Realtime authentication.

Does an ephemeral token expire the voice session after five minutes?

Five minutes is the secret lifetime chosen in the example issuance request. The reference separately lists token expiry, and the model card separately lists a 120-minute session maximum. Do not treat the issuance lifetime as a documented five-minute session cap. The precise behavior of an already-open connection at credential expiry was not tested here. Client-secret reference, session limits.

Is response.done enough to tell the user the reply finished?

It tells you generation ended, and you must still inspect response status. A streaming player's queue can continue after that event. Show playback completion when the queue has drained, or report that the reply was interrupted if playback stopped. Realtime response events.

Do I need the Voice Agent API just to generate speech?

Use the separate TTS API when you already have text and need audio output. Use standalone STT for transcription. The realtime Voice Agent API is the route for a spoken conversation that produces its own replies. xAI audio routes.

Sources7

External pages this guide links to, in the order they appear. Last updated Oct 8, 2026.

  1. 1.xAI model selectiondocs.x.ai/developers/model-capabilities/audio/speech-to-speech
  2. 2.voice pricing and duration metersdocs.x.ai/developers/models/speech-to-speech
  3. 3.Realtime referencedocs.x.ai/developers/rest-api-reference/inference/voice
  4. 4.xAI ephemeral-token guidedocs.x.ai/developers/model-capabilities/audio/ephemeral-tokens
  5. 5.xAI pricingdocs.x.ai/developers/pricing
  6. 6.current event schemasdocs.x.ai/voice-realtime.ws.json
  7. 7.audio overviewdocs.x.ai/developers/model-capabilities/audio/voice
More in API Guides