Gemini 3.5 Transcribe API: Recorded Audio, Live Captions, and a Safe First Build

Use gemini-3.5-transcribe when the audio already exists as a recording. Use gemini-3.5-transcribe-live when text must appear while someone is speaking. That choice determines the API, media pipeline, output structure, supported features, and maximum duration.

These models are dedicated speech-to-text routes. They are not aliases for Gemini 3.5 Flash, a spoken Live Agent, or Gemini 3.5 Live Translate. If the product must answer questions about an audio file, speak back, translate speech into another language, or call tools, transcription should be one explicit stage in a larger workflow rather than the only model call.

Google introduced both developer routes in public preview on August 26, 2026. The current Gemini 3.5 Transcribe model page is the contract to recheck before shipping.

Start with the output, then choose the route

Recorded processing is the safer first route when immediate captions are not a hard requirement. It accepts common audio files, supports richer annotations, and avoids a persistent connection. Live transcription earns its operational complexity only when incremental text changes the user experience.

Verify one recorded file before adding features

The current audio transcription guide uploads a file first, then sends the returned URI to the Interactions API. Install the current Google Gen AI SDK and keep the key in an environment variable:

bash
python -m pip install -U google-genai
export GEMINI_API_KEY="YOUR_API_KEY"

The smallest useful Python request is deliberately plain:

python
from google import genai

client = genai.Client()
audio_file = client.files.upload(file="sample.mp3")

interaction = client.interactions.create(
    model="gemini-3.5-transcribe",
    input=[
        {
            "type": "audio",
            "uri": audio_file.uri,
            "mime_type": audio_file.mime_type,
        }
    ],
)

print(interaction.output_text)

Treat this as a transport test, not a quality verdict. A successful first run means the upload completed, the interaction completed without an error, output_text is non-empty, and the text corresponds to the submitted audio. Log the request status, model ID, audio duration, MIME type, and whether output was empty. Do not add language hints, vocabulary biasing, smart cleanup, speakers, and timestamps all at once; a failed combined request is much harder to diagnose.

The API can automatically detect language and code-switching. When the language is known, a supported BCP-47 hint such as en-US can reduce ambiguity. A custom vocabulary can contain up to 1,000 terms, although Google says customers generally see the best results with up to 100. Begin with names, product terms, abbreviations, and alphanumeric patterns that are both important and genuinely uncommon.

Clean prose and traceable transcripts require different modes

The default verbatim mode preserves disfluencies, repetitions, and self-corrections. It is the appropriate base when the transcript is an auditable record, when subtitles must align to audio, or when downstream code needs speakers and word offsets.

smart mode removes filler words, resolves corrections, and formats items such as numbers and lists for readability. It is useful for dictation or a clean draft, but it cannot be combined with diarization or word-level timestamps. That is an API boundary, not something a prompt can fix.

For a traceable transcript, enable structure under mode:

python
interaction = client.interactions.create(
    model="gemini-3.5-transcribe",
    input=[
        {
            "type": "audio",
            "uri": audio_file.uri,
            "mime_type": audio_file.mime_type,
        }
    ],
    generation_config={
        "transcription_config": {
            "custom_vocabulary": ["Gemini", "BigQuery"],
            "mode": {
                "type": "verbatim",
                "diarization_mode": "speaker",
                "timestamp_granularities": ["word"],
            },
        }
    },
)

interaction.output_text remains the convenient merged transcript. Speaker labels and word start/end offsets live in content annotations. If an application needs SRT cues, click-to-seek playback, speaker time, or evidence tied to a recording, saving only output_text silently throws away the required structure.

Google currently documents up to eight diarized speakers, with attribution for three or more marked experimental. Word timestamps may also reduce transcription accuracy. A three-person meeting should therefore be tested as a quality boundary, not treated as a guaranteed easy case.

Live transcription is a media state machine

Live transcription accepts a continuous stream and emits changing hypotheses. The Live transcription guide currently requires raw 16-bit PCM, 16 kHz, mono, little-endian audio in 100 ms chunks. Browser microphones commonly produce WebM/Opus, and phone systems may produce 8 kHz μ-law. The MIME label does not perform decoding or resampling; the bytes must actually match the declared format.

A minimal connection can receive finalized segments like this:

python
import asyncio
from google import genai
from google.genai import types

client = genai.Client()

config = types.LiveConnectConfig(
    response_modalities=["TEXT"],
    input_audio_transcription=types.AudioTranscriptionConfig(
        language_codes=[],
    ),
)


async def receive_transcripts(session):
    async for response in session.receive():
        content = response.server_content
        if content and content.interim_input_transcription:
            update_caption_preview(content.interim_input_transcription.text)
        if content and content.input_transcription:
            commit_final_segment(content.input_transcription.text)


async def main():
    async with client.aio.live.connect(
        model="gemini-3.5-transcribe-live",
        config=config,
    ) as session:
        receiver = asyncio.create_task(receive_transcripts(session))

        async for pcm_chunk in your_pcm_chunk_source():
            await session.send_realtime_input(
                audio=types.Blob(
                    data=pcm_chunk,
                    mime_type="audio/pcm;rate=16000",
                )
            )

        await session.send_realtime_input(audio_stream_end=True)
        await receiver


asyncio.run(main())

your_pcm_chunk_source, update_caption_preview, and commit_final_segment are application boundaries, not SDK functions. Interim text should replace the current preview; final text should be appended to committed transcript state. Persisting every interim update creates duplicates and leaves words that the model later revised.

For a browser or mobile client that connects directly, do not embed a permanent Gemini API key. The official guide provides constrained, short-lived ephemeral tokens that a trusted server creates for the client. A production Live path also needs session renewal before the 10-minute limit, reconnection handling, transcript deduplication, explicit end-of-stream behavior, and metrics for time to first partial and time to final.

Limits and price are architecture inputs

As of August 27, 2026, the Gemini Developer API pricing page estimates recorded Transcribe at about $0.003/min for audio input plus $0.002/min for text output, or roughly $0.005/min blended. Live Transcribe is about $0.005/min input plus $0.004/min output, or roughly $0.009/min blended. Billing still follows actual token consumption.

At those estimates, 100 hours is approximately $30 for recorded transcription or $54 for Live. Those figures exclude storage, decoding, media servers, failed retries, downstream summarization, monitoring, and human correction.

The free-tier rows currently show no input or output charge, but also say free-tier data is used to improve Google products; the paid-tier rows say it is not. Sensitive calls, internal meetings, legal material, financial data, or health information require a policy and consent decision before upload. “Free” is not a privacy conclusion. Account and regional availability should be checked separately; the pricing page does not guarantee that every reader can enable the same route.

Evaluate the fields that can hurt the business

Google reports an average Word Error Rate of 4.0% for streaming and 2.6% for non-streaming, citing Artificial Analysis, and reports faster final transcription than Chirp 3. These launch figures are useful as a reason to test, not as a guarantee for a specific accent, microphone, room, vocabulary, or workflow.

Build a small gold set from the workload that will actually ship. Include clean and noisy audio, near and far microphones, accents, rapid speech, code-switching, overlapping speakers, and the codecs used by production. Score high-cost fields separately: names, order IDs, addresses, dates, amounts, medical terms, and commitments. For Live, measure partial churn, duplicate final segments, disconnect recovery, and final latency. For recorded audio, inspect speaker attribution and seek accuracy, not only aggregate WER.

A model can have a strong overall WER and still fail the one entity your workflow cannot afford to corrupt. Track human correction time and cost per accepted transcript alongside the API bill.

Diagnose the contract before changing prompts

• No speakers or timestamps: confirm the recorded model, verbatim mode, the two configuration fields, and annotation parsing. output_text alone is not the structured result.
• Structure disappears in smart mode: expected behavior; choose clean prose or traceable annotations, then perform controlled downstream cleanup if both are needed.
• Live captions repeat or roll back: replace interim state, commit only final segments, then inspect the actual PCM format and chunk cadence.
• A successful transcript cannot summarize or call tools: expected for the dedicated Transcribe model. Send the verified transcript to a suitable second model.
• Long sessions truncate or fail: design file segmentation around the 60/30-minute boundary and Live renewal around ten minutes.

The safest rollout is a short, non-sensitive recorded file first; add only the structure the product needs; then build the Live path separately if immediate text is essential. Recheck the preview contract and Gemini API pricing and key boundaries before launch, and promote the model only after the real gold set, failure logs, cost, and data handling all meet a written threshold.