OpenAI Whisper API: Speech-to-Text in Production Apps

If you have wired up a transcription endpoint, watched it work perfectly on a 30-second test clip, and then watched it fall over on a 90-minute recording, this guide is for you. The OpenAI Whisper API is straightforward to call and genuinely hard to run well at scale. The gap between the two is where most teams lose a week.

This tutorial covers the production side of speech-to-text: choosing between the transcription models, working around the 25 MB upload ceiling, getting word-level timestamps for captions, streaming partial results, and structuring the background job pipeline that makes all of it reliable. Code examples are in Python and Node, with the failure modes called out as they appear.

What the OpenAI Whisper API Actually Gives You

The OpenAI Whisper API is a hosted speech-to-text service that accepts an audio file and returns a text transcript. It exposes two endpoints: /v1/audio/transcriptions, which transcribes audio in its original language, and /v1/audio/translations, which transcribes and translates into English. Both accept mp3, mp4, mpeg, mpga, m4a, wav, and webm files up to 25 MB.

That description covers the whole API surface, which is deceptively small. Notably, the service is stateless: there is no job ID, no polling, and no place to retrieve a transcript you forgot to save. Each call is a single HTTP request that either returns a transcript or fails. Consequently, every durability concern — retries, partial results, storage, ordering — belongs to your code, not OpenAI’s.

That design is fine for a demo. However, it means a 45-minute podcast episode is not one API call. It is a chunking strategy, a retry policy, and a stitching step that you have to build.

Which Transcription Model Should You Use?

OpenAI has shipped several models behind the same endpoint, and they are not interchangeable. Picking the wrong one is the most common early mistake, because the differences only surface once you need timestamps or speaker labels.

Model	Best for	Word timestamps	Streaming	Speaker labels
whisper-1	Captions, subtitles, searchable audio	Yes	No	No
gpt-transcribe	General-purpose accuracy, newest option	No	Yes	No
gpt-4o-transcribe	Noisy audio, accents, domain jargon	No	Yes	No
gpt-4o-mini-transcribe	High-volume, cost-sensitive workloads	No	Yes	No
gpt-4o-transcribe-diarize	Meetings, interviews, multi-speaker calls	No	No	Yes

The practical rule is short. If you need word-level timing, use whisper-1, because it is the only model that returns it. If you need speaker attribution, use gpt-4o-transcribe-diarize. For everything else, start with one of the GPT transcription models, since they handle accented speech and background noise better than the original Whisper model does.

Translation is a similar special case. The /v1/audio/translations endpoint only supports whisper-1 and only outputs English. Therefore, if you need Spanish audio turned into French text, transcribe first and translate the text separately with a chat model.

Setting Up Your First Transcription Call

Start with the smallest thing that works, then harden it. Install the SDK and set your key as an environment variable rather than hardcoding it.

pip install openai
export OPENAI_API_KEY="sk-..."

Here is a minimal Python transcription with the error handling you will actually want on day one:

import os
from pathlib import Path
from openai import OpenAI, APIError, APITimeoutError

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=120.0)

def transcribe(path: Path, language: str | None = None) -> str:
    """Transcribe a single audio file under 25 MB.

    `language` is an ISO-639-1 code ("en", "de"). Passing it skips
    language detection, which cuts latency and prevents the model from
    guessing wrong on short or noisy clips.
    """
    try:
        with path.open("rb") as audio:
            result = client.audio.transcriptions.create(
                model="gpt-4o-transcribe",
                file=audio,
                response_format="json",
                language=language,
            )
        return result.text
    except APITimeoutError:
        # Long files legitimately exceed the default timeout; retry with backoff.
        raise
    except APIError as exc:
        # 4xx errors are usually a bad file format or an oversized upload.
        raise RuntimeError(f"Transcription failed ({exc.status_code}): {exc.message}") from exc

Why this works: the explicit timeout=120.0 matters more than it looks. Transcription latency scales with audio duration, so the SDK’s default timeout will fire on longer files while the request is still healthy. Meanwhile, passing language when you already know it removes an entire inference step and stops the model from misidentifying a short clip as the wrong language.

The Node equivalent uses a read stream instead of a buffer, which keeps memory flat regardless of file size:

import fs from "node:fs";
import OpenAI from "openai";

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY, timeout: 120_000 });

export async function transcribe(filePath, language) {
  const result = await client.audio.transcriptions.create({
    model: "gpt-4o-transcribe",
    file: fs.createReadStream(filePath),
    response_format: "json",
    language,
  });

  return result.text;
}

If you are new to the OpenAI SDK surface generally, the patterns in building apps with the OpenAI API apply here too — the client configuration, error classes, and retry semantics are shared across endpoints.

How Do You Handle the 25 MB File Limit?

Two techniques solve this, and you should reach for them in order: compress first, chunk second.

Compression alone handles a surprising number of cases. Transcription models downsample audio to 16 kHz mono internally, so shipping a 48 kHz stereo WAV wastes bandwidth without improving accuracy. Re-encoding to 16 kHz mono MP3 typically shrinks a file by an order of magnitude with no meaningful transcript difference.

# Normalize to 16 kHz mono MP3 before upload.
# -ac 1 = mono, -ar 16000 = 16 kHz sample rate, -b:a 32k = low but sufficient bitrate
ffmpeg -i meeting.wav -ac 1 -ar 16000 -b:a 32k meeting.mp3

# Check the result before uploading
ls -lh meeting.mp3
# -rw-r--r--  1 user  staff   9.8M  meeting.mp3

When compression is not enough — roughly anything past 90 minutes — split the file. Importantly, split on time boundaries with overlap rather than on exact byte counts, because a cut mid-word produces two garbled fragments.

import subprocess
from pathlib import Path

CHUNK_SECONDS = 600  # 10 minutes stays well under 25 MB at 16 kHz mono

def split_audio(source: Path, out_dir: Path) -> list[Path]:
    """Split audio into fixed-length segments using ffmpeg's segment muxer.

    ffmpeg cuts on the nearest frame boundary, so segments are gapless
    when concatenated — no audio is lost between chunks.
    """
    out_dir.mkdir(parents=True, exist_ok=True)
    pattern = str(out_dir / "chunk_%03d.mp3")

    subprocess.run(
        [
            "ffmpeg", "-hide_banner", "-loglevel", "error",
            "-i", str(source),
            "-f", "segment",
            "-segment_time", str(CHUNK_SECONDS),
            "-ac", "1", "-ar", "16000", "-b:a", "32k",
            "-reset_timestamps", "1",
            pattern,
        ],
        check=True,
    )

    return sorted(out_dir.glob("chunk_*.mp3"))

Next, transcribe the chunks in order and carry context forward. The prompt parameter accepts up to 224 tokens on whisper-1 and acts as a hint about what the model is about to hear. Feeding it the tail of the previous chunk keeps names, jargon, and sentence flow consistent across the seam.

def transcribe_long_audio(source: Path, work_dir: Path) -> str:
    chunks = split_audio(source, work_dir)
    transcripts: list[str] = []
    context = ""

    for chunk in chunks:
        with chunk.open("rb") as audio:
            result = client.audio.transcriptions.create(
                model="whisper-1",
                file=audio,
                response_format="json",
                prompt=context,  # tail of previous chunk keeps terminology stable
            )
        transcripts.append(result.text)
        # Keep the last ~200 characters as context for the next segment.
        context = result.text[-200:]

    return " ".join(transcripts)

Why the prompt carry-over matters: without it, each chunk is transcribed in isolation. As a result, a product name spelled correctly in chunk one frequently comes back misspelled in chunk two, and mid-sentence splits produce awkward capitalization. Passing context costs nothing and measurably improves seam quality on technical content.

Getting Word-Level Timestamps for Captions and Search

Captions, audio search, and click-to-seek transcripts all need timing data. Only whisper-1 provides it, and only when you request verbose_json together with an explicit granularity.

with open("interview.mp3", "rb") as audio:
    result = client.audio.transcriptions.create(
        model="whisper-1",
        file=audio,
        response_format="verbose_json",
        timestamp_granularities=["word", "segment"],
    )

# Segments are sentence-ish blocks; words are individual tokens with timing.
for segment in result.segments:
    print(f"[{segment.start:.2f} -> {segment.end:.2f}] {segment.text}")

for word in result.words[:5]:
    print(f"{word.word}: {word.start:.2f}s")

Segments map cleanly onto subtitle cues, whereas words map onto search highlighting and karaoke-style playback. In practice, most teams store both: segments drive the UI, and words power the search index.

One caveat deserves emphasis. Word timings are estimates derived from the model’s alignment, not from a forced aligner. Therefore, they are accurate enough for seeking and highlighting but not for frame-exact editing. If your product does precise audio editing, treat these values as a starting point that a human adjusts.

Streaming Transcripts for Faster Perceived Latency

A user staring at a spinner for 40 seconds assumes the app is broken. Streaming fixes the perception problem even though total processing time stays the same. The GPT transcription models support stream=true, which emits transcript.text.delta events as text becomes available and a final transcript.text.done event when the file is complete.

with open("call-recording.mp3", "rb") as audio:
    stream = client.audio.transcriptions.create(
        model="gpt-4o-mini-transcribe",
        file=audio,
        response_format="json",
        stream=True,
    )

    buffer: list[str] = []
    for event in stream:
        if event.type == "transcript.text.delta":
            buffer.append(event.delta)
            print(event.delta, end="", flush=True)  # push to your client here
        elif event.type == "transcript.text.done":
            final_text = event.text

Two constraints shape how you use this. First, whisper-1 does not support streaming at all, so choosing streaming means giving up word timestamps. Second, the deltas arrive over Server-Sent Events, which means your own transport needs to forward them incrementally rather than buffering the whole response. The trade-offs between SSE and WebSockets for exactly this kind of forwarding are covered in streaming LLM responses with SSE vs WebSockets.

For genuinely live audio — a phone call being transcribed as it happens — file-based streaming is the wrong tool. Streaming a completed file only accelerates the delivery of results; it does not let you feed a microphone into the endpoint. That case requires a realtime session API instead.

Building a Production Transcription Pipeline

The single biggest architectural mistake is transcribing inside the HTTP request that receives the upload. A 60-minute file takes minutes to process, which exceeds nearly every load balancer and gateway timeout in existence. Move transcription to a background worker before you do anything else.

The pipeline that holds up in production looks like this:

1. Client uploads directly to object storage using a presigned URL, so audio bytes never pass through your API servers.
2. The API enqueues a job containing the object key, the requested model, and the tenant ID — then returns 202 Accepted immediately.
3. A worker normalizes the audio with ffmpeg to 16 kHz mono, which both shrinks the file and standardizes formats.
4. The worker chunks if needed, transcribes each chunk with retries, and stitches the results.
5. Results land in the database as a transcript row plus segment rows, keyed by job ID.
6. The client learns it is done through a webhook, an SSE stream, or polling a status endpoint.

Steps one and three carry most of the operational value. Direct-to-storage uploads keep large files off your application servers entirely, and the presigned URL patterns in AWS S3 best practices apply directly. Meanwhile, normalizing early means the rest of the pipeline only ever sees one audio format.

Retries need care, because transcription failures are not uniform. Rate limits and 5xx errors are worth retrying; a malformed file never will be.

import time
from openai import APIStatusError, APIConnectionError

RETRYABLE_STATUS = {408, 429, 500, 502, 503, 504}

def transcribe_with_retry(path: Path, max_attempts: int = 4) -> str:
    delay = 2.0

    for attempt in range(1, max_attempts + 1):
        try:
            return transcribe(path)
        except APIStatusError as exc:
            if exc.status_code not in RETRYABLE_STATUS or attempt == max_attempts:
                raise  # 400 = bad file. Retrying will never fix it.
            time.sleep(delay)
            delay *= 2  # exponential backoff
        except APIConnectionError:
            if attempt == max_attempts:
                raise
            time.sleep(delay)
            delay *= 2

    raise RuntimeError("unreachable")

Why the status check matters: blind retry loops on a 400 burn your rate limit budget and delay the failure the user needs to see. Distinguishing retryable from terminal errors is the difference between a queue that drains and a queue that jams. For deeper treatment of backoff, jitter, and queue-level throttling, see LLM rate limiting and retry strategies.

One more detail on the worker side: stream files to and from disk rather than loading them into memory. A worker that reads three 24 MB files into buffers concurrently is a worker that gets OOM-killed under load. Since the SDKs accept file handles and read streams directly, this costs you nothing beyond remembering to use them.

Improving Accuracy on Domain-Specific Audio

Out of the box, transcription models handle conversational speech well and technical vocabulary poorly. Product names, drug names, ticker symbols, and internal acronyms come back phonetically mangled. Three levers fix most of it.

Pass the language explicitly. Language detection is a guess made from the first few seconds of audio. On a clip that opens with silence or music, that guess goes wrong often enough to matter.

Use the prompt parameter as a vocabulary hint. The prompt is not an instruction — it is a sample of the kind of text the model should expect. Consequently, listing the terms verbatim works better than describing them.

# Good: the terms appear as they should be spelled.
VOCAB_PROMPT = (
    "The call discusses Kubernetes, PostgreSQL, pgvector, Terraform, "
    "and the customer accounts Acme Corp and Zenith Robotics."
)

# Bad: an instruction the model has no mechanism to follow.
BAD_PROMPT = "Please spell all technical terms correctly and use proper punctuation."

Post-process with a chat model when stakes are high. Feeding the raw transcript plus a glossary into a chat completion cleans up terminology reliably, and it separates the two concerns cleanly: the audio model handles acoustics, while the text model handles domain knowledge. Constrain that cleanup step to a fixed schema so the corrected transcript stays machine-parseable rather than turning into free-form prose.

Watch for one well-documented Whisper behavior: on silent, music-only, or heavily noisy segments, the model sometimes emits plausible-sounding text that was never spoken — often a repeated phrase or a stray “Thank you for watching.” Trimming leading and trailing silence with ffmpeg’s silenceremove filter eliminates most occurrences. Additionally, flag any segment whose text repeats more than twice for human review.

Real-World Scenario: Transcribing a Support Call Backlog

Consider a small backend team at a B2B SaaS company adding transcript search to a support product. The archive holds thousands of call recordings ranging from four minutes to well over an hour, and new calls arrive continuously during business hours.

The first implementation transcribed synchronously inside the upload endpoint. Predictably, it worked in staging with short test clips and started returning gateway timeouts the moment real recordings hit it. Anything past roughly ten minutes of audio exceeded the load balancer’s request timeout, and because there was no job record, a timed-out request left no trace to retry.

The rewrite changed three things over a couple of weeks. First, uploads went directly to object storage and transcription moved into a queue-backed worker. Second, the backlog and the live traffic were split into separate queues, so a large historical import could not starve same-day calls. Third, the team standardized on whisper-1 with word timestamps, because click-to-seek playback turned out to be the feature customers actually cared about — which ruled out the streaming-capable models entirely.

The trade-off was real and worth naming. Word timestamps cost them streaming, so agents wait for a complete transcript instead of watching text appear. For a support archive, where transcripts are read after the call rather than during it, that was the correct compromise. For a live agent-assist product, it would have been the wrong one.

What Does the OpenAI Whisper API Cost?

Billing differs by model family, which affects architecture more than most teams expect. The whisper-1 model bills per minute of audio, making cost trivially predictable: duration times a fixed rate. The GPT transcription models bill per token across audio input, text input, and text output, so cost varies with speech density rather than wall-clock duration.

That difference has a practical consequence. Per-minute billing means silence costs the same as speech, so trimming dead air genuinely saves money. Per-token billing means a fast talker costs more than a slow one at identical durations. Check the current rates on OpenAI’s pricing page before modeling your unit economics, since these numbers change.

Three habits keep spend predictable regardless of model. Trim silence before upload. Cache transcripts by file hash, because the same recording gets re-submitted more often than you would guess. Finally, track per-tenant audio minutes in your own database — the general approach in token counting and budget management for LLM apps transfers directly to audio workloads.

When to Use the OpenAI Whisper API

• You need accurate multilingual transcription without training or hosting a model
• Your audio arrives as discrete files: recordings, voicemails, uploads, podcast episodes
• You want captions or subtitles and need word-level or segment-level timing
• Transcription volume is variable enough that a per-request API beats reserved GPU capacity
• Your team has no ML infrastructure and no appetite for acquiring one

When NOT to Use the OpenAI Whisper API

• You need true realtime transcription of a live stream, where a realtime session API fits better
• Regulatory or contractual rules prevent audio from leaving your infrastructure
• Volume is high, steady, and predictable, at which point self-hosted inference often costs less
• You need sub-second latency on short utterances, such as voice command interfaces
• Your audio is dominated by overlapping speakers and you need reliable turn-by-turn separation beyond what diarization delivers

Common Mistakes with the OpenAI Whisper API

• Transcribing inside the HTTP request. Long files exceed gateway timeouts, and a timed-out request leaves nothing to retry.
• Uploading raw high-bitrate audio. Models downsample to 16 kHz mono anyway, so the extra bytes buy nothing and push you into chunking sooner.
• Splitting files by byte offset. Byte-boundary cuts land mid-frame and produce corrupted segments. Split on time with ffmpeg instead.
• Choosing a model before checking feature support. Word timestamps exist only on whisper-1; streaming exists only on the GPT transcription models. Discovering this after building the UI is expensive.
• Retrying every error identically. A 400 from a malformed file never succeeds on attempt four. Separate retryable status codes from terminal ones.
• Ignoring hallucinated text on silence. Untrimmed silence produces confident-sounding invented phrases. Trim it, then flag repeated output for review.
• Storing only the flattened transcript. Once you discard segments and timing, adding search highlighting later means re-transcribing everything.

Conclusion and Next Steps

The OpenAI Whisper API is easy to call and demanding to operate. Almost every production problem traces back to three decisions: picking a model whose feature set matches your product, moving transcription into a background worker before the first real file arrives, and normalizing audio to 16 kHz mono so that compression handles what would otherwise require chunking. Get those right and the rest is ordinary queue engineering.

Start with a single job: take your longest existing audio file, normalize it with ffmpeg, and run it through the chunking function above. The seams in that transcript will tell you more about your accuracy needs than any benchmark will. If your next requirement is live rather than recorded audio, building voice agents with the Gemini Live API covers the realtime session model that file-based transcription cannot provide.