Source of raw audio input
AI Models & Assistants / Knowledge library
GPT-4o Realtime
Model capable of realtime text and audio inputs and outputs
Why learn GPT-4o Realtime?
Offers the documented OpenAI model capabilities through hosted.
The problem it solves
Provides model inference for supported text / audio inputs.
Where it came from
GPT-4o Realtime: Model capable of realtime text and audio inputs and outputs Published aliases: gpt-4o-realtime-preview-2025-06-03, gpt-4o-realtime-preview-2024-12-17, gpt-4o-realtime-preview-2024-10-01. Input: text, audio. Output: text, audio. Checked 2026-10-13. Model capabilities are distinct from tools available in a particular agent.
Build your understanding
Work through these exercises in order. Try the task before opening a reference.
- Explain what GPT-4o Realtime does.
- Identify a use case for GPT-4o Realtime.
- Describe a limitation of GPT-4o Realtime.
- Name a compatible tool for GPT-4o Realtime.
- Compare GPT-4o Realtime against an alternative.
Documentation and sources
- Primary source / model card (developers.openai.com)
Catalog research notes
- See the AI Workbench for source dates, pricing evidence and configuration details.
Connect the ideas
Built on
Real World Example
Multilingual Emergency Dispatcher Assistance
Emergency response centers handling high-stress, multi-language calls.
The business problem
Traditional text-based or sequential audio processing introduces critical latency, preventing dispatchers from understanding urgent, non-native language distress signals in real-time.
How this topic solves it
The GPT-4o Realtime model enables bidirectional audio streaming via WebSockets, allowing the system to process incoming distress calls and provide immediate, low-latency conversational support or translation without the delay of separate transcription and synthesis steps.
- Establish a persistent WebSocket connection to the v1/realtime endpoint.
- Stream raw audio packets from the emergency caller directly to the model session.
- Configure the model to output audio responses that guide the caller through triage protocols.
- Utilize function_calling to trigger automated dispatch alerts based on extracted location or incident data.
- Monitor session health and latency metrics to ensure continuous, uninterrupted communication.
The flow of audio data from a caller through the Realtime model to the dispatcher interface.
v1/realtime connection
Processes audio and triggers functions
Receives audio responses and alerts
- Emergency Caller → streams audio → WebSocket Interface
- WebSocket Interface → forwards packets → GPT-4o Realtime
- GPT-4o Realtime → sends audio response → Dispatcher Console
- GPT-4o Realtime → triggers dispatch alert → Dispatcher Console
Implementation example
This demonstrates the WebSocket connection to the v1/realtime endpoint required to stream audio directly to the model for immediate processing.
// Initialize WebSocket for realtime audio
const ws = new WebSocket('wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2025-06-03');
// Send audio chunks as binary frames
ws.send(audioBuffer);
// Handle incoming audio responses from the model
ws.onmessage = (event) => { /* Playback audio response */ };Expected business outcome
Dispatchers receive immediate, context-aware assistance, reducing the time required to assess and categorize emergency situations regardless of the caller's language.
Limitations and checks
The model's reliance on a persistent WebSocket connection means that network instability can cause audio gaps; developers must implement robust reconnection logic and monitor packet loss to ensure session reliability.