Source of raw audio input
AI Models & Assistants / Knowledge library
GPT-Realtime-1.5
The best voice model for audio in, audio out
Why learn GPT-Realtime-1.5?
Offers the documented OpenAI model capabilities through hosted.
The problem it solves
Provides model inference for supported text / audio / image inputs.
Where it came from
GPT-Realtime-1.5: The best voice model for audio in, audio out Published aliases: gpt-realtime-1.5. Input: text, audio, image. Output: text, audio. Checked 2026-10-13. Model capabilities are distinct from tools available in a particular agent.
Build your understanding
Work through these exercises in order. Try the task before opening a reference.
- Explain what GPT-Realtime-1.5 does.
- Identify a use case for GPT-Realtime-1.5.
- Describe a limitation of GPT-Realtime-1.5.
- Name a compatible tool for GPT-Realtime-1.5.
- Compare GPT-Realtime-1.5 against an alternative.
Documentation and sources
- Primary source / model card (developers.openai.com)
Catalog research notes
- See the AI Workbench for source dates, pricing evidence and configuration details.
Connect the ideas
Built on
Real World Example
Automated Emergency Triage for Remote Field Clinics
Remote medical field clinics operating in low-connectivity, high-stress environments.
The business problem
Medical staff in remote areas often face critical delays when manually documenting patient symptoms while simultaneously performing life-saving procedures, leading to incomplete triage records and potential diagnostic errors.
How this topic solves it
By integrating GPT-Realtime-1.5, clinics can deploy a hands-free voice agent that listens to patient-provider interactions in real-time. The model processes raw audio input to generate structured clinical notes and immediate triage recommendations, allowing providers to maintain focus on the patient rather than data entry.
- Establish a persistent WebSocket connection to the v1/realtime endpoint to stream raw audio from the clinic environment.
- Configure the session to utilize function_calling to trigger external medical database lookups based on detected symptoms.
- Enable prompt_caching for standardized medical protocols to reduce latency during high-frequency triage sessions.
- Process the model output as audio to provide immediate, synthesized guidance back to the field clinician.
The flow of audio data from the clinic to the model and back.
Transport layer for audio
Audio processing and reasoning
Storage for structured notes
- Field Clinician → streams audio → WebSocket Stream
- WebSocket Stream → input modalities → GPT-Realtime-1.5
- GPT-Realtime-1.5 → triggers function call → Electronic Health Record
Implementation example
This code demonstrates the core interaction pattern for Automated Emergency Triage for Remote Field Clinics, showing how audio is streamed to the model and how function calling is used to integrate medical protocols.
// Establish connection to the realtime endpoint
const ws = new WebSocket('wss://api.openai.com/v1/realtime?model=gpt-realtime-1.5');
// Send audio chunks as they are captured from the microphone
ws.send(JSON.stringify({
type: 'input_audio_buffer.append',
audio: base64EncodedAudioData // Raw audio stream from clinician
}));
// Handle function calling for medical database lookups
ws.on('message', (data) => {
if (data.type === 'function_call') {
// Execute local medical protocol lookup
}
});Expected business outcome
Clinicians achieve higher focus on patient care, resulting in more comprehensive and accurate triage documentation generated concurrently with the examination.
Limitations and checks
The model relies on continuous audio streaming; network jitter or packet loss in remote areas can interrupt the session. To check for this, developers must monitor the WebSocket connection state and implement local audio buffering to ensure the model receives a coherent stream.