OpenAI GPT-4o

Overview

GPT-4o ("omni") is OpenAI's flagship multimodal model, designed for real-time interaction across text, audio, and images. It represents a significant leap in speed, efficiency, and multimodal capabilities compared to previous GPT-4 models.

Key Features

🎯 Native Multimodal Processing

GPT-4o processes text, audio, and images in a single neural network:

from openai import OpenAI

client = OpenAI()

# Text input
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is this image showing?"}],
    extra_body={"input_image": "base64_encoded_image"}
)

# Audio input (real-time)
# Use the Realtime API for audio streaming

⚡ Real-Time Response

• Latency: ~320ms average response time
• Streaming: Native streaming support for all modalities
• Efficiency: 2x faster than GPT-4 Turbo

📊 Enhanced Capabilities

Capability	GPT-4o	GPT-4 Turbo
Text	✅ Excellent	✅ Excellent
Images	✅ Native	✅ Via API
Audio	✅ Native	❌ No
Video	⚠️ Frame analysis	❌ No
Speed	2x faster	Baseline
Context	128K tokens	128K tokens

Installation

pip install openai

Quick Start

from openai import OpenAI

client = OpenAI(api_key="sk-...")

# Simple text query
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in simple terms."}
    ]
)

print(response.choices[0].message.content)

Image Analysis

import base64

def encode_image(path):
    with open(path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode('utf-8')

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {
                "url": f"data:image/jpeg;base64,{encode_image('image.jpg')}"
            }}
        ]}
    ]
)

Realtime API (Audio)

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI()

async def realtime_session():
    async with client.beta.realtime.connect(
        model="gpt-4o-realtime-preview-2024-10-01"
    ) as conn:
        # Send audio input
        await conn.input_audio_buffer.append(audio_data)
        
        # Receive response
        async for event in conn:
            if event.type == "response.audio.delta":
                print(event.delta, end="", flush=True)

asyncio.run(realtime_session())

Pricing

Input	Output
$2.50 / 1M tokens	$10.00 / 1M tokens
$1.25 / 1M tokens (cached)	-

Context Window

• 128K tokens - Can process entire books, long documents, or extended conversations

Supported Languages

GPT-4o supports 50+ languages with native fluency:

• English, Spanish, French, German, Chinese, Japanese, Korean
• Portuguese, Italian, Russian, Arabic, Hindi, and more

Best Use Cases

• Real-time assistants - Voice-first applications
• Multimodal analysis - Images + text combined
• Customer support - Fast, natural conversations
• Content creation - High-quality text generation
• Code assistance - Understanding and generating code
• Education - Interactive learning experiences

Limitations

• ❌ No video understanding (frame-by-frame only)
• ❌ Knowledge cutoff: October 2024
• ❌ Can still hallucinate facts
• ❌ Rate limits apply

Resources

• Docs: https://platform.openai.com/docs/models/gpt-4o
• API Reference: https://platform.openai.com/docs/api-reference
• Realtime API: https://platform.openai.com/docs/guides/realtime

---

Last updated: May 2026