ElevenLabs shipped Eleven Realtime on Oct 20, 2026, introducing two speech-to-speech models, v1 Mini and v1 Max, that speak OpenAI's Realtime protocol. For developers already using @openai/agents-realtime over WebSocket, the migration is surprisingly minimal: a new endpoint URL and a model string. However, those locked into WebRTC transports face a significant hurdle, as the new API currently supports WebSocket only, forcing a transport layer rewrite before any code changes.

The Architecture and Cost Reality

Under the hood, Eleven Realtime utilizes a cascaded stack of speech recognition, turn-taking, LLM, and voice synthesis, tuned end-to-end but exposed as a single model ID. The pricing structure reveals a critical insight for builders: connected audio costs $0.08 per minute regardless of the model, while token costs vary significantly. For v1 Mini, input tokens are $0.20 per million and output tokens are $6.00 per million. In contrast, v1 Max charges $1.80 per million input tokens and $9.00 per million output tokens. This means for many use cases, the per-minute audio fee dominates the total cost, making prompt optimization less impactful than call duration management.

Migration Path and Technical Catches

The migration process involves minting a short-lived client secret on the server using the ElevenLabs API key, ensuring the master key never touches the browser. On the client side, developers simply swap the WebSocket URL to wss://api.elevenlabs.io/v1/convai/realtime and update the model string to eleven_realtime_v1_mini or eleven_realtime_v1_max. A major catch for existing OpenAI Realtime apps is the loss of automatic audio handling. The OpenAI SDK only wires mic and speaker automatically over WebRTC; over WebSocket, developers must manually capture and play PCM16 audio at 24 kHz, adding complexity to the frontend implementation.

Model Selection and Performance Benchmarks

Choosing between v1 Mini and v1 Max depends heavily on the agent's function. v1 Mini is optimized for fast, natural conversation and scores 95.8% on Full-Duplex-Bench for pauses and interruptions, making it ideal for chatbots and tutors. v1 Max, however, excels at complex reasoning and tool use, scoring 87.1% on tool-use task completion compared to Mini's 60.3%. If your agent needs to book slots or look up orders, Mini will drop roughly 4 in 10 tasks, making Max the necessary choice despite its higher token costs. Both models support 90+ languages and access to over 17,000 voices in the ElevenLabs library.

Key Takeaways

  • WebSocket users can migrate with a two-line change, but WebRTC users must rewrite their transport layer first.
  • Connected audio at $0.08/min is the primary cost driver for v1 Mini, while input token bloat significantly impacts v1 Max costs.
  • v1 Max is required for reliable tool calling (87.1% success vs. 60.3% for Mini), but v1 Mini offers better turn-taking for conversational agents.
  • Server-side OpenAI Realtime SDK compatibility is currently untested, limiting backend integration options.

The Bottom Line

If you are already on WebSocket, switching to ElevenLabs Realtime is a low-friction upgrade that offers better voice quality and cost transparency for conversational agents. However, WebRTC developers should treat this as a full transport migration project, not a simple API swap, and carefully audit their tool-calling workflows to avoid the 40% failure rate inherent in using v1 Mini for complex tasks.