A production voice agent has a latency budget of roughly 800 milliseconds before a conversation starts to feel slow. That budget is spent across eight sequential stages, typically split across three to five different vendors depending on how the stack is assembled.
This post maps that voice AI pipeline stage by stage: mic capture, WebRTC transport in, turn detection, speech-to-text, LLM inference, text-to-speech, WebRTC transport back, and playout. Each adds its own piece to the latency budget. Understanding where those milliseconds go makes it easier to know what’s actually worth optimizing.
Voice-to-Voice: Defining the Round Trip
Voice-to-voice latency is the elapsed time from the moment a user stops speaking to the moment the agent’s audio starts playing. Each of the above-mentioned stages contribute to this number, each running on different infrastructure, owned by different vendors, and measured with different methodologies. Some call this voicebot latency, others call it conversational AI latency. Both describe the same measurement.
The path looks like this, following the stage breakdown in our Slow Voicebot? How to Fix Latency in Voice-Enabled Conversational AI Systems post:
- Mic capture and local processing. Echo cancellation, noise suppression, and voice activity detection run on the client before any audio leaves the device.
- WebRTC transport in. Audio travels from the client to the media server over UDP, through STUN/TURN negotiation already established at session start.
- Speech-to-text. The STT engine converts the audio stream into text, typically producing partial transcripts as the user speaks.
- Turn detection. A voice activity detector or a semantic end-of-turn model decides the user has actually finished, not just paused.
- LLM inference. The language model processes the transcript and context, then generates a response, token by token.
- Text-to-speech. The TTS engine converts the LLM’s output text into an audio stream, ideally starting on the first sentence rather than waiting for the full response.
- WebRTC transport out. Synthesized audio travels back to the client over the same media path.
- Jitter buffer and playout. The client buffers incoming audio briefly to smooth out network variance before it reaches the speaker.
WebRTC shows up twice in this list. It is the transport which carries audio in both directions with low, predictable overhead. It does not do speech recognition, language modeling, or synthesis. When a vendor says their “WebRTC latency” is low, they usually mean network transport plus jitter buffering, which is a small fraction of the total budget. The STT, LLM, and TTS stages, which run on top of that transport, are where most of the round trip actually happens. We go deeper on the transport layer itself in Why WebRTC Is the Best Transport for Real-Time Voice AI Architectures.
This breakdown describes a cascade approach to media pipeline processing: separate STT, LLM, and TTS services chained together. This is how most production voice agents are built today. Speech-to-speech models, such as OpenAI’s Realtime API, don’t eliminate this pipeline; they abstract it inside a single multimodal model that handles transcription, reasoning, and synthesis together. The stages listed above still happen, they’re just no longer separately observable or independently swappable. Cascade vs. multimodal processing approaches are explained in more detail in our post: How to Build Voice AI Applications: A Complete Developer Guide.
Where the Voice AI Latency Budget Goes: A Stage-by-Stage Breakdown
The table below ranks each stage by the relative size of its typical contribution to the round trip for a well-optimized cascade pipeline, the standard shape of a voice AI pipeline, not by exact milliseconds. Precise figures vary by provider, model choice, region, and how a given vendor measures its own benchmark. The only figures worth trusting for a real comparison are the ones you measure on your own integration.
| Stage | Relative contribution | Why |
| Mic capture, VAD, echo cancellation | Smallest | Runs locally, on the client’s own hardware |
| WebRTC transport in | Small | Bound by physical distance to the nearest media server; largely fixed |
| Turn / end-of-turn detection | Medium, and directly tunable | Trades against the risk of interrupting the user mid-sentence |
| Speech-to-text | Large | One of the two biggest single contributors in a cascade pipeline |
| LLM time-to-first-token | Large | The other biggest single contributor; varies enormously by model size and hosting |
| Text-to-speech time-to-first-audio | Small to medium | Generally faster than STT or LLM inference, still model-dependent |
| WebRTC transport out | Small | Symmetric with transport in |
| Jitter buffer / playout | Small | Tunable, trades against audio glitching under network jitter |
Stacked end to end, speech-to-text and LLM inference consistently account for most of the round trip. Transport, playout, and mic-side processing are comparatively fixed and small by comparison. The highest-leverage place to spend optimization effort is almost never the network. It’s the two AI inference stages in the middle.
This table is deliberately relative rather than absolute. Exact millisecond figures depend on the specific STT, LLM, and TTS providers in your stack, the region they’re hosted in, and your own prompt and audio characteristics. Before quoting a number externally, measure it on your own integration.
Physics-Bound, Model-Bound, and Architecture-Bound Voice AI Latency
Most production teams aim for sub-second voice agent latency overall, with individual stages tuned well under that to leave room for network variance. The stages in that table fall into three groups.
Physics-bound
WebRTC transport is bound by distance and the speed of light in fiber. Co-locating STT, LLM, and TTS services in the same region as your media servers removes cross-region hops, but it does not remove the underlying transport time. Below roughly 20-30ms one-way for a well-routed session, there is nothing left to optimize; the packet is already taking close to the shortest available path. The same applies to the human side of the conversation: research on conversational turn-taking puts the average human response gap around 200ms, with a natural range up to 500ms. A voice agent that responds well inside that window, often described as sub-500ms latency, does not read as unnaturally slow. There is no latency budget below that point worth spending engineering effort on, because a human listener would not perceive the difference.
Model-bound, but shrinking
STT time-to-final, LLM time-to-first-token, and TTS time-to-first-audio are all a function of model architecture and inference hardware. STT, LLM, and TTS vendors compete most directly on speed, and where the numbers move fastest. Groq and Cerebras compress LLM TTFT by running on custom inference silicon rather than GPUs. Deepgram’s Flux and similar integrated end-of-turn models fold turn detection into the STT pass rather than running a separate VAD, which is a genuine architectural win, not just faster hardware. These gains are real, but they require picking specific models and providers, and they often trade against accuracy: the STT benchmark data available shows a consistent pattern where the fastest models on time-to-first-token are not the most accurate, and the two properties don’t move together.
Architecture-bound
This is the largest lever, and the one most within a vendor’s or integrator’s control. The biggest latency wins don’t come from a faster model. They come from removing sequential dependencies. Streaming text to TTS as the LLM generates it, rather than waiting for the full response, overlaps two stages that would otherwise run back to back. Starting STT processing on partial audio rather than waiting for silence does the same on the front end. Parallel execution of a small, fast model for an immediate acknowledgment alongside a larger model generating the full response is a variant of the same idea, an approach we detail in Reducing Voice Agent Latency with Parallel SLMs and LLMs: hide latency by giving the user something to listen to while the slower stage finishes. None of this requires a faster model. It requires not waiting for one stage to fully finish before starting the next.
Six Levers for Reducing Voice AI Latency in Production
These six levers apply regardless of which specific STT, LLM, and TTS vendors are in the stack.
- Stream every stage that supports it. STT partials, LLM tokens, and TTS audio chunks should each start flowing before the upstream stage is complete. This is the single highest-leverage change available and it costs no additional inference spend, and it’s the core idea behind the parallel-execution approach in Reducing Voice Agent Latency with Parallel SLMs and LLMs.
- Choose models by TTFT and TTFA, not by total generation time. A model that finishes generating a full response faster but takes longer to produce its first token will feel slower to a listener even if the underlying task completes sooner.
- Route by query complexity. Not every turn needs a frontier model. A tiered setup that sends greetings, confirmations, and simple lookups to a fast, smaller model and reserves a larger model for genuine reasoning keeps the median turn fast without giving up capability on the turns that need it.
- Co-locate STT, LLM, TTS, and your media servers in the same region. Cross-region calls between services add pure network overhead with no offsetting benefit, and they’re one of the most common causes of a pipeline that benchmarks well in isolation but performs poorly in production, a point covered in our Slow Voicebot post.
- Measure your own pipeline, not just vendor benchmarks. Vendor-published TTFT and TTFB numbers are a starting point for shortlisting providers, not a substitute for instrumenting your own integration. The gap between a vendor’s published median and what you measure end to end, under your own network conditions and with your own prompt lengths, is often the difference that matters.
- Treat turn detection as a latency lever in its own right. An overly conservative endpointing threshold adds hundreds of milliseconds of silence to every turn before the pipeline even starts. An overly aggressive one causes the agent to interrupt users mid-sentence. Tuning this setting is cheap relative to model selection and has an outsized effect on perceived responsiveness, as our Slow Voicebot post also covers under real-time monitoring and adaptive optimization.
Building the Voice AI Latency Budget That Fits Your Use Case
There is no single correct latency budget for Voice AI. A drive-thru ordering bot and a telehealth intake agent tolerate different amounts of silence, and the acceptable trade-off between speed and accuracy shifts with the stakes of the conversation. What holds across every use case is that the budget should be measured stage by stage, not treated as one opaque number, because that is the only way to know which vendor swap or architectural change will actually move it.
WebRTC.ventures builds the media transport and orchestration layer that sits underneath the STT, LLM, and TTS choices covered here. Whether you’re evaluating a Voice AI stack or building or integrating one, contact WebRTC.ventures. We can run a latency assessment on your stack or help with the integration itself.
Further Reading:
