On September 22–23, I was in Chicago for the IIT Real-Time Communications Conference at Illinois Tech, an annual IEEE conference where industry engineers and academic researchers present work on real-time communications. This year’s tracks covered programmable networks, WebRTC applications, emergency communications, and AI in real-time communications.
I had the privilege of presenting a talk in the WebRTC track titled “eBPF Makes Python a Good Language for a TURN Server.” Across the other sessions and tracks, three concerns came up repeatedly:
- Scale: carrying more real-time traffic without overloading infrastructure or budgets.
- Latency: keeping interactions responsive as each second of a conversation requires more processing.
- Trust: keeping real-time systems safe when no human reviews their output before it reaches a user.
Let’s start with scale, because that’s where my talk lived.
My talk: eBPF Makes Python a Good Language for a TURN Server
If you run WebRTC at scale, you already know the quiet truth: media doesn’t always go peer-to-peer. Restrictive NATs, enterprise firewalls, and privacy-conscious clients all push traffic onto your TURN servers. TURN (Traversal Using Relays around NAT) is the relay that makes WebRTC “just work” when a direct connection isn’t possible, and because everyone leans on it, it quietly becomes the bottleneck nobody plans for.
My talk showed how an eBPF program can forward TURN ChannelData packets inside the kernel once a channel is bound, leaving only control messages for the userspace server. The demo used a TURN server written in Python to show that the offload works independently of the control plane’s language, so the same approach applies in front of Coturn or STUNner. Stay tuned for a post on the full build.
Scale was my corner of the conference. The moment I stepped into the other sessions and tracks, the conversation shifted to latency, and specifically to voice AI.
The latency budget of real-time voice AI
Voice AI was everywhere this year. The recurring question was brutally practical: when a user stops talking, why does it take the agent so long to answer? Karthik Kaiplody (CCC Intelligent Solutions) framed it as a budget problem in his talk “Where Did My 800 Milliseconds Go?” That ~800ms of silence between a human finishing and the agent replying gets spent across a pipeline:
- Turn detection: deciding the human has actually stopped, not just paused
- Speech-to-text (STT): transcribing what they said
- LLM: generating a response
- Text-to-speech (TTS): turning that response back into audio
His line that stuck with me: “endpointing is the jitter buffer of AI.” Just as a jitter buffer trades a little latency for smoother playback, turn detection is where you trade responsiveness against the risk of interrupting the user. Every stage offers a streaming-vs-wait tradeoff, and the cost of streaming is throwaway work when a user changes direction mid-sentence.
In a recent WebRTC.ventures blog post, my colleague Jen Oppenheimer maps that budget across all eight pipeline stages including WebRTC transport and playout, in The Voice AI Latency Budget: Where Every Millisecond Goes.
Krishna Tyagi (Cisco Systems) picked up the same thread from the enterprise side, where CRM integrations, tool calls, and PII all sit in the path. His focus was yield latency: how long it takes the agent to stop talking when a human interrupts it, especially when the agent is busy running a tool and isn’t “listening.” His prescription was a tight loop — observe, isolate, tune, validate — with advanced voice activity detection separating a quick interruption from a genuine turn handoff.
In How Voice AI Agents Handle Interruption: State Machines vs Streaming Approaches, Arin Sime interviews three of our Voice AI engineers about how their production agents detect a real interruption, cancel generation, and flush queued audio so the agent stops talking.
Latency you can measure. Trust is harder, and that was the next theme.
Trust and security when the safety net is gone
Neha Prakash (KR3 Information Systems) opened with a point that reframes the whole space: voice AI is in production now, and the traditional safety net is gone. There’s no human reviewing outputs before they reach a caller in real time. That exposes a new risk surface, voice cloning and impersonation, prompt injection through insecure tool calling, and data leakage in transcripts and session memory.
Her practical framework layered defenses without wrecking the latency budget:
- Authenticate at the edge as the first line of defense
- Constrain the action space so the agent can only do what it should
- Segment and redact sensitive data
- Run guardrails in parallel rather than in series
- Escalate to a human when risk crosses a threshold
The key insight is that security and latency are in direct tension here, so you risk-tier the checks: let low-risk actions through, run synchronous checks only for high-risk ones, pre-compile guardrails, and push heavy analysis to async paths. It’s the same budgeting discipline from the voice AI talks, applied to safety instead of speed.
Our CTO, Alberto Gonzalez, shows how to constrain a voice agent’s actions and hand off to a human in The Control Plane Between What a Voice Agent Hears and What It Does, using an open source reference stack that checks every action the model proposes against policy before it runs.
Scale, latency, trust, all so far framed around media and AI. But the conference kept widening the aperture on what “real-time” even means.
Real-time from the network fabric to the sensor bus
The last set of talks reminded me that real-time constraints reach far beyond WebRTC.
Greg Agami (Origami Networks) looked at the network fabric itself in a talk on autonomous network capacity management. As AI-driven traffic grows and threatens to outpace available spectrum, the answer is squeezing more efficiency out of what exists using AI-powered rApps. What made it click was the reminder that these problems span timescales, from monthly forecasting down to real-time slot scheduling, and that autonomy has to be earned in stages, from visibility and recommendations up to closed-loop control.
Next Generation 911 got a fascinating treatment, both in a panel on emergency communication over the internet and in focused talks. The vision is an all-IP overhaul: text, photos, and video flowing to dispatchers, better location accuracy, and telemetry from connected devices. The engineering is real, but the most memorable takeaway was that the hardest problems are political and cultural, not technical. Independent 911 centers rightly ask what new data actually helps them do their job, and funding to adopt it is uneven.
Finally, a sharp talk by Sierra Quintana from Redwire on serial communication protocols in embedded sensor systems brought a comparative analysis of SPI, I²C, and CAN Bus, a clean reminder that real-time deadlines exist all the way down at the wire between a microcontroller and its sensors, each protocol trading off speed, wiring, and robustness for its niche.
Wrapping up
The through-line of this year’s IIT RTC Conference was that real-time systems are being pushed on three fronts at once. Scale is why I took the TURN relay out of the hot path with eBPF. Latency is why voice AI teams are accounting for every millisecond of that ~800ms gap. Trust is why security now has to run in parallel with the media path instead of behind it. And the breadth of the room, from spectrum to sensor buses, showed just how far the real-time discipline now reaches.
At WebRTC.ventures, this is our home turf: designing and building real-time media applications that hold up under real load. If your TURN tier is straining, or you’re building voice AI and fighting the latency budget, Let’s Make it Live!.
