SIP and RTP feed a call and its media into a voice AI pipeline. What happens next runs through the control plane: what the voice agent is told, and what it decides to do about it. In my talk at ClueCon this year, From SIP to Tokens:
Human speech is often unstructured and messy. To keep up with real conversation, Voice AI bots must parse partial sentences, filler words, and interruptions, all without the visual cues a human listener relies on. On top of that linguistic challenge, real-world latency budgets and network conditions test
In the space of thirteen months, Google took real-time speech translation from an English-and-Spanish demo at I/O 2025 to Gemini 3.5 Live Translate, a 70-language streaming model announced in June 2026 and now expanding into Google Meet through private preview, on top of the five-language version already
Voice AI tooling broadly falls into two categories: managed platforms that get you a production agent fast, and custom-built frameworks that give you full ownership of your voice pipeline. Within each category, the specific tools differ in ways that are worth understanding before you commit engineering time
