Many WebRTC applications struggle with outdated or inappropriate media server infrastructure, limiting their ability to scale effectively and support powerful AI features.
Alfred Gonzalez, Senior WebRTC Engineer at WebRTC.ventures, walks us through the considerations, options, and steps to successfully migrate to another media server. He’ll then show how this rearchitecting opens the door to implement AI-powered voice assistants using STT, TTS, and LLM services to intelligently handle inbound customer calls.
Alfred will also demonstrate how telephony agents can power outbound campaigns using the example of Conectara, a solution that combines Amazon Connect, AWS Lambda, and Amazon Bedrock to automate outreach campaigns such as proactive notifications, surveys, and data checks. Whether managing incoming calls or initiating outbound campaigns, modern voice AI agents are transforming how we interact with telephony networks.
The episode also features Arin Sime and Tsahi Level-Levi’s Monthly WebRTC Industry Chat. This month’s topic: Cloudflare is Making Big Waves in the WebRTC Space. Watch just this segment on YouTube.
Key insights and episode highlights below.
Watch Episode 106!
Key Insights
⚡ Rearchitecting legacy WebRTC systems is inevitable for scalability. When your media infrastructure struggles under load and your systems have multiple points of failure, it’s a strong signal that it’s time to modernize. Alfred explains the impact of migration, “So why migrate? In our case, we had some issues, performance issues in meetings, especially in webinar meetings. We were able to handle up to 200+ participants and multiple hosts. But at that point, we would suffer some audio and video quality issues. And also, in big conference meetings, we could see those issues happening as well with 20+ participants. This also translated to high server resource usage in those cases. We also had an outdated IBR system in our call center solution, and it had multiple services, which means multiple points of failure.”
⚡ Modernizing opens the door to AI-powered innovation. Migrating to a more efficient media server not only improves performance but also creates the foundation for future AI integrations. As Alfred explains, “Since we are using four times less CPU, which was our limit, we can scale up to four times more users. And another really important thing that we benefited from is that now we are open to other AI implementations. We are only using it now for the call center solution and for the transcription and translation, but we are open to implementing a call assistant or have summaries or analysis, or many more options.”
⚡ Technical challenges still exist. Latency, model performance, and the consistency of structured outputs can all impact the effectiveness of your system. Alfred explains how his team tackled these challenges: “We faced multiple challenges, one of them is latency. We found a few things that helped us, which is to run smaller models, it would give faster results, but you have to try to find the balance between speed and performance because if you have really big context, the smaller models get confused, and they do not work as expected.”
Episode Highlights
The tangible benefits of modernizing your media infrastructure
A strategic migration doesn’t just fix performance issues. It unlocks advanced features, smarter AI integrations, and far more efficient resource usage. Alfred explains the impact of their upgrade: “The results of this migration, we fixed the webinar meetings’ performance issues that we were experiencing. The audio and video performance was really good, and it also reduced a lot of the server resource usage. I think it was around four times less usage for the CPU, which is quite a lot, and of course, we also modernized our call center solution, we are now using modern AI with modern LLMs, which allows you to have more natural language understanding, and with the text-to-speech services, so also natural human-like voices.”
The hidden signs it’s time to upgrade your media server
As WebRTC applications evolve, it’s easy to overlook the silent signs that your infrastructure is falling behind. But overlooking them can slow down performance, limit scalability, and block future integrations. So, how do you know when your current setup is holding you back?
Alfred explains, “I would start with the two main issues that we experience. If you are experiencing audio and video issues in your calls that are not related to network connectivity, that is inherent to the media server you are using; that’s a good sign. And also, if you are planning to integrate AI solutions into your system and your media server or your WebRTC application doesn’t allow you to use all the really new cool tools that we have, STT, TTS, and LLM, you don’t have that ability, and you know that you need it, or you would benefit from it. I would say it’s a sign that you need to think about doing this kind of migration.”
Voice AI agents are transforming telephony and contact centers.
These agents are redefining how businesses handle both inbound and outbound communications, enabling smarter interactions and automated workflows. Alfred explains, “We just discussed the power of voice agents in call center solutions with this last example with the livekit agents. But we also have other ways to use voice agents to power your telephony solution, which in this case is for outreach campaign automation. And I will share this example that we built at WebRTC.ventures, which is Conectara. And as I said, it does outreach campaign automation, and it uses voice sessions with modern LLMs, and it’s powered with multiple AWS services, like Amazon Connect, Amazon Lex, AWS Lambda, Amazon Bedrock, DynamoDB, Amazon SQS, and a few more also.”
Up Next! WebRTC Live #107
MOQ vs. WebRTC: A Panel Discussion with Cloudflare
Renan Dincer, Systems Engineer
Mike English, Principal Engineer
Nils Ohlmeier, Principal Systems Engineer
Wednesday, November 19 at 12:30 pm Eastern
