Codecs are essential to WebRTC, enabling efficient media compression and decompression for transmission. They not only ensure seamless communication but also impact quality, latency, and compatibility across devices.

In this post we will talk about the importance of codecs for WebRTC, including essential codecs, codecs that enable additional features, and emerging codecs such as AV1 and AI audio codecs.

The Role of Codecs

The main goal of WebRTC is to enable real-time exchange of video, audio and arbitrary data between two or more peers. This exchange requires the sender to compress media data before sending it. This process is known as encoding.

The receiver obtains the compressed media, decompresses it, and then displays the video on the screen while playing the audio through the device’s speakers. This process is known as decoding.

In order for both encoding and decoding work correctly, they both need to follow the same set of rules. A codec dictates these rules. The word codec comes from the blend of the words coder and decoder.

There are many types of codecs available, each one with its own set of characteristics and applications. Given the real-time nature of WebRTC, we’re interested in the ones that allow us to reliably exchange media with minimal latency.

Mandatory Codecs in WebRTC

The WebRTC standard mandates that all implementations support a specific set of codecs. This requirement ensures seamless compatibility and interoperability between different devices and platforms during real-time communication sessions. Such mandatory codecs are:

Mandatory Video Codecs

• H.264: Also known as AVC (Advanced Video Coding), H.264 is a widely adopted video codec renowned for its high compression efficiency and broad compatibility across a vast array of devices and platforms. Its gained widespread adoption because of its ability to deliver high-quality video at relatively low bit rates, making it ideal for real-time communication scenarios.
• VP8: Developed by Google, VP8 is an open-source video codec that offers competitive compression efficiency and good video quality. While not as universally supported as H.264, VP8 has gained traction due to its open-source nature and royalty-free licensing model, making it an attractive option for developers and organizations seeking to avoid licensing costs.

Mandatory Audio Codecs

• Opus: Opus is a modern, open-source audio codec that excels in delivering high-quality audio at varying bit rates. Its versatility and ability to adapt to different network conditions make it well-suited for real-time communication applications. Opus supports a wide range of audio bandwidths and sampling rates, making it adaptable to diverse audio content and network environments.
• G.711: G.711 is an older audio codec commonly used in telephony systems. While not as efficient as Opus in terms of compression, G.711 is included in the WebRTC standard due to its widespread compatibility and legacy support in telecommunication infrastructure. Its inclusion ensures that WebRTC applications can interoperate with older telephony systems and devices that may still rely on it for audio encoding and/or decoding.

For Further Reading: “Optimizing WebRTC Performance on Slow Networks: Key Network-Level Considerations“

Optional Codecs

In addition to the mandatory codecs that are required for WebRTC compatibility, some vendors have adopted additional codecs that enhance capabilities and improve performance. These codecs deliver comparable quality audio and video at lower bitrates, but they are not mandated by the WebRTC standard. Consequently, support varies based on the specific implementation or browser that is used.

Optional Video Codecs

• VP9: This video codec provides advanced compression techniques that can significantly reduce file sizes without compromising visual quality. One particular benefit of it is the ability to perform Scalable Video Coding (SVC).

Optional Audio Codecs

• G.722: This wideband audio codec supports a broader frequency range than traditional telephone codecs, resulting in higher fidelity audio.
• Internet Low Bitrate Codec (iLBC): This narrowband audio codec is designed for efficient streaming of voice audio, making it ideal for applications with limited bandwidth.
• Internet Speech Audio Codec (iSAC): This wideband speech codec offers high-quality audio compression and is well-suited for real-time communication applications.

Striving for an Open (and Intelligent) Future

In an attempt to enable seamless audio-visual experiences through the development of royalty-free standards and software, multiple companies -including industry leaders like Google and Mozilla- formed the Alliance for Open Media. This collaborative effort produced the AV1 codec, which provides high-quality video compression with greater efficiency than previous codecs.

AV1 now powers multiple video conferencing platforms including Google Meet, Microsoft Teams and Facebook Messenger.

On the other hand, advancements in audio codec technology have leveraged the power of artificial intelligence. Newer codecs, such as Google’s Lyra, Microsoft’s Satin and Webex AI Codec, utilize AI algorithms to achieve higher compression rates while maintaining or even enhancing audio quality.

These advancements in audio and video compression not only provide optimal ways for exchanging media but also pave the way for innovative applications and use cases that rely on high-quality, real-time communication.

Wrapping Up

Codecs are essential components of WebRTC, enabling real-time communication by efficiently encoding and decoding media data. The mandatory codecs ensure compatibility, while optional codecs offer additional features and performance enhancements. As technology advances, we can expect to see even more efficient and intelligent codecs that further improve the quality and accessibility of real-time communication.

Need help with encoding and decoding media for your real-time communication application? WebRTC.ventures offers expert guidance and solutions tailored to your specific needs. Contact us today to learn how we can help you optimize your media processing and deliver a seamless real-time communication experience. Let’s make it live!