A Media Control Unit or MCU is the most established and time tested approach to setting up conferences via bridges. Conference bridges add centralized call and media features like mixing and quality control among other advanced controls such as a DID number, announcement, admin control, secure PIN-based access. Due to centralized media management, they are also ideal for connecting mixed streams with media pipelines for recording, broadcasting or plugging into machine learning models for transcription for NLP or sentiment analysis. This post will cover using FreeSWITCH as a WebRTC Multipart conferencing server using a video mixer and conference bridge.
This is heavily reliant on a media server with a mixer for audio and/or video streams. Meeting rooms often involve a bridge with dozens of participants that multiplex their incoming audio and video streams into one output and input stream per participant.
Sources of energy consumption
Sources of energy consumption in a MCU based conference:
- Transmission: 2 * transmission_energy (kWh) * n
- Device (media): 2 * device_energy (kWh) * n
- Signaling Server: signaling_energy (kWh) * n
- Media Server: mediaserver_energy (kWh) * n
Relatively higher energy consumption in Mesh based (peer to peer) conference:
- Transmission: transmission_energy (kWh) * n * (n-1)
- Device: device_energy (kWh) * n * (n-1)
- Signaling Server: signaling_energy (kWh) * n
- Media Server: 0
In this fashion of a centralized setup, the CPU consumption is highest at the centre at the media server. This is due to MCU’s real-time media processing that may include decoding, mixing and re-encoding every stream and mixing as one. It should be noted that CPU consumption could be higher where a centralized server does extensive transcoding between peers supporting different codecs.
On the other hand, the CPU utilization is lowest at receivers who only need to decode one single stream incoming stream from the media server. On the side, extra latency caused by media server decoding necessitates the use of a jitter buffer.
Installation and Setup of FreeSWITCH for Video MCU
FreeSWITCH™ is an open source carrier-grade telephony platform implemented as a back-to-back user agent.
FreeSWITCH is a SIP standard specific communication platform that forms the core of many cloud telephony and communication services. The pluggable modules make FreeSWITCH suited to almost any role in a SIP platform (SBC, gateway, SIP application server, media server, etc.). Let’s set up FreeSWITCH as a WebRTC MCU using a video mixer and conference bridge.
More configuration files related to this setup can be located at https://github.com/agilityfeat/Energyefficient_WebRTC_Asterisk_systems/tree/master/Freeswitch_MCU_Webrtc
Configuration files for freeswitch. Below is a sippet of freeswitch.xml
The vars.xml is the most important file ( according to me ) for starting freeswitch as it contains the Preprocessor Variables. These values should be consistent across the modules
The internal.xml manages the configuration for internal clients such as SIP endpoints registered with the SIP server.
<profile name="internal">
<settings>
<param name="auth-calls" value="true"/>
<param name="apply-nat-acl" value="nat.auto"/>
<param name="debug" value="0"/>
<param name="sip-trace" value="yes"/>
<param name="dialplan" value="XML"/>
<param name="context" value="default"/>
<param name="codec-prefs" value="$${global_codec_prefs}"/>
<param name="rtp-ip" value="$${local_ip_v4}"/>
<param name="sip-ip" value="$${local_ip_v4}"/>
<param name="ext-rtp-ip" value="auto-nat"/>
<param name="ext-sip-ip" value="auto-nat"/>
<param name="sip-port" value="$${internal_sip_port}"/>
</settings>
</profile>Conference Layout
The Conference.xml is critical to setup the bridges and allow for mixing. The Conference_layout.conf enables us to add various layouts for users. Once a conference begins this is the layout for incoming video that will be assembled and sent out to all participants.
<configuration name="conference_layouts.conf" description="Audio Conference">
<layout-settings>
<layouts>
<layout name="presenter-overlap-small-top-right">
<image x="0" y="0" scale="360" floor-only="true"/>
<image x="300" y="0" scale="60" overlap="true" reservation_id="presenter"/>
</layout>
<layout name="1up_top_left+5">
<image x="0" y="0" scale="240" floor="true" audio-position="-.5:0:0"/>
<image x="240" y="0" scale="120" audio-position=".25:0:0"/>
<image x="240" y="120" scale="120" audio-position=".25:0:-.25"/>
<image x="0" y="240" scale="120" audio-position="-.5:0:-1"/>
<image x="120" y="240" scale="120" audio-position="0:0:-1"/>
<image x="240" y="240" scale="120" audio-position=".25:0:-1"/>
</layout>
<layout name="1x1">
<image x="0" y="0" scale="360" floor="true"/>
</layout>
<layout name="2x2">
<image x="0" y="0" scale="180"/>
<image x="180" y="0" scale="180"/>
<image x="0" y="180" scale="180"/>
<image x="180" y="180" scale="180"/>
</layout>
</layouts>
<groups>
<group name="grid">
<layout>1x1</layout>
<layout>2x2</layout>
</group>
</groups>
</layout-settings>
<configuration>Users
Each extension contains a parameter “context” that matches a diaplan in conf/dialplan/.
When FreeSWITCH gets a register packet it looks for the user in the directory based on the domain in the packet depending on how sofia profile is configured. For this demo, I have set auth to false since I will only be using dial in extensions for users to join the bridge.
My user profile
<users>
<user id="altanai">
<params>
<param name="password" value="$${default_password}"/>
<param name="vm-password" value="1000"/>
</params>
<variables>
<variable name="toll_allow" value="domestic,international,local"/>
<variable name="accountcode" value="987"/>
<variable name="user_context" value="video-mcu-stereo"/>
<variable name="effective_caller_id_name" value="altanai"/>
<variable name="outbound_caller_id_name" value="altanai_outbound"/>
</variables>
</user>
</users>Dialplan
conf/dialplan/ contain XML files for dialplan
For this demo, I am using the video-mcu-stereo conf profile (shown below) which allows the communication service provider to add sounds for user activity as join, leave, mute, locked out, wrong pin and even other advanced features such as comfort noise, video canvas layout among others. There are many other samples for the video MCU configuration and one can always design their custom configuration.
<profile name="video-mcu-stereo">
<param name="domain" value="$${domain}"/>
<param name="rate" value="48000"/>
<param name="channels" value="2"/>
<param name="interval" value="20"/>
<param name="energy-level" value="200"/>
<!-- <param name="tts-engine" value="flite"/> -->
<!-- <param name="tts-voice" value="kal16"/> -->
<param name="muted-sound" value="conference/conf-muted.wav"/>
<param name="unmuted-sound" value="conference/conf-unmuted.wav"/>
<param name="alone-sound" value="conference/conf-alone.wav"/>
<param name="moh-sound" value="$${hold_music}"/>
<param name="enter-sound" value="tone_stream://%(200,0,500,600,700)"/>
<param name="exit-sound" value="tone_stream://%(500,0,300,200,100,50,25)"/>
<param name="kicked-sound" value="conference/conf-kicked.wav"/>
<param name="locked-sound" value="conference/conf-locked.wav"/>
<param name="is-locked-sound" value="conference/conf-is-locked.wav"/>
<param name="is-unlocked-sound" value="conference/conf-is-unlocked.wav"/>
<param name="pin-sound" value="conference/conf-pin.wav"/>
<param name="bad-pin-sound" value="conference/conf-bad-pin.wav"/>
<param name="caller-id-name" value="$${outbound_caller_name}"/>
<param name="caller-id-number" value="$${outbound_caller_id}"/>
<param name="comfort-noise" value="false"/>
<param name="conference-flags" value="livearray-json-status|json-events|video-floor-only|rfc-4579|livearray-sync|minimize-video-encoding|manage-inbound-video-bitrate|video-required-for-canvas|video-mute-exit-canvas|mute-detect"/>
<param name="video-auto-floor-msec" value="1000"/>
<param name="video-mode" value="mux"/>
<param name="video-layout-name" value="3x3"/>
<param name="video-layout-name" value="group:grid"/>
<param name="video-canvas-size" value="1920x1080"/>
<param name="video-canvas-bgcolor" value="#333333"/>
<param name="video-layout-bgcolor" value="#000000"/>
<param name="video-codec-bandwidth" value="3mb"/>
<param name="video-fps" value="30"/>
<!-- <param name="video-codec-config-profile-name" value="conference"/> -->
</profile>Security certs and WSS Binding
Add certs like /usr/local/freeswitch/certs/wss.pem shown below
Add wss-bind-url in internal.xml
<!-- for sip over websocket support -->
<param name="ws-binding" value=":5066"/>
<!-- for sip over secure websocket support -->
<!-- You need wss.pem in $${certs_dir} for wss or one will be created for you -->
<param name="wss-binding" value=":7443"/>Check that profile is loaded
sofia status profile internal
Give permission to wss page. For this example my ip and port made this url which I gave advanced permissions to.
WebRTC Clients
Using jssip for 2 clients.
Monitor Successful registration on sofia by checking registrations by profile with the command
sofia status profile <ip> reg
For Multiparty conference trials one can use Cyber Mega Phone 2K Ultimate Dynamic Edition browser side client application by Digium.
Up Next
The last blog post in this series will compare the SFU and MCU against Mesh using Chrome’s energy usage in ‘Analyzing compute and energy consumption of the most popular video conferencing topologies’ that indicates the carbon footprint of a conference call.
About the Author: Altanai Bisht
Here at WebRTC.ventures, we pride ourselves on offering our clients the opportunity to work with extremely bright developers, like Altanai. Don’t trust your custom video solution to just anyone. Trust the experts!
