Build WebRTC Media Muscle

Watch for the swerve

Real-time WebRTC communication has become the controlling force in communications technology and yet, somewhat ironically, there are still many WebRTC variables beyond our control. 

While WebRTC’s availability across all major browsers and on native clients for all major platforms is undoubtedly a good thing, the result is a massive disparity in media quality that hinges on issues of frame size, frame-rate, codecs, bitrate, latency, and transport protocols. 

In this blog we will learn how to watch for that swerve and build WebRTC media muscle by controlling the complaint and employing our best wrestling moves on the shifty. Let’s get to it.

The tag-team

As we touched upon in the introduction, WebRTC media quality variables are a disparate tag-team composed of both the submissive and the slippery.  The latter group causes the most difficulties because it is notoriously tricky to predict how and when it might affect media output quality. 

Let’s wrangle with the more serpentine elements first and determine what measures we can employ to manage the unmanageable.

The Slippery

Bandwidth

In a nutshell, bandwidth is the maximum level of data transfer over an internet connection in a given time period. Bandwidth juggles a lot of balls in the air, any number of which, if dropped will bring the whole act down. Here are some of the issues and possible remedies at our disposal:

Issue	Possible fix
Users’ distance from access points	Position servers closer to users, or add more servers.
Users’ internet quality.	Encourage users to turn off their automatic update processes
Quality of users’ hardware
How many people will use the same access points simultaneously
Users’ firewalls configuration and intentional or unintentional throttling of bandwidth.	Estimate bandwidth using WebRTC’s bandwidth estimation tools.
The number of simultaneous background uploads and downloads.	Inform users about router QoS settings that prioritize device bandwidth for more demanding applications

Transport protocol 

Transport protocols are the mechanisms by which the internet carries information in the right order to the right computers. 

UDP and  TCP transport protocols are the most common means of data transfer across networks. For our money, prioritizing TCP (TURN or ICE) connections over everything else can have a negative impact on transmission and media quality. TCP delivery and retransmission guarantees have no relevance for data that requires real-time transmission.  If packet loss occurs, TCP breaks and your media quality is affected. Stick with UDP and decrease overall latency. 

Now, onto the more compliant elements in WebRTC media quality.

The submissive

Bitrate

Bandwidth, as we’ve just discussed, determines how much data the network can send or receive, and is beyond our control. On the other hand, bitrate, or what we actually send or receive, is something we can control. 

We can achieve optimal bitrate by carefully estimating available bandwidth, and using as much of this as we can.

Maintain a bitrate that’s lower than or equal to your estimate.

Latency

Often referred to as lag, latency is the time it takes to capture, transmit and process data through the necessary devices and channels.

Real-time WebRTC communications is always a trade-off between latency and quality, however, by placing media servers closer to our users, we can tilt the latency balance in our favor. Rather than using a single server, distributing media servers over shorter distances in larger group calls reduces latency and increases media quality. 

So, while we cannot control where our users are, we can control where we are.

Use RRT  regularly to measure latency along with bandwidth, packet loss, and jitter. 

Codecs

Codecs are the devices or applications that compress and decompress media files for transmission across devices and networks. Codecs have a huge impact on media quality, and can become a massive drain on your time and resources if you fall down the rabbithole. 

WebRTC is codec agnostic, but it is always wise to spend time analyzing the available options.

Codecs	Consideration
VP8  H.264	VP8 has temporal scalability H.264 has more hardware acceleration  Both work across all browsers
VP9	Can provide higher quality than VP8 or H.264  Not as widespread as either VP8 or H.264  Its SVC may require  some reverse engineering
AV1	Has the potential be the best codec for media quality Still new and not well understood Heavy on CPU
HEVC	Apple exclusive.  Group sessions may require simulcast or SVC – this may not be available with HEVC

CPU

You don’t want your media muscle to overwhelm your users’ CPU, which could lead to issues of overheating and dropped or skipped frame-rates. The end result would mean poor media quality, lower battery life, and network congestion. Not a good look. 

Regular monitoring of CPU, adjusting for bitrate spikes will help you to sidestep these issues. Bear the following in mind:

Issue	Consideration
Code	Don’t run logic that switches up UI elements too often
Bitrates	Simulcasts send different quality media to different users in a session reducing overall bitrate
Lag	Mute participants who are not contributing – take care to do this in a manner that doesn’t affect the overall satisfaction of the users.

Media type and placement

Bandwidth estimators tell us the expected range within which we can safely transmit our media. It is up to us, however, to set out the media transmission parameters. The key questions to ask here are:

Questions

What resolution should the media display at?

What frame-rate is needed to reduce motion? Is sharpness or motion more important?

What resolution is the user’s screen?

Which content elements are more or less important? (This will determine the bitrate investment)

The WebRTC triforce

Now that we have gotten to know the distinct influences of bitrate, frame-rate, and resolution on WebRTC media quality, it is important to understand their collective power over WebRTC media output.

Bitrate is the nucleus of WebRTC media quality.  Changes to bitrate directly affect both frame-rate and resolution. Know your bitrate limitations and determine your available frame-rate and resolution options from there. 

The following guidelines will help when tweaking frame-rates and resolutions:

Content type	Considerations
Static	Choose a higher resolution at a lower frame-rate.  Opt for VBR encoding where possible to optimize transmission.
Dynamic	Choose a higher frame-rate – 30fps. When the bitrate is low, lower the frame-rate accordingly.
Sharing content from Youtube/Streaming platforms	Frame-rate is always more important than resolution.
For optimal video/image sharpness	Bump up resolution; don’t worry so much about the frame-rate.
Large group calls	Drop the frame-rate to 15fps or less

Check that you are not receiving video at resolutions higher than those you are displaying.

WebRTC is tough but, armed with the information in this blog, you can prove that you have the WebRTC media brawn to meet the challenge. 

If you are looking for some support around your WebRTC media challenges, why not call in the pros? Our WebRTC experts can help you to set up your testing, monitoring, and benchmarking infrastructure so that you can concentrate on what you do best. Our best in-class automation and analysis solutions will help to ensure that your users have the very best experience with your application.