February 24, 2022
Comments Off on Real-Time High Performance Rendering with the Offscreen Canvas API

Offscreen Canvas is an up and coming API that allows a developer to offload canvas rendering from the main JS thread in the browser. This allows a developer to provide high resolution or graphically intensive applications directly in the browser without affecting the user’s experience. Offscreen Canvas is still a little bit experimental, and is just now becoming standard in the major browsers like Chrome, Firefox and Edge, making this the perfect time to introduce it into your WebRTC application.

What is the Offscreen Canvas API?

  • Allows a developer to offload canvas rendering from the main JS thread in the browser
  • Significant performance boost to rendering heavy applications such as games or multimedia mixing, 3-D rendering
  • Canvas frames are transferred from a worker thread over shared memory as a bitmap image
Event Model for Web Workers
Worker Shared Memory Buffer

Lots of documentation on how web workers work can be found on the Mozilla developer network site.

How early is this?

  • Experimental; mileage may vary per browser
  • Flags may be necessary for the time being
Offscreen Canvas Browser Compatibility as of February 2022

Is it worth it to refactor my realtime AV application to use offscreen canvas?

Yes! Checkout the github demo repo for Jacob’s benchmark demo. This demo should give you an idea of how well the offscreen canvas performs at high resolutions (like 4k) performing CPU and GPU intensive processes.

Implementing Offscreen Canvas API

Asynchronous Worker Implementation

main.js: 

var htmlCanvas = document.getElementById("canvas");
var offscreen = htmlCanvas.transferControlToOffscreen();

var worker = new Worker("offscreencanvas.js");
worker.postMessage({canvas: offscreen}, [offscreen]);

offscreen.js (worker code):

onmessage = function(evt) {
  var canvas = evt.data.canvas;
  var gl = canvas.getContext("webgl");

  // ... some drawing using the gl context ...
};

 

Synchronous Bitmap Transfer Implementation

main.js:

var one = document.getElementById("one").getContext("bitmaprenderer");
var two = document.getElementById("two").getContext("bitmaprenderer");

var offscreen = new OffscreenCanvas(256, 256);
var gl = offscreen.getContext('webgl');

// ... some drawing for the first canvas using the gl context ...

// Commit rendering to the first canvas
var bitmapOne = offscreen.transferToImageBitmap();
one.transferFromImageBitmap(bitmapOne);

// ... some more drawing for the second canvas using the gl context ...

// Commit rendering to the second canvas
var bitmapTwo = offscreen.transferToImageBitmap();
two.transferFromImageBitmap(bitmapTwo);

 

See the MDN OffscreenCanvas docs for more details on these two different implementations.

Disadvantages to using Offscreen Canvas

  • Have to refactor to existing canvas projects which can be more involved that you think
  • Workers can’t interface with the DOM
  • Experimental and difficult to oklement in webpacks/SPA apps

However, it is well worth it!! If it is a good fit for your application, the benefits definitely outweigh the disadvantages. 

For more on the subject (and a demo!):

These types of innovative solutions are the reason people come to our expert team at WebRTC.ventures when they are interested in integrating (or optimizing!) live video or audio into their applications. Learn more about our WebRTC development services and tell us how we can help your business! 

Recent Blog Posts