WebCodecs and Offscreen Renderer

Vindral Web SDK supports low-level browser video decoding via the WebCodecs API and, when supported, off-main-thread rendering via an offscreen renderer backed by OffscreenCanvas. Together these can unlock a lower-latency, lower-overhead playback pipeline compared to the traditional MSE path, where drift management is harder to achieve. Whether hardware or software decoding is used is configurable — see Hardware acceleration and decoder latency.

The WebCodecs decoder is opt-in and can be enabled by setting the decoders option to include webcodecs. When the resolved decoder is webcodecs, the SDK will also automatically prefer OffscreenCanvas rendering unless offscreenCanvasEnabled is explicitly set to false.

Decoder pipeline

The SDK selects a decoder based on the decoders option, which is an ordered priority list:

import { Vindral } from "@vindral/web-sdk"

const instance = new Vindral({
  url: "https://lb.cdn.vindral.com",
  channelId: "...",
  decoders: ["webcodecs", "mse", "wasm"], // default is ["mse", "wasm"]
})

Decoder	Description
webcodecs	Uses the browser's WebCodecs API for low-level decoding with configurable hardware or software acceleration. Lowest overhead and best latency control where supported.
mse	Uses Media Source Extensions — the standard browser media pipeline. Widest compatibility.
wasm	Software decoder compiled to WebAssembly. Used as a fallback when neither WebCodecs nor MSE is available.

The SDK will try each decoder in order and use the first one supported by the browser. Specifying ["webcodecs", "mse", "wasm"] means WebCodecs is preferred, with automatic fallback to MSE and then WASM.

If DRM is active, the SDK will use mse regardless of decoder order.

Rendering path and media-element bridge

The WebCodecs/offscreen path is controlled by offscreenCanvasEnabled, while streamToMediaElementEnabled controls whether canvas-rendered output is bridged back into a <video> element via captureStream(). These options can be used together.

Option names across integration surfaces

Core Web SDK	vindral-player attribute	Embed / QoS query string	QoS label
decoders	decoders	core.decoders	Decoder preference
offscreenCanvasEnabled	offscreen-canvas-enabled	core.offscreenCanvasEnabled	Enable OffscreenCanvas
streamToMediaElementEnabled	stream-to-media-element-enabled	core.streamToMediaElementEnabled	Stream to media element
webcodecsHardwareAcceleration	webcodecs-hardware-acceleration	core.webcodecsHardwareAcceleration	WebCodecs hardware acceleration

OffscreenCanvas rendering (offscreenCanvasEnabled)

When WebCodecs is active, offscreenCanvasEnabled is automatically enabled by default. Keep it enabled to render video frames in a Web Worker via OffscreenCanvas. Set it to false if you want WebCodecs decoding with main-thread canvas rendering instead.

const instance = new Vindral({
  url: "https://lb.cdn.vindral.com",
  channelId: "...",
  decoders: ["webcodecs", "mse", "wasm"],
  offscreenCanvasEnabled: true, // optional: enabled automatically when WebCodecs is used
})

Benefits:

• Lower latency — Decoded frames are rendered immediately to the canvas without going through the MSE media pipeline.
• Off-main-thread rendering — When OffscreenCanvas is available, video compositing happens in a Web Worker, freeing the main thread for UI responsiveness.
• Lower overhead — Reduces main-thread work, especially at high frame rates or when displaying multiple streams.
• Better for custom rendering — Canvas-based output integrates naturally with WebGL, 2D compositing, or any custom visual pipeline.

info

If OffscreenCanvas is unavailable, unsupported, or cannot be transferred to the worker, the SDK falls back to main-thread canvas rendering automatically.

Media element bridge (streamToMediaElementEnabled)

Set streamToMediaElementEnabled: true to route canvas-rendered output through a standard <video> element via captureStream(). This keeps media-element features such as fullscreen and Picture-in-Picture available for non-MSE playback. In the Web SDK this option is disabled by default.

const instance = new Vindral({
  url: "https://lb.cdn.vindral.com",
  channelId: "...",
  decoders: ["webcodecs", "mse", "wasm"],
  streamToMediaElementEnabled: true, // default is false in the Web SDK
})

This can be combined with WebCodecs and OffscreenCanvas. In that configuration, frames are still decoded and rendered through the canvas path, but the canvas output is also streamed into a <video> element for platform features.

info

streamToMediaElementEnabled is separate from DRM support. If DRM is active, the SDK uses the mse decoder path regardless of the decoders order.

Choosing the right mode

With the WebCodecs decoder and OffscreenCanvas renderer, Vindral can achieve real-time playback with sub-100ms latency and minimal drift, even on high frame rate streams, as there is no built-in browser buffering as with MSE.

Path	Latency profile	Media-element features	DRM	Good fit
mse	Highest of the three, but widest compatibility	Full browser media-element behavior	Yes	Consumer playback, DRM, broadest compatibility
webcodecs + offscreenCanvasEnabled: true + streamToMediaElementEnabled: true	Lower than MSE	Fullscreen/PiP for non-MSE playback via media-element bridge	No, DRM forces MSE	Low-latency playback that still needs media-element fullscreen/PiP
webcodecs + offscreenCanvasEnabled: true + streamToMediaElementEnabled: false	Lowest latency and least main-thread work	No media-element bridge; no PiP/AirPlay/background-play behavior	No, DRM forces MSE	Operator tools, monitoring, multiview, custom canvas/WebGL experiences

Goal	Recommended options
Lowest possible latency and least main-thread work	decoders: ["webcodecs", ...] + leave offscreenCanvasEnabled automatic (or set it to true) + streamToMediaElementEnabled: false
Lowest latency while retaining fullscreen/PiP for non-MSE playback	decoders: ["webcodecs", ...] + offscreenCanvasEnabled: true + streamToMediaElementEnabled: true
DRM / encrypted playback	Include mse in decoders; DRM will use the MSE path regardless of decoder order

Example use cases

The combination decoders: ["webcodecs", ...] + offscreenCanvasEnabled: true + streamToMediaElementEnabled: false is a good fit when the stream is primarily viewed in an active foreground tab and the integration does not depend on browser media-element features such as background playback, Picture-in-Picture, AirPlay, or DRM.

• Live casino and other real-time gaming surfaces — Lower latency and lower overhead are typically more important than background-tab playback, and these integrations often use custom overlays and UI around the video.
• Remote production and operator monitoring — Control-room tools benefit from immediate frame delivery and low drift, while browser media features are usually not required.
• Security and surveillance dashboards — Low delay and efficient rendering across one or many streams are often more valuable than consumer playback features.
• Multiview, kiosk, and monitor-wall deployments — Dedicated foreground displays are a strong match for the pure canvas path, especially when many concurrent streams need to stay responsive.
• Custom-composited experiences — If your application blends video with WebGL, canvas overlays, telemetry, or interactive graphics, keeping the stream on the canvas path usually gives you the cleanest integration.

When not to use this path

The pure canvas path is usually the wrong choice if your integration depends on browser media-element behavior more than raw latency:

• DRM / encrypted playback — DRM uses the mse path regardless of decoder order
• Picture-in-Picture, AirPlay, or similar media-element features — Keep streamToMediaElementEnabled: true or stay on mse
• Background-tab or background-audio playback requirements — A standard media element is the safer fit
• Compatibility-first consumer playback — mse remains the broadest and most predictable path across browsers
• Integrations that rely on native media-element track handling — Browser-native subtitles, media controls, and related APIs are a better match for the media-element path

Hardware acceleration and decoder latency

By default ("no-preference"), the browser decides whether to use hardware or software decoding. While hardware decoding reduces CPU load, some GPUs and drivers introduce internal buffering in their decode pipeline — meaning a hardware decoder may actually deliver frames with more latency than a software decoder on the same machine.

The webcodecsHardwareAcceleration option maps directly to the hardwareAcceleration hint in the WebCodecs API and lets you control this trade-off:

const instance = new Vindral({
  url: "https://lb.cdn.vindral.com",
  channelId: "...",
  decoders: ["webcodecs", "mse", "wasm"],
  webcodecsHardwareAcceleration: "prefer-software", // "prefer-hardware" | "prefer-software" | "no-preference"
})

Value	Behavior
"no-preference"	Default. The browser chooses whether to use hardware or software decoding.
"prefer-hardware"	Hints to the browser to use hardware decoding. Best for reducing CPU usage.
"prefer-software"	Hints to the browser to use software decoding. Can reduce frame latency on devices where the hardware decoder introduces buffering.

tip

If you are seeing unexpectedly high end-to-end latency with the WebCodecs decoder enabled, try setting webcodecsHardwareAcceleration: "prefer-software". On some hardware/driver combinations this will reduce the latency at the cost of slightly higher CPU usage.

Note that Vindral will fall back to software decoding if hardware acceleration is unavailable or fails, so setting no-preference or prefer-hardware is always safe.

Browser support

Support for WebCodecs varies by browser and version. Before depending on it for a broad deployment, check current compatibility for WebCodecs / VideoDecoder and transferControlToOffscreen(). The SDK automatically falls back to the next decoder in the decoders list if WebCodecs is not available.

Worker-side offscreen rendering requires both WebCodecs and browser support for transferring a canvas to a worker. If those capabilities are unavailable, or if transferring the canvas fails, Vindral falls back to main-thread canvas rendering.

Troubleshooting

Why did WebCodecs or OffscreenCanvas not activate?

Check these first:

• Make sure decoders includes webcodecs, and put it first if you want it preferred over mse
• If DRM is active, the SDK will use the mse path regardless of decoder order
• If the browser does not expose VideoDecoder, the SDK will fall back to the next decoder in the list
• If OffscreenCanvas cannot be transferred to the worker, Vindral will keep WebCodecs decoding but fall back to main-thread canvas rendering
• If you need Picture-in-Picture or media-element fullscreen behavior on a canvas-rendered path, enable streamToMediaElementEnabled: true

How to verify the active path

There are two easy ways to confirm which playback path is actually in use after browser capability checks and fallback logic:

• QoS client — In Decode Performance, look at Video Decoder and Video Renderer. Typical values are WebCodecs or WASM for the decoder, and OffscreenCanvas, Canvas, or MSE for the renderer.
• SDK logging — Set a visible logLevel and watch for initialization messages such as Initialized WebCodecs video decoder, Initialized WASM video decoder, and OffscreenCanvas rendering enabled.

This is especially useful when the browser falls back automatically because WebCodecs, OffscreenCanvas transfer, or DRM constraints prevent the preferred path from being used.

Testing with the QoS client

You can verify this configuration using the Vindral QoS client. The URL parameters map directly to SDK options:

• core.decoders=webcodecs,mse,wasm — sets decoder priority to WebCodecs, then MSE, then WASM
• core.offscreenCanvasEnabled=true — enables worker-side OffscreenCanvas rendering when supported
• core.streamToMediaElementEnabled=false — keeps the output on the canvas instead of bridging it into a <video> element
• core.webcodecsHardwareAcceleration=prefer-software — switches to software decoding for latency troubleshooting

See the QoS client documentation for more details on how query parameters work.