Overview
Motivation: Alex worked for different media companies for the past 10 years. Some of the issues they always run into when enabling advanced features: new codecs, HDR, etc.
For product planning: nobody goes and test for features they don’t support at the moment. Try to estimate ahead of time.
Closer we are: HTML media capabilities API. Fairly recent, not widely supported, inconsistent behavior.
Figure out:
Doesn’t seem that complicated on paper:
Wonder: as an industry, can we come up with a cross-platform universal standard on how to detect media capabilities? Is there some progress that can be made so it’s more consistent: what API you call, what response you get.
If you look at media capabilities API: describes syntax, but not semantics. Eg: HLG, Dynamic HDR metadata, is it supported by the decoder? Is there a screen supporting it?
Supplemental codecs in the MIME type. Eg: codec HEVC, supplemental codec is some Dolby specific attribute.
Depends on how the feature intersects: Apple TV 4k, doesn’t support HDR & 60fps at the same time, but it supports them separately.
The way that chrome has done it: based on historical data from previous playouts.
It’s fair to separate the question of performance from capability: Can I play this vs. How well will it perform?
HDR content or Multi-channel audio: you have content available in SDR / HDR, ATMOS & stereo audio. A lot of the players are not good at figuring out what they can play vs. what they should play.
Risk: fingerprinting problem.
Proposed solution: Library (Alex Z)
Main Stakeholders:
In terms of spec (specially on the browser), for dynamic environments it makes sense to have a listener service. It changes if you plug / unplug a device and have HDR or HDCP support. Key status change.
The idea behind media capabilities is that it’s invariant!
CastLabs? implementation: now they have a lot of IFs? On Android, there are some weird chipsets that you need to do some workarounds. For instance text matching to identify hardware vs. software decoders. There’s no competitive advantage between players.
Is this w3c? Wave?
Does it make sense to define something like “profile / level” from codecs for capabilities?
Is there a point where you stop using this capabilities API?
Some challenges:
Where to have this discussion:
Detection of peripherals connected?=
Aggregation for information (Christian):
CTA-Wave is doing something similar for DASH.
The downside about doing it on a database is that media capabilities change, even depending on what the device is connected to (e.g. DD+ passthrough)
Thasso: As a community we should have a caniplay.tv
Next Steps / Action Items: