MSE
MSE
- What’s changed in iOS MSE over the past few years?
* Back in the days, you could only do full screen video on iPhones
* And then you could do inline video, and there was much rejoicing
* But the video devs not happy, they wanted MSE
* MSE had drawbacks which made it unsuitable for mobile
* It’s memory inefficient
* It doesn’t give device vendors the flexibility to purge buffers, for example, when switching apps
* So Apple did not want to ship MSE on iPhones
* Managed MSE gives browsers the license to purge buffers
* But they’ll tell you about it
* It’s seen a lot of uptake, mostly from YouTube?
* Mostly on iOS over Mac
* Using MMS on iPhone allows you to load media over the 5G modem
* Super fast, but super power hungry
* The UA will gives a heads up when you can load while the modem is active
* Simple heuristic, only fires when MMS is turning on the modem, not when it’s turned on by another app
- Can we make decode errors more useful?
* Client engs spend a lot of time investigating decode errors * Fingerprinting concerns, but we can add things to devtools * Submitting an Apple Radar submits 12hrs of console logs, so turn on debug media logging! * The browser doesn’t always know more than us
- Switching from encrypted to clear content
* Bug in Safari and Chrome * Unclear what’s going wrong * John to send Ted the Chromium bug
- Detaching MediaSource? from MediaElement? without clearing SourceBuffers?
* Apple tried it & added it to the HLS spec (for interstitials) * At creation time, you can add a flag to make it detachable * Have to remember to seek back to where you were when reattaching
- MSE 2.0
* SourceBuffer? purge rules * Segment metadata API * Would be a method on the SourceBuffer? object * Gap skip behavior * All browsers do this differently, should be controllable/standardized * & more
- MSE in Workers for Safari
* It’s a thing!
- <video> in a worker
* Not super important for MSE in Workers, since MSE doesn’t need a ton of access to the video element * But it would be nice * More helpful for requstVideoFrameCallback * What if we could just attach the video element to a GPU texture?