MSE
Notes.MSE History
Hide minor edits - Show changes to output
Added lines 1-45:
MSE
* What’s changed in iOS MSE over the past few years?
* Back in the days, you could only do full screen video on iPhones
* And then you could do inline video, and there was much rejoicing
* But the video devs not happy, they wanted MSE
* MSE had drawbacks which made it unsuitable for mobile
* It’s memory inefficient
* It doesn’t give device vendors the flexibility to purge buffers, for example, when switching apps
* So Apple did not want to ship MSE on iPhones
* Managed MSE gives browsers the license to purge buffers
* But they’ll tell you about it
* It’s seen a lot of uptake, mostly from YouTube
* Mostly on iOS over Mac
* Using MMS on iPhone allows you to load media over the 5G modem
* Super fast, but super power hungry
* The UA will gives a heads up when you can load while the modem is active
* Simple heuristic, only fires when MMS is turning on the modem, not when it’s turned on by another app
* Can we make decode errors more useful?
* Client engs spend a lot of time investigating decode errors
* Fingerprinting concerns, but we can add things to devtools
* Submitting an Apple Radar submits 12hrs of console logs, so turn on debug media logging!
* The browser doesn’t always know more than us
* Switching from encrypted to clear content
* Bug in Safari and Chrome
* Unclear what’s going wrong
* John to send Ted the Chromium bug
* Detaching MediaSource from MediaElement without clearing SourceBuffers
* Apple tried it & added it to the HLS spec (for interstitials)
* At creation time, you can add a flag to make it detachable
* Have to remember to seek back to where you were when reattaching
* MSE 2.0
* SourceBuffer purge rules
* Segment metadata API
* Would be a method on the SourceBuffer object
* Gap skip behavior
* All browsers do this differently, should be controllable/standardized
* & more
* MSE in Workers for Safari
* It’s a thing!
* <video> in a worker
* Not super important for MSE in Workers, since MSE doesn’t need a ton of access to the video element
* But it would be nice
* More helpful for requstVideoFrameCallback
* What if we could just attach the video element to a GPU texture?
* What’s changed in iOS MSE over the past few years?
* Back in the days, you could only do full screen video on iPhones
* And then you could do inline video, and there was much rejoicing
* But the video devs not happy, they wanted MSE
* MSE had drawbacks which made it unsuitable for mobile
* It’s memory inefficient
* It doesn’t give device vendors the flexibility to purge buffers, for example, when switching apps
* So Apple did not want to ship MSE on iPhones
* Managed MSE gives browsers the license to purge buffers
* But they’ll tell you about it
* It’s seen a lot of uptake, mostly from YouTube
* Mostly on iOS over Mac
* Using MMS on iPhone allows you to load media over the 5G modem
* Super fast, but super power hungry
* The UA will gives a heads up when you can load while the modem is active
* Simple heuristic, only fires when MMS is turning on the modem, not when it’s turned on by another app
* Can we make decode errors more useful?
* Client engs spend a lot of time investigating decode errors
* Fingerprinting concerns, but we can add things to devtools
* Submitting an Apple Radar submits 12hrs of console logs, so turn on debug media logging!
* The browser doesn’t always know more than us
* Switching from encrypted to clear content
* Bug in Safari and Chrome
* Unclear what’s going wrong
* John to send Ted the Chromium bug
* Detaching MediaSource from MediaElement without clearing SourceBuffers
* Apple tried it & added it to the HLS spec (for interstitials)
* At creation time, you can add a flag to make it detachable
* Have to remember to seek back to where you were when reattaching
* MSE 2.0
* SourceBuffer purge rules
* Segment metadata API
* Would be a method on the SourceBuffer object
* Gap skip behavior
* All browsers do this differently, should be controllable/standardized
* & more
* MSE in Workers for Safari
* It’s a thing!
* <video> in a worker
* Not super important for MSE in Workers, since MSE doesn’t need a ton of access to the video element
* But it would be nice
* More helpful for requstVideoFrameCallback
* What if we could just attach the video element to a GPU texture?