MSE

MSE

  • What’s changed in iOS MSE over the past few years?
    * Back in the days, you could only do full screen video on iPhones
    * And then you could do inline video, and there was much rejoicing
    * But the video devs not happy, they wanted MSE
    * MSE had drawbacks which made it unsuitable for mobile
        * It’s memory inefficient
        * It doesn’t give device vendors the flexibility to purge buffers, for example, when switching apps
    * So Apple did not want to ship MSE on iPhones
    * Managed MSE gives browsers the license to purge buffers
        * But they’ll tell you about it
    * It’s seen a lot of uptake, mostly from YouTube?
        * Mostly on iOS over Mac
    * Using MMS on iPhone allows you to load media over the 5G modem
        * Super fast, but super power hungry
        * The UA will gives a heads up when you can load while the modem is active
            * Simple heuristic, only fires when MMS is turning on the modem, not when it’s turned on by another app
  • Can we make decode errors more useful?
    * Client engs spend a lot of time investigating decode errors
    * Fingerprinting concerns, but we can add things to devtools
    * Submitting an Apple Radar submits 12hrs of console logs, so turn on debug media logging!
    * The browser doesn’t always know more than us
  • Switching from encrypted to clear content
    * Bug in Safari and Chrome
    * Unclear what’s going wrong
    * John to send Ted the Chromium bug
    * Apple tried it & added it to the HLS spec (for interstitials)
    * At creation time, you can add a flag to make it detachable
    * Have to remember to seek back to where you were when reattaching
  • MSE 2.0
    * SourceBuffer? purge rules
    * Segment metadata API
        * Would be a method on the SourceBuffer? object
    * Gap skip behavior
        * All browsers do this differently, should be controllable/standardized
    * & more
  • MSE in Workers for Safari
    * It’s a thing!
  • <video> in a worker
    * Not super important for MSE in Workers, since MSE doesn’t need a ton of access to the video element
        * But it would be nice
    * More helpful for requstVideoFrameCallback
        * What if we could just attach the video element to a GPU texture?