Main.MediaSource History
Hide minor edits - Show changes to output
November 17, 2013, at 11:12 PM
by - Minutes of the FOMS 2013 session on SME
Added lines 1-94:
!!! Media Source Extensions (Aaron)
!! 1. What Changed since last year's FOMS
Many features added to SME since last year:
* appendMode attribute to sourceMode object (to handle transport streams; ignore timestamps;
* to offload discontinuities handling to framework
* also good for advertising insertion.
* All append operations are a-synchronous now
* Append Window: support appending sections of chunks.
* Better TrackList management
* New API to remove segments (to out-smart the browser around garbage collection)
* Statistics functionality: videoPlaybackQuality object:
* corruptedVideoFrames
* creationTime
* droppedVideoFrames
* totalVideoFrames
* totalFrameDelay
At risk for removal: "appendStream" and "totalFrameDelay".
videoPlaybackQuality is on video element, so could be used for non-MSE playback as well. Still part of MSE specification though.
!! 2. Codec/container support in implementations:
* Chrome supports:
* WebM with VP8 and Vorbis
* ISOBFF with H264 and AAC/MP3.
* VP9 only on desktop, Opus coming soon?
* TS support only in Chromecast
* MP3 audio-only support behind a flag (skips ID3 tags and Shoutcast headers)
* Internet Explorer 11 supports ISOBFF with H264/AAC
* Only on Windows 8.1 (NOT Windows 7)
* Firefox working on implementations
* See bug #855130 on Bugzilla
* Apple looking into MSE support
* Some work done; desktop-only
* Further down the road.
* WebKit gStreamer MSE work in progress too
* All Webkit bugs tagged "mse"
!! 3. Is API compatible between browsers?
* MSE on Chrome actually behind the spec
* no "mode", no "totalframedelay"
* "appendStream" behind a flag
* no A/V trackList, because Blink doesn't support it yet.
* there's currently no place that lists what's supported
* maybe on Chromium.org?
* All layout tests in Blink are written against W3C spec.
* Some are currently WebM biased
* IE seems pretty complete and based upon latest spec
* YouTube plays MSE in IE11.
!! 4. How does memory management / garbage collection works?
* Not clear how this works on Webkit.
* Simple solutions don't always work well.
* How to do AES decryption on top of MSE?
* Using web workers to decouple.
* Should be possible on Chrome/Android
* FYI: updating bytestreams while appending not supported
* (similar issue was raised with web audio)
* Chrome actually makes copy instead of holding reference
* Neutering is problematic, since other Views could be blown away (needs further defined first)
!! 5. What about DASH?
* YouTube is doing it today.
* Including VP8/VP9
* DASH.js reference implementation on DASH evangelist site.
* Contains several example streams.
* Wowza servers are supporting DASH.
* DASH264 only
!! 6. What about text tracks?
* Currently reviewing VTT support in WebM
* Enabled in both SME and WebM progressive.
* No overlapping, just full cues
* Framework in Chrome for in-band text tracks in development
* Muxed or not
* Landing soon...
* would also open support for multiple A/V tracks.
* Spec for VTT and TTML in MP4 written
* Only supports full cues (no overlapping)
* Spec lives on Text Tracks working group wiki (by Cyril)
* What about in-band cues?
* Just VTT track with the "kind" set to metadata
!! 1. What Changed since last year's FOMS
Many features added to SME since last year:
* appendMode attribute to sourceMode object (to handle transport streams; ignore timestamps;
* to offload discontinuities handling to framework
* also good for advertising insertion.
* All append operations are a-synchronous now
* Append Window: support appending sections of chunks.
* Better TrackList management
* New API to remove segments (to out-smart the browser around garbage collection)
* Statistics functionality: videoPlaybackQuality object:
* corruptedVideoFrames
* creationTime
* droppedVideoFrames
* totalVideoFrames
* totalFrameDelay
At risk for removal: "appendStream" and "totalFrameDelay".
videoPlaybackQuality is on video element, so could be used for non-MSE playback as well. Still part of MSE specification though.
!! 2. Codec/container support in implementations:
* Chrome supports:
* WebM with VP8 and Vorbis
* ISOBFF with H264 and AAC/MP3.
* VP9 only on desktop, Opus coming soon?
* TS support only in Chromecast
* MP3 audio-only support behind a flag (skips ID3 tags and Shoutcast headers)
* Internet Explorer 11 supports ISOBFF with H264/AAC
* Only on Windows 8.1 (NOT Windows 7)
* Firefox working on implementations
* See bug #855130 on Bugzilla
* Apple looking into MSE support
* Some work done; desktop-only
* Further down the road.
* WebKit gStreamer MSE work in progress too
* All Webkit bugs tagged "mse"
!! 3. Is API compatible between browsers?
* MSE on Chrome actually behind the spec
* no "mode", no "totalframedelay"
* "appendStream" behind a flag
* no A/V trackList, because Blink doesn't support it yet.
* there's currently no place that lists what's supported
* maybe on Chromium.org?
* All layout tests in Blink are written against W3C spec.
* Some are currently WebM biased
* IE seems pretty complete and based upon latest spec
* YouTube plays MSE in IE11.
!! 4. How does memory management / garbage collection works?
* Not clear how this works on Webkit.
* Simple solutions don't always work well.
* How to do AES decryption on top of MSE?
* Using web workers to decouple.
* Should be possible on Chrome/Android
* FYI: updating bytestreams while appending not supported
* (similar issue was raised with web audio)
* Chrome actually makes copy instead of holding reference
* Neutering is problematic, since other Views could be blown away (needs further defined first)
!! 5. What about DASH?
* YouTube is doing it today.
* Including VP8/VP9
* DASH.js reference implementation on DASH evangelist site.
* Contains several example streams.
* Wowza servers are supporting DASH.
* DASH264 only
!! 6. What about text tracks?
* Currently reviewing VTT support in WebM
* Enabled in both SME and WebM progressive.
* No overlapping, just full cues
* Framework in Chrome for in-band text tracks in development
* Muxed or not
* Landing soon...
* would also open support for multiple A/V tracks.
* Spec for VTT and TTML in MP4 written
* Only supports full cues (no overlapping)
* Spec lives on Text Tracks working group wiki (by Cyril)
* What about in-band cues?
* Just VTT track with the "kind" set to metadata
