Media Source Extensions (Aaron)
1. What Changed since last year's FOMS
Many features added to SME since last year:
- appendMode attribute to sourceMode object (to handle transport streams; ignore timestamps;
- to offload discontinuities handling to framework
- also good for advertising insertion.
- All append operations are a-synchronous now
- Append Window: support appending sections of chunks.
- Better TrackList? management
- New API to remove segments (to out-smart the browser around garbage collection)
- Statistics functionality: videoPlaybackQuality object:
- corruptedVideoFrames
- creationTime
- droppedVideoFrames
- totalVideoFrames
- totalFrameDelay
At risk for removal: "appendStream" and "totalFrameDelay".
videoPlaybackQuality is on video element, so could be used for non-MSE playback as well. Still part of MSE specification though.
2. Codec/container support in implementations:
- Chrome supports:
- Internet Explorer 11 supports ISOBFF with H264/AAC
- Only on Windows 8.1 (NOT Windows 7)
- Firefox working on implementations
- See bug #855130 on Bugzilla
- Apple looking into MSE support
3. Is API compatible between browsers?
- MSE on Chrome actually behind the spec
- no "mode", no "totalframedelay"
- "appendStream" behind a flag
- no A/V trackList, because Blink doesn't support it yet.
- there's currently no place that lists what's supported
- maybe on Chromium.org?
- All layout tests in Blink are written against W3C? spec.
- IE seems pretty complete and based upon latest spec
4. How does memory management / garbage collection works?
- Not clear how this works on Webkit.
- Simple solutions don't always work well.
- How to do AES decryption on top of MSE?
- Using web workers to decouple.
- Should be possible on Chrome/Android
- FYI: updating bytestreams while appending not supported
- (similar issue was raised with web audio)
- Chrome actually makes copy instead of holding reference
- Neutering is problematic, since other Views could be blown away (needs further defined first)
5. What about DASH?
- YouTube? is doing it today.
- DASH.js reference implementation on DASH evangelist site.
- Contains several example streams.
- Wowza servers are supporting DASH.
6. What about text tracks?
- Currently reviewing VTT support in WebM?
- Framework in Chrome for in-band text tracks in development
- Muxed or not
- Landing soon...
- would also open support for multiple A/V tracks.
- Spec for VTT and TTML in MP4? written
- Only supports full cues (no overlapping)
- Spec lives on Text Tracks working group wiki (by Cyril)
- What about in-band cues?
- Just VTT track with the "kind" set to metadata