- Track A3 - Web subtitles and captions
Michael:
- Positioning of auto generated closed captioning texts is not context aware and can overlay text in non optimal locations
- Group thoughts:
- Humans should be in the chain at some point.
- AI on bottom third of frame to analysis, any more would get more expensive.
Eric:
Text tracks in MSE
- No browser pays attention to MSE text tracks
- Apple bringing support to Webkit
- Real potential problem
- Spec says, when new initialization segment comes through if the characteristics of all the tracks are not the same, the stream will fail with an error. Someone inserts an Ad within primary and the Video and Audio tracks have same characteristics, but Audio tracks are likely to be different and typically ignored by packagers. Apple thinks the spec should change. Tracks can change quite often with HLS playback.
- Group thoughts:
- There are no hardware initialization, so no reason to maintain this requirement for Audio Tracks.
- Track selection logic/hierarchy should be documented by WebKit?
- Should browser do anything new in the case where there are inband captions that the player doesn't know how to handle?
- Group thoughts:
- Maybe there should be a way to expose events from the stream. Could tie in the text track cue constructor with an array buffer of raw data. Client is responsible for parsing and feeding back into the track? 608/708 are not event based.
- How can we help?
- Keep discussion up, get firm requirements and decisions, make PR's against spec.
- To talk with Marcos(?) to setup a Task force.
- Gaps in text tracks should not stall playback like gaps in audio/video=
- Initialization segments do not contain characteristics of audio tracks if embedded 608/708. This would fail the current spec.
Gary:
- RemoveTextTrack` API should be added back !!!
- Maybe only allow tracks that were added via text track API ?
Eric:
- Should the User Agent be responsible for clearing cues outside of the buffer range?
- Group thoughts:
- Agree that these is a good idea, but to make sure when seeking occurs outside of buffer range to re populate the cues
Michael:
- WebVTT? with LL-HLS is a bit ambigous and player support is unclear.
- Group thoughts:
- Potential spec change Don't flash captions when switching cues
Eric:
- Push to have WebVTT? in Interop 2025!
Pierre:
- Time text engine in ffmpeg for vtt and ttml! This would be very helpful for the community and the world! Bring the code from various projects.
- Please do not do rendering/converter/parser from scratch!
Action Items:
- To talk with Marcos to regarding setting up media working group task force to create PR to change Spec on MSE text tracks.
- RemoveTextTracks? API Html more limited in scope
- Eric to document caption selection in WebKit?