Intros
- ML vision implementations
- Decentralized ML video
- AI for bit rate ladder
- AI encode quality, configure encoder to optimize
- AI for subtitle conversions;
- Crunchyroll; looking at translation across graphic overlays matching styles and color.
- AI for ABR / Content aware encoding
- ML understand terms. ABR grand challenge; AI whatever operate on large buffers. Operate quickly on small samples.
- Web live captions; IVS WebGPU? live captions; Live Super Resolution; 16ms APIs? only on Chrome. Building their own foundational model; license speed quality.
- Client side upscaling; lots of complexity.
- Some discussion on WebCodes? and WebGPU?
AI for ABR
- ABR works pretty well; more buffer.
- Problem areas - When measuring the TCP connection becomes more challenging ?
- How do you measure ABR busy waiting time before connection.
- Patterns in ABR; when are these patterns interesting do they have positive impact ?
- https://ieeexplore.ieee.org/document/9429986 - paper on AI bitrate ladder
- are we talking client side or sever side.
- just because a device has 1080P support; not sure it means thats the right choice.
- Tune things that are the best output.
- 720 better color & quality may be preferred.
- Clients that play the lowest common denominator?
What do we mean by AI ?
WebGPU? browser
Point of interest tracking
- Post processing trying to to track the ball to help create vertical video from horizontal video.
- AI for ads; around content detection.
- point of interest detection and avoid captions overlapping that.
- its done manually today; but can it be done automatically
- Would be of interest to have a point of interest to block out areas for captions
- analyzing the video for advertising reasons; to have better bidding relative to whats happening in the video.
Action items
- look into protocol for areas to avoid listed of the frame to avoid displaying captions (where automated captions are displayed)
- AI ABR for VOD seems like overkill ( don't do it )
- More standardization around IAB content category