Ai Video

Intros

  • ML vision implementations
  • Decentralized ML video
  • AI for bit rate ladder
  • AI encode quality, configure encoder to optimize
  • AI for subtitle conversions;
  • Crunchyroll; looking at translation across graphic overlays matching styles and color.
  • AI for ABR / Content aware encoding
  • ML understand terms. ABR grand challenge; AI whatever operate on large buffers. Operate quickly on small samples.
  • Web live captions; IVS WebGPU? live captions; Live Super Resolution; 16ms APIs? only on Chrome. Building their own foundational model; license speed quality.
  • Client side upscaling; lots of complexity.
  • Some discussion on WebCodes? and WebGPU?

AI for ABR

  • ABR works pretty well; more buffer.
    • Problem areas - When measuring the TCP connection becomes more challenging ?
    • How do you measure ABR busy waiting time before connection.
    • Patterns in ABR; when are these patterns interesting do they have positive impact ?
  • https://ieeexplore.ieee.org/document/9429986 - paper on AI bitrate ladder
  • are we talking client side or sever side.
  • just because a device has 1080P support; not sure it means thats the right choice.
  • Tune things that are the best output.
  • 720 better color & quality may be preferred.
  • Clients that play the lowest common denominator?

What do we mean by AI ?

WebGPU? browser

Point of interest tracking

  • Post processing trying to to track the ball to help create vertical video from horizontal video.
  • AI for ads; around content detection.
  • point of interest detection and avoid captions overlapping that.
  • its done manually today; but can it be done automatically
  • Would be of interest to have a point of interest to block out areas for captions
  • analyzing the video for advertising reasons; to have better bidding relative to whats happening in the video.

Action items

  • look into protocol for areas to avoid listed of the frame to avoid displaying captions (where automated captions are displayed)
  • AI ABR for VOD seems like overkill ( don't do it )
  • More standardization around IAB content category