Main.AutomatedTesting History

Hide minor edits - Show changes to output

November 18, 2013, at 06:47 PM by 208.70.31.249 -
Changed lines 17-20 from:
unit frameworks
 - qunit
 - tape
to:
JavaScript unit frameworks
 * qunit
 * tape
Changed lines 24-26 from:
 - each browser vendor has suite of browser tests

to:
 * each browser vendor has suite of browser tests

Changed lines 33-42 from:
 - Xiph: Testing for Opus
  - 160 core cluster w/ Jenkins
  - Each commit 1000's of hours of audio processed
  - Performance testing is harder
  - Multi-platform support (ARM, etc)
  - Lots of different chips
  - Consistent test results: hard
  - What test hooks can be added to codec implementations?
  - Support extensions for test hooks
to:
 * Xiph: Testing for Opus
  * 160 core cluster w/ Jenkins
  * Each commit 1000's of hours of audio processed
  * Performance testing is harder
  * Multi-platform support (ARM, etc)
  * Lots of different chips
  * Consistent test results: hard
  * What test hooks can be added to codec implementations?
  * Support extensions for test hooks
Changed lines 44-46 from:
 - not generalizable

Expose device hardware capabilitiesvia API?
to:
 * not generalizable

Expose device hardware capabilities via API?
Restore
November 18, 2013, at 06:46 PM by 208.70.31.249 -
Added lines 1-48:
Automated testing

Selenium -> browser
Mobile -> ??

Canvas; screen grab w/ control stream

Mozilla eideticker (monitors output of device)
 - https://wiki.mozilla.org/Project_Eideticker

Remote developer console on devices

Wienre / WebDriver (Wire)

What to do with permission dialogs / extra-DOM stuff

unit frameworks
 - qunit
 - tape

Instrumetnation in the wild (w/ YouTube)

webplatform.org
 - each browser vendor has suite of browser tests


Orchestrated multiple browser w/ mult. participants very hard to test (WebRTC)

KitKat will solve a lot of problems (runs on older, weaker hardware)

Success Stories?

 - Xiph: Testing for Opus
  - 160 core cluster w/ Jenkins
  - Each commit 1000's of hours of audio processed
  - Performance testing is harder
  - Multi-platform support (ARM, etc)
  - Lots of different chips
  - Consistent test results: hard
  - What test hooks can be added to codec implementations?
  - Support extensions for test hooks

Rdio: Capture HDMI output and compare to expected decoded frames
 - not generalizable

Expose device hardware capabilitiesvia API?

Performance tests / certification
Restore