stbt batch: Stb tester's bulk test running and reporting tool

David Röthlisberger edited this page Mar 18, 2015 · 1 revision

stbt batch run will run a set of test scripts with stbt run, automatically triage known failure reasons, and generate an html report:

The above report shows the result of running a simple stb-tester test script that uses GStreamer's videotestsrc as the video input. The first time the test was run (the red row in the report) it failed (in this case, due to a bug in the test; red rows normally indicate a possible defect in the system under test). The next run passed because the test had been fixed (note the change in the "commit" column, showing the revision of the test). The last run failed because the user interrupted it with Control-C.

Features:

  • Clicking on a row shows detailed logs for that test run.
  • The up/down arrow keys select the previous/next test run.
  • A screenshot is taken if the test failed.
  • A video of each test run is captured. The video's format is HTML5 compatible, so it is playable in most modern browsers (it doesn't work in Safari but Firefox, Chrome and VLC all play it).
  • Stb-tester detects why the test failed and logs this failure reason in the "exit status" column of the report. If stb-tester can be sure that the failure wasn't caused by a defect in the system under test, the row is coloured yellow instead of red.
  • Users can provide their own custom classification scripts via hooks specified in the stbt configuration file.
  • Running the optional stbt batch instaweb starts a webserver that allows interactive manual editing of the "failure reason" and "notes" fields.

For details see stbt-batch.d/README.rst in the stb-tester source repository. stbt batch was first available in stb-tester 0.18.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.