Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
stbt batch: Stb tester's bulk test running and reporting tool
stbt batch run will run a set of test scripts with stbt run, automatically
triage known failure reasons, and generate an html report:
The above report shows the result of running a simple stb-tester test script that uses GStreamer's videotestsrc as the video input. The first time the test was run (the red row in the report) it failed (in this case, due to a bug in the test; red rows normally indicate a possible defect in the system under test). The next run passed because the test had been fixed (note the change in the "commit" column, showing the revision of the test). The last run failed because the user interrupted it with Control-C.
- Clicking on a row shows detailed logs for that test run.
- The up/down arrow keys select the previous/next test run.
- A screenshot is taken if the test failed.
- A video of each test run is captured. The video's format is HTML5 compatible, so it is playable in most modern browsers (it doesn't work in Safari but Firefox, Chrome and VLC all play it).
- Stb-tester detects why the test failed and logs this failure reason in the "exit status" column of the report. If stb-tester can be sure that the failure wasn't caused by a defect in the system under test, the row is coloured yellow instead of red.
- Users can provide their own custom classification scripts via hooks specified in the stbt configuration file.
- Running the optional
stbt batch instawebstarts a webserver that allows interactive manual editing of the "failure reason" and "notes" fields.
For details see stbt-batch.d/README.rst in the stb-tester source repository.
stbt batch was first available in stb-tester 0.18.