Skip to content
Automated Testing for Set-Top Boxes and Smart TVs
Python Shell C HTML Makefile Ruby Other
Latest commit 582acc8 @wmanley wmanley Merge pull request #331 from stb-tester/load-template-error-message
core: Fix path in error message if template image not found
Failed to load latest commit information.
_stbt core: Fix path in error message if template image not found
docs virtual-stb: Add virtual stb - useful for testing against STB simulators
extra virtual-stb: Add virtual stb - useful for testing against STB simulators
stbt-batch.d stbt batch: Fix report generation when output directory name has spaces
stbt-camera.d Fix isort errors reported by `make check`
stbt Python API: docstrings: Document additions/changes since 0.21
tests core: Fix path in error message if template image not found
.gitignore .gitignore generated extra/debian/changelog file
.isort.cfg Add .isort.cfg to fix false positives on Travis
.travis.yml virtual-stb: Add virtual stb - useful for testing against STB simulators Add extra/debian/ instead of `make deb` recipe
LICENSE Man page (generated from README.rst) and LGPL v2.1 LICENSE file.
Makefile virtual-stb: Add virtual stb - useful for testing against STB simulators
README.rst x11 control: Add support for mapping input.h names to X keysyms
irnetbox-proxy Replace references to{drothlis => stb-tester}/stb-tester
stbt-batch Rename `stbt-batch.d/run` -> `stbt-batch.d/`
stbt-camera Replace references to{drothlis => stb-tester}/stb-tester
stbt-completion stbt run: Print details of failed assert statement
stbt-config Replace references to{drothlis => stb-tester}/stb-tester
stbt-control librification: Pull code into `_stbt/`
stbt-lint Replace references to{drothlis => stb-tester}/stb-tester
stbt-power stbt power: Add support for the Aviosys USB Net Power 8800 Pro
stbt-record librification: Pull code into `_stbt/`
stbt-run stbt-run: Fix printing unicode tracebacks
stbt-screenshot Replace references to{drothlis => stb-tester}/stb-tester
stbt-templatematch librification: Pull code into `_stbt/`
stbt-tv Replace references to{drothlis => stb-tester}/stb-tester
stbt.conf stbt.conf: Change capitalisation of `control` value "None" => "none" virtual-stb: Add virtual stb - useful for testing against STB simulators virtual-stb: Add virtual stb - useful for testing against STB simulators



Automated User Interface Testing for Set-Top Boxes & Smart TVs
Copyright:Copyright (C) 2013-2014 Ltd, 2012-2014 YouView TV Ltd. and other contributors
License:LGPL v2.1 or (at your option) any later version (see LICENSE file in the source distribution for details)
Manual section:1
Manual group:stb-tester


stbt record [options]

stbt run [options] script[::testcase]


stbt record will record a test case by listening for remote-control keypresses, taking screenshots from the set-top box as it goes.

You then (manually) crop the screenshots to the region of interest.

(Optionally) you manually edit the generated test script, which will look something like this:

def test_that_i_can_tune_to_bbc_one_from_the_guide():
    wait_for_match("BBC One.png")

stbt run will play back the given test script, returning an exit status of success or failure for easy integration with your existing test reporting system.

stbt has other auxiliary sub-commands; run stbt --help for details.


Global options


A remote control to use for controlling the set top box. uri can be:


A hardware infrared emitter controlled by the lirc (Linux Infrared Remote Control) daemon.

  • If lircd_socket is specified (or none of lircd_socket, hostname and port are specified) remote control commands are sent via a lircd socket file. lircd_socket defaults to /var/run/lirc/lircd.
  • If port is specified, remote control commands are sent via a lircd TCP listener on localhost.
  • If hostname and port are specified, remote control commands are sent via a lircd TCP listener on a remote host.

remote_control_name is the name of a remote-control specification in lircd.conf.

RedRat irNetBox network-controlled infrared emitter hardware. hostname is the hostname or IP address of the irNetBox device. port is the TCP port of the irNetBox device. It defaults to 10001, which is the port used by real irNetBox devices; override if using an irNetBox proxy that listens on a different port. output is the infrared output to use, a number between 1 and 16 (inclusive). config_file is the configuration file that describes the infrared protocol to use; it can be created with RedRat's (Windows-only) "IR Signal Database Utility". stbt supports the irNetBox models II and III.
Controls Roku players using the Roku's HTTP control protocol. Stb-tester's standard key names (like "KEY_HOME") will be converted to the corresponding Roku key name, or you can use the Roku key names directly.
Can be used to control Samsung Smart TVs using the same TCP network protocol that their mobile phone app uses. Tested against a Samsung UE32F5370 but will probably work with all recent Samsung Smart TVs.
Ignores key press commands.
Used by the selftests to change the input video stream. Only works with --source-pipeline=videotestsrc. A script like press("18") will change videotestsrc's pattern property (see gst-inspect videotestsrc).

Send keypresses to a given X display using the xtest extension. Can be used with GStreamer's ximagesrc for testing desktop applications, websites and set-top box software running on a PC.

The (optional) key_mapping_file is used to translate between the stb-tester keynames that you use in your test-scripts and X keysyms that X understands. The file looks like:

# This is a comment

KEY_FASTFORWARD   parenright
KEY_REWIND        parenleft

The column on the left is the key name you'll be using in your test-cases, the column on the right is the X keysym that that key will be translated to. For a full list of X keysyms see .

stbt provides some sensible default mappings when there is an obvious match for our `standard key names<>`_.

The x11 control requires that xdotool is installed.


A GStreamer pipeline providing a video stream to use as video output from the set-top box under test. For the Hauppauge HD PVR use:

v4l2src device=/dev/video0 ! tsdemux ! h264parse
 A GStreamer pipeline to use for video output, like xvimagesink.
 Restart the GStreamer source pipeline when video loss is detected, to work around the behaviour of the Hauppauge HD PVR video-capture device.
-v, --verbose

Enable debug output.

With stbt run, specify -v twice to dump intermediate images from the image processing algorithm to the ./stbt-debug directory. Note that this will dump a lot of files -- several images per frame processed. This is intended for debugging the image processing algorithm; it isn't intended for end users.

Additional options to stbt run

 Record a video (in the HTML5-compatible WebM format) to the specified file.

Additional options to stbt record


The source of remote control presses. uri can be:

A hardware infrared receiver controlled by the lirc (Linux Infrared Remote Control) daemon. Parameters are as for --control.
Reads remote control keypresses from a newline-separated list of key names. For example, file:///dev/stdin to use the keyboard as the remote control input.
Launches stbt control to record remote control keypresses using the PC keyboard. See stbt control --help for details. Disables --verbose parameter.
-o <filename>, --output-filename=<filename>
 The file to write the generated test script to.


All parameters that can be passed to the stbt tools can also be specified in configuration files. Configuration is searched for in the following files (with later files taking precedence):

  1. /etc/stbt/stbt.conf
  2. ~/.config/stbt/stbt.conf

$STBT_CONFIG_FILE is a colon-separated list of files where the item specified at the beginning takes precedence.

These files are simple ini files with the form:

source_pipeline = videotestsrc
sink_pipeline = xvimagesink sync=false
control = None
verbose = 0
save_video = video.webm
output_file =
control_recorder = file:///dev/stdin

Each key corresponds to a command line option with hyphens replaced with underscores.


0 on success; 1 on test script failure; 2 on any other error.

Test scripts indicate failure (the system under test didn't behave as expected) by raising an instance of stbt.UITestFailure (or a subclass thereof) or AssertionError (which is raised by Python's assert statement). Any other exception is considered a test error (a logic error in the test script, an error in the system under test's environment, or an error in the test framework itself).


The test rig consists of a Linux server, with:

  • A video-capture card (for capturing the output from the system under test)
  • An infrared receiver (for recording the system-under-test's infrared protocol)
  • An infrared emitter (for controlling the system under test)

Video capture card

You'll need a capture card with drivers supporting the V4L2 API (Video-for-Linux 2). We recommend a capture card with mature open-source drivers, preferably drivers already present in recent versions of the Linux kernel.

The Hauppauge HD PVR works well (and works out of the box on recent versions of Fedora), though it doesn't support 1080p. If you need an HDCP stripper, try the HD Fury III.

Infra-red emitter and receiver

An IR emitter+receiver such as the RedRat3, plus a LIRC configuration file with the key codes for your set-top box's remote control.

Using software components instead

If you don't mind instrumenting the system under test, you don't even need the above hardware components.

stb-tester uses GStreamer, an open source multimedia framework. Instead of a video-capture card you can use any GStreamer video-source element. For example:

  • If you run tests against a VM running the set-top box software instead of a physical set-top box, you could use the ximagesrc GStreamer element to capture video from the VM's X Window.
  • If your set-top box uses DirectFB, you could install the DirectFBSource GStreamer element ( on the set-top box to stream video to a updsrc GStreamer element on the test rig.

Instead of a hardware infra-red receiver + emitter, you can use a software equivalent (for example a server running on the set-top box that listens on a TCP socket instead of listening for infra-red signals, and your own application for emulating remote-control keypresses). Using a software remote control avoids all issues of IR interference in rigs testing multiple set-top boxes at once.

Linux server

An 8-core machine will be able to drive 4 set-top boxes simultaneously with at least 1 frame per second per set-top box.


  • A Unixy operating system (we have only tested on Linux and Mac OS X).
  • Drivers for any required hardware components.
  • GStreamer 1.0 (multimedia framework) + gstreamer-plugins-base + gstreamer-plugins-good.
  • python 2.7 + pygst + docutils (for building the documentation) + nose (for the self-tests).
  • OpenCV (image processing library) version >= 2.0.0, and the OpenCV python bindings.
  • For the Hauppauge video capture device you'll need the gstreamer-libav package (e.g. from the rpmfusion-free repository) for H.264 decoding.


Run "make install" from the stb-tester source directory.

See for the required dependencies and configuration.


The test scripts produced and run by stbt record and stbt run, respectively, are actually python scripts, so you can use the full power of python. Don't get too carried away, though; aim for simplicity, readability, and maintainability.

See the Python API documentation at


  • When cropping images to be matched by a test case, you must select a region that will not be present when the test case fails, and that does not contain any elements that might be absent when the test case succeeds. For example, you must not include any part of a live TV stream (which will be different each time the test case is run), nor translucent menu overlays with live TV showing through.
  • Crop template images as tightly as possible. For example if you're looking for a button, don't include the background outside of the button. (This is particularly important if your system-under-test is still under development and minor aesthetic changes to the UI are common.)
  • Always follow a press with a wait_for_match -- don't assume that the press worked.
  • Use press_until_match instead of assuming that the position of a menu item will never change within that menu.
  • Use the timeout_secs parameter of wait_for_match and wait_for_motion instead of using time.sleep.
  • Rename the template images captured by stbt record to a name that explains the contents of the image.
  • Extract common navigation patterns into separate python functions. It is useful to start each test script by calling a function that brings the system-under-test to a known state.



Something went wrong with that request. Please try again.