Running OmnisTAP from the command line
There are two commands available in OmnisTAP from the command-line.
runtests tap_output [regexp]
gettests will list all tests that are available in the library whose startup task subclasses
runtests will run these tests, saving output to
tap_output. Both commands can limit the tests they see using an Omnis-compatible regular expression passed to
regexp. If you need to pass additional parameters, such as a test database or host, use
. as the
regexp value to run all tests.
TAP output is saved as one file per test class under
[tap_output]/[test_method].tap. You can evaluate this output using any TAP-compatible consumer.
OmnisTAP will enforce tests run within a certain amount of time. These limits are 5 seconds for unit tests and 90 seconds for integration tests. You can adjust this timing from the command line by exporting the
OMNIS_TEST_TIMING_MULTIPLIER variable as a floating-point number. For example, a macOS machine that runs tests twice a slowly would run:
On Windows, you can use:
When running from the CLI the startup and shutdown methods are available to bootstrap a test environment. Add a test class to your library that subclasses
$startup() to run code at the start of the run, and
$shutdown() to run code at the end of the run.
When running from the command line, OmnisTAP will write a number of csv files to a
diagnostics directory in the
tap_output directory. These files are:
coverage.csv tracks code coverage during the test run. Coverage is measured at the method, or functional, level. Method execution is tracked using Omnis'
$collectperformancedata feature. Because this feature tracks method calls, not individual lines of code, there may be execution branches in your methods that are not evaluated even though the method is considered "covered". Methods from OmnisCLI and OmnisTAP are excluded from this coverage measurement.
profile.csv outputs the raw profiling performance data for all methods that were executed at least once during the test run, including those in OmnisCLI and OmnisTAP. See the Omnis Programming manual under Chapter 4 -> Method Performance -> Collecting Performance Data for details on this data. This output can be helpful for identifying performance issues in your application and optimizing them.
test_types.csv reports the number of unit and integration tests run. This output can help ensure your test suite follows Martin Fowler's Test Pyramid concept.
timings.csv reports how long each test took to run, sorted in descending order by time. Use this output to identify slow tests for optimization.
These diagnostics are reported as CSV files to allow easy parsing and analysis from run to run.