A tiny shell-script based TAP-compliant testing framework
______ ______ __ __ ______
/\__ _\ /\ ___\ /\ \_\ \ /\__ _\
\/_/\ \/ \ \___ \ \ \ __ \ \/_/\ \/
\ \_\ \/\_____\ \ \_\ \_\ \ \_\
\/_/ \/_____/ \/_/\/_/ \/_/
- Installation
- Usage
- API - core
- API - file
- API - string
- API - jq
- API - diff
- API - colordiff
- API - shxml
In general you don't have to install tsht, simply add the wrapper script to your project.
- Create a test directory, e.g.
test - Download the wrapper script
cd test && wget 'https://cdn.rawgit.com/kba/tsht/master/tsht' - Create your unit tests
- Execute all tests using
./tshtor specific tests using./tsht <path/to/unit-test.tsht>...
The first time you execute the wrapper script, it will clone this repository to
.tsht and execute the runner. Whenever you want to update the tsht framework,
simply delete the .tsht folder and an up-to-date version of the framework
will be cloned when you next run your tests.
To execute tsht scripts without the runner, you will need to have tsht in
your $PATH. You can either set $PATH up manually to include the directory that contains the
tsht wrapper or clone this repository and use the Makefile.
To install system-wide:
sudo make install
To install to your home directory:
make PREFIX=$HOME/.local install
Tsht unit tests are written in a DSL superset of the Bash shell scripting language. This means that any bash code can be used in a tsht script.
All tsht scripts must end with .tsht.
All tsht scripts should start with #!/usr/bin/env tsht
Tsht scripts are executed in alphabetic order, so prefix the scripts you want to run early with a low number.
Usage: tsht [options...] [<path/to/unit.tsht>...]
Options:
--help -h Show this help
--color Highlight passing/failing tests in green/red
--update Update the tsht framework from git
--version -V Show last revision of the runner
In addition to the core functionality, tsht can be extended with extensions. An extension is a subdirectory of tsht that has this structure:
/ext/<name>
ext/
└── <name>
├── Makefile
└── <name>.sh
The Makefile must have an install target that can install necessary binaries
into $(PREFIX)/bin.
To use an extension, call the use directive:
use 'jq'This will call make install in the extension directory and set the PATH
variable to let the extension use the locally installed software.
Currently, these extensions are available:
- jq (API, test): A wrapper around the jq CLI JSON query tool
- diff (API, test): Show differences with color-highlighted diff
- diff (API, test): Show differences with diff
- shxml (API, test) Use XML based tools in tests (XSD, XSD, XPATH…)
Some unit tests require setup work before they run and teardown work
after they run. To make this easier and reusable, these tasks can be
grouped in before and after hooks, which are shell scripts or
shell functions.
Hooks are always test-specific and are looked for in three places:
- Shell function
after - Shell scripts named like the test with suffix
.before/.after - Shell scripts in the same directory as the test named
.before/.after
Note: before cannot be a shell function in the script because the script
must be sourced to declare it and once it has been sourced, it's too late to
call the before hook.
#!/usr/bin/env tsht
plan 4
equals $(( 84 / 2 )) 42 "three score and six"
exec_ok "ls /"
match "oo" "foobar"
not_ok $(( 0 / 42 )) "Nothing divided is nothing"If you are a vim user, try out the tsht.vim
plugin which will detect .tsht files, highlight the builtin functions
and execute scripts with the closest wrapper.
There are various TAP consumers that can produce nice output, the tape NodeJS TAP-based framework lists a few.
For example, using the tap-spec TAP reporter can be installed using
npm install -g tap-spec
For the tests of tsht itself, it will produce output like this:
$ ./test/tsht | tap-spec
Testing ./runner/update/update.tsht
✔ Executed: git clone ../../../ /home/kb/build/github.com/kba/tsht/test/runner/update/test-project/.tsht
✔ Executed: git checkout master
✔ Executed: git reset --hard bd9fbafa643f10087cb24ff0f3b47a9d33a12a26
✔ HEAD is bd9fbafa643f10087cb24ff0f3b47a9d33a12a26
✔ Executed: ./tsht --update
✔ HEAD is 81ce4a381d416855eb37b99a087e8d01edae661c
Testing ./runner/help/help.tsht
✔ Executed: /home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh --help
✔ Executed: /home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh -h
✔ -h == --help
✔ Matches '--color': 'Usage: tsht [options...] [...] Options: --help -h Show this help --color Highlight passing/failing tests in green/red --update '
✔ Matches '--update': 'Usage: tsht [options...] [...] Options: --help -h Show this help --color Highlight passing/failing tests in green/red --update '
✔ Matches '--version': 'Usage: tsht [options...] [...] Options: --help -h Show this help --color Highlight passing/failing tests in green/red --update '
✔ Failed as expected (2) '/home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh --foobar'
Testing ./runner/color/color-test.tsht
/home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh --color thetest
✔ Color output as expected
Testing ./runner/0tshtlib/tshtlib.tsht
✔ TSHTLIB is set
✔ TSHTLIB is relative to this dir
✔ /home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh is the right tsht-runner.sh
Testing ./file.tsht
✔ Failed as expected (2) 'ls does-not-exist'
✔ Executed: touch does-not-exist
✔ File exists: does-not-exist
✔ Not empty file: does-not-exist
✔ (unnamed equals assertion)
Testing ./api/core/not_ok.tsht
✔ Empty string
✔ 0
✔ "0"
Testing ./api/core/exec_fail.tsht
✔ Failed as expected (2) 'ls does-not-exist'
Testing ./api/core/ok.tsht
✔ Me testing my existence
Testing ./api/core/exec_ok.tsht
✔ Executed: touch does-not-exist
Testing ./api/file/file_exists.tsht
✔ File exists: does-not-exist
Testing ./api/file/file_not_empty.tsht
✔ Not empty file: does-not-exist
Testing ./api/string/match.tsht
✔ Matches '^\d+': '1234'
✔ Matches '^\d+$': '1234'
✔ Matches '^a\d+$': 'a1234'
Testing ./api/string/equals.tsht
✔ (unnamed equals assertion)
✔ (unnamed equals assertion)
✔ (unnamed equals assertion)
✔ (unnamed equals assertion)
Testing ./api/string/not_match.tsht
✔ Not like '^\d+$': 'string'
Testing ./api/string/not_equals.tsht
✔ 1984 test
Testing ./ext/shxml/shxml-basic.tsht
✔ Executed: shxml --help
Testing ./ext/colordiff/diff.tsht
✔ A colordiff is a colordiff is a colordiff
Testing ./ext/jq/jq.tsht
✔ Executed: jq --version
✔ From string
✔ From STDIN (1)
✔ From STDIN (2)
✔ JSON: .foo.bar[1] -> '42'
✔
Testing ./ext/diff/diff.tsht
✔ A diff is a diff is a diff
Testing ./issues/issue_8.tsht
✔ Matches '--foo': '--foo'
Testing ./before-after/ba2.tsht
✔ File does not exist: DELETEME
Testing ./before-after/ba.tsht
Sourcing suffix suffixscript ba.tsht.before for ba.tsht
✔ File exists: DELETEME
Sourcing suffix suffixscript ba.tsht.after for ba.tsht
total: 51
passing: 51
duration: 226ms
This library the core functions of tsht. It is always included and includes the most commonly used libraries:
Specify the number of planned assertions
plan <number-of-tests>
Fail unconditionally
fail <message> [<additional-output>]
The additional output will be prefixed with #.
Succeed unconditionally.
See fail
Execute a command (or function) and succeed when its return code matches the parameter
exec_fail <expected-return> [<cmd-args>...]
Example
exec_fail 2 "ls" "-la" "DOES-NOT-EXIST"
Execute a command (or function) and succeed when it returns zero.
Example
exec_ok "ls" "-la"
Succeed if the first argument is a non-empty non-zero string
Succeed if the first argument is an empty string or zero.
Use an extension library
use 'jq'
Succeed if a file (or folder or symlink...) exists.
file_exists ".git"
Succeed if a file (or folder or symlink...) does not exist.
not_file_exists "temp"
ALIAS: file_not_empty
Succeed if a file exists and is a non-empty file.
Succeed if the first arguments match the contents of the file in the second argument.
Succeed if the contents of two files match, filenames passed as arguments.
This library contains functions testing strings and numbers
Test for equality of strings
equals <expected> <actual> [<message>]
Example:
equals "2" 2 "two equals two"
equals 2 "$(wc -l my-file)" "two lines in my-file"
Inverse of equals.
Succeed if a string matches a pattern
match "^\d+$" "1234" "Only numbers"
Succeed if a string does not match a pattern
not_match "^\d+$" "abcd" "Only numbers"
Extension that allows testing JSON strings.
Enable with
use jq
See jq Github repo.
Test if jq expression validates
Test if jq expression is as exepected
Extension that replaces the builtin equals with
a function that shows the difference as a unified diff
Enable with
use diff
Requires diff(1).
Test for equality of strings and output unified diff on fail.
equals <expected> <actual> [<message>]
Example:
equals "2" 2 "two equals two"
equals 2 "$(wc -l my-file)" "two lines in my-file"
Extension that replaces the builtin equals with
a function that shows the difference in a colored diff output.
Enable with
use colordiff
Requires perl.
Test for equality of strings and output colored diff on fail.
equals <expected> <actual> [<message>]
Example:
equals "2" 2 "two equals two"
equals 2 "$(wc -l my-file)" "two lines in my-file"
Extension that makes shxml available
Enable with
use shxml
See shxml Github repo.