forked from todotxt/todo.txt-cli
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Initial test framework and first test.
Add a basic test framework, borrowed from the framework used by git.git. A shell script library (tests/test-lib.sh) helps generate the fixtures, and simplifies the process of writing test scripts. Tests can be run as a suite (via 'make test') or individually (sh tests/t0000-config.sh). Results are aggregated upon completion. Includes a detailed README. A basic test of config file processing is part of this commit. Signed-off-by: Emil Sit <sit@emilsit.net>
- Loading branch information
Showing
6 changed files
with
800 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,3 @@ | ||
VERSION-FILE | ||
tests/test-results | ||
tests/trash\ directory.* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,219 @@ | ||
todo.sh tests | ||
============= | ||
|
||
This directory holds test scripts for todo.sh. The | ||
first part of this short document describes how to run the tests | ||
and read their output. | ||
|
||
When fixing the tools or adding enhancements, you are strongly | ||
encouraged to add tests in this directory to cover what you are | ||
trying to fix or enhance. The later part of this short document | ||
describes how your test scripts should be organized. | ||
|
||
|
||
Running Tests | ||
------------- | ||
|
||
The easiest way to run tests is to say "make test" from the top-level. | ||
This runs all the tests. | ||
|
||
rm -rf tests/test-results "tests/trash directory"* | ||
cd tests && sh t0000-config.sh | ||
* ok 1: no config file | ||
* ok 2: config file (default location 1) | ||
* ok 3: config file (default location 2) | ||
* ok 4: config file (command line) | ||
* ok 5: config file (env variable) | ||
* passed all 5 test(s) | ||
cd tests && sh t0001-null.sh | ||
* ok 1: null ls | ||
* passed all 1 test(s) | ||
rm -rf tests/test-results | ||
|
||
Or you can run each test individually from command line, like | ||
this: | ||
|
||
$ ./t0001-null.sh | ||
* ok 1: null ls | ||
* passed all 1 test(s) | ||
|
||
You can pass --verbose (or -v), --debug (or -d), and --immediate | ||
(or -i) command line argument to the test, or by setting GIT_TEST_OPTS | ||
appropriately before running "make". | ||
|
||
--verbose:: | ||
This makes the test more verbose. Specifically, the | ||
command being run and their output if any are also | ||
output. | ||
|
||
--debug:: | ||
This may help the person who is developing a new test. | ||
It causes the command defined with test_debug to run. | ||
|
||
--immediate:: | ||
This causes the test to immediately exit upon the first | ||
failed test. | ||
|
||
--long-tests:: | ||
This causes additional long-running tests to be run (where | ||
available), for more exhaustive testing. | ||
|
||
--tee:: | ||
In addition to printing the test output to the terminal, | ||
write it to files named 't/test-results/$TEST_NAME.out'. | ||
As the names depend on the tests' file names, it is safe to | ||
run the tests with this option in parallel. | ||
|
||
Skipping Tests | ||
-------------- | ||
|
||
In some environments, certain tests have no way of succeeding | ||
due to platform limitation, such as lack of 'unzip' program, or | ||
filesystem that do not allow arbitrary sequence of non-NUL bytes | ||
as pathnames. | ||
|
||
You should be able to say something like | ||
|
||
$ SKIP_TESTS=t0000.2 sh ./t0000-config.sh | ||
|
||
and even: | ||
|
||
$ SKIP_TESTS='t[0-4]??? t91?? t9200.8' make | ||
|
||
to omit such tests. The value of the environment variable is a | ||
SP separated list of patterns that tells which tests to skip, | ||
and either can match the "t[0-9]{4}" part to skip the whole | ||
test, or t[0-9]{4} followed by ".$number" to say which | ||
particular test to skip. | ||
|
||
Note that some tests in the existing test suite rely on previous | ||
test item, so you cannot arbitrarily disable one and expect the | ||
remainder of test to check what the test originally was intended | ||
to check. | ||
|
||
|
||
Naming Tests | ||
------------ | ||
|
||
The test files are named as: | ||
|
||
tNNNN-commandname-details.sh | ||
|
||
where N is a decimal digit. | ||
|
||
First digit tells the family: | ||
|
||
0 - the absolute basics and global stuff | ||
1 - basic every-day usage | ||
2 - add ins | ||
|
||
Second digit tells the particular command we are testing. | ||
|
||
Third digit (optionally) tells the particular switch or group of switches | ||
we are testing. | ||
|
||
If you create files under tests/ directory (i.e. here) that is not | ||
the top-level test script, never name the file to match the above | ||
pattern. The Makefile here considers all such files as the | ||
top-level test script and tries to run all of them. A care is | ||
especially needed if you are creating a common test library | ||
file, similar to test-lib.sh, because such a library file may | ||
not be suitable for standalone execution. | ||
|
||
|
||
Writing Tests | ||
------------- | ||
|
||
The test script is written as a shell script. It should start | ||
with the standard "#!/bin/sh" with copyright notices, and an | ||
assignment to variable 'test_description', like this: | ||
|
||
#!/bin/sh | ||
# | ||
# Copyright (c) 2005 Junio C Hamano | ||
# | ||
|
||
test_description='xxx test (option --frotz) | ||
|
||
This test registers the following structure in the cache | ||
and tries to run git-ls-files with option --frotz.' | ||
|
||
|
||
Source 'test-lib.sh' | ||
-------------------- | ||
|
||
After assigning test_description, the test script should source | ||
test-lib.sh like this: | ||
|
||
. ./test-lib.sh | ||
|
||
This test harness library does the following things: | ||
|
||
- If the script is invoked with command line argument --help | ||
(or -h), it shows the test_description and exits. | ||
|
||
- Creates an empty test directory with an empty todo file | ||
database and chdir(2) into it. This directory is 't/trash directory' | ||
if you must know, but I do not think you care. | ||
|
||
- Defines standard test helper functions for your scripts to | ||
use. These functions are designed to make all scripts behave | ||
consistently when command line arguments --verbose (or -v), | ||
--debug (or -d), and --immediate (or -i) is given. | ||
|
||
|
||
End with test_done | ||
------------------ | ||
|
||
Your script will be a sequence of tests, using helper functions | ||
from the test harness library. At the end of the script, call | ||
'test_done'. | ||
|
||
|
||
Test harness library | ||
-------------------- | ||
|
||
There are a handful helper functions defined in the test harness | ||
library for your script to use. | ||
|
||
- test_expect_success <message> <script> | ||
|
||
This takes two strings as parameter, and evaluates the | ||
<script>. If it yields success, test is considered | ||
successful. <message> should state what it is testing. | ||
|
||
Example: | ||
|
||
test_expect_success \ | ||
'git-write-tree should be able to write an empty tree.' \ | ||
'tree=$(git-write-tree)' | ||
|
||
- test_expect_failure <message> <script> | ||
|
||
This is NOT the opposite of test_expect_success, but is used | ||
to mark a test that demonstrates a known breakage. Unlike | ||
the usual test_expect_success tests, which say "ok" on | ||
success and "FAIL" on failure, this will say "FIXED" on | ||
success and "still broken" on failure. Failures from these | ||
tests won't cause -i (immediate) to stop. | ||
|
||
- test_debug <script> | ||
|
||
This takes a single argument, <script>, and evaluates it only | ||
when the test script is started with --debug command line | ||
argument. This is primarily meant for use during the | ||
development of a new test script. | ||
|
||
- test_done | ||
|
||
Your test script must have test_done at the end. Its purpose | ||
is to summarize successes and failures in the test script and | ||
exit with an appropriate error code. | ||
|
||
|
||
Credits | ||
------- | ||
|
||
This test framework was derived from the framework used by | ||
git itself, written originally by Junio Hamano and licensed | ||
for use under the GPL. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
#!/bin/sh | ||
|
||
fixed=0 | ||
success=0 | ||
failed=0 | ||
broken=0 | ||
total=0 | ||
|
||
for file | ||
do | ||
while read type value | ||
do | ||
case $type in | ||
'') | ||
continue ;; | ||
fixed) | ||
fixed=$(($fixed + $value)) ;; | ||
success) | ||
success=$(($success + $value)) ;; | ||
failed) | ||
failed=$(($failed + $value)) ;; | ||
broken) | ||
broken=$(($broken + $value)) ;; | ||
total) | ||
total=$(($total + $value)) ;; | ||
esac | ||
done <"$file" | ||
done | ||
|
||
printf "%-8s%d\n" fixed $fixed | ||
printf "%-8s%d\n" success $success | ||
printf "%-8s%d\n" failed $failed | ||
printf "%-8s%d\n" broken $broken | ||
printf "%-8s%d\n" total $total |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
#!/bin/sh | ||
|
||
test_description='todo.sh configuration file location | ||
This test just makes sure that todo.sh can find its | ||
config files in the default locations and take arguments | ||
to find it somewhere else. | ||
' | ||
. ./test-lib.sh | ||
|
||
# Remove the pre-created todo.cfg to test behavior in its absence | ||
rm -f todo.cfg | ||
echo "Fatal error: Cannot read configuration file $HOME/todo.cfg" > expect | ||
test_expect_success 'no config file' ' | ||
todo.sh > output 2>&1 || test_cmp expect output | ||
' | ||
|
||
# All the below tests will output the usage message. | ||
cat > expect << EOF | ||
Usage: todo.sh [-fhpantvV] [-d todo_config] action [task_number] [task_description] | ||
Try 'todo.sh -h' for more information. | ||
EOF | ||
|
||
cat > test.cfg << EOF | ||
export TODO_DIR=. | ||
export TODO_FILE="$TODO_DIR/todo.txt" | ||
export DONE_FILE="$TODO_DIR/done.txt" | ||
export REPORT_FILE="$TODO_DIR/report.txt" | ||
export TMP_FILE="$TODO_DIR/todo.tmp" | ||
touch used_config | ||
EOF | ||
|
||
rm -f used_config | ||
test_expect_success 'config file (default location 1)' ' | ||
cp test.cfg todo.cfg | ||
todo.sh > output; | ||
test_cmp expect output && test -f used_config && | ||
rm -f todo.cfg | ||
' | ||
|
||
rm -f used_config | ||
test_expect_success 'config file (default location 2)' ' | ||
cp test.cfg .todo.cfg | ||
todo.sh > output; | ||
test_cmp expect output && test -f used_config && | ||
rm -f .todo.cfg | ||
' | ||
|
||
rm -f used_config | ||
test_expect_success 'config file (command line)' ' | ||
todo.sh -d test.cfg > output; | ||
test_cmp expect output && test -f used_config | ||
' | ||
|
||
rm -f used_config | ||
test_expect_success 'config file (env variable)' ' | ||
TODOTXT_CFG_FILE=test.cfg todo.sh > output; | ||
test_cmp expect output && test -f used_config | ||
' | ||
|
||
test_done |
Oops, something went wrong.