Permalink
Browse files

doc updates

  • Loading branch information...
simonmichael committed Jan 21, 2011
1 parent 8eefac6 commit d80c0f2f5899045a1a4c4a33e3af0d0f78cdc320
Showing with 58 additions and 85 deletions.
  1. +1 −0 .authorspellings
  2. +32 −25 shelltest.hs
  3. +25 −60 shelltestrunner.cabal
@@ -1 +1,2 @@
Bernie Pope <bjpope@unimelb.edu.au>, bjpop@unimelb.edu.au
John MacFarlane <jgm@berkeley.edu>
@@ -84,37 +84,44 @@ argmodes = [
,testpaths = def &= CmdArgs.args & typ "TESTFILES|TESTDIRS" & text "test files or directories"
,otheropts = def &= unknownFlags & explicit & typ "OTHER FLAGS" & text "any other flags are passed to test runner"
}) &= helpSuffix [
"A test file contains one or more shell tests, which look like this:"
"Test file format:"
,""
," # optional comment lines"
," a one-line shell command to be tested"
," # optional comment"
," a shell command line"
," <<<"
," stdin lines"
," >>> [/regexp to match in stdout/]"
," [or expected stdout lines"
," >>>2 [/regexp to match in stderr/]"
," [or expected stderr lines]"
," >>>= expected exit status or /regexp/"
," 0 or more lines of input"
," >>>"
," 0 or more lines of expected output (or a /regexp/ on the above line)"
," >>>2"
," 0 or more lines of expected stderr output (or a /regexp/ on the above line)"
," >>>= expected numeric exit status or a /regexp/"
,""
,"The command line is required; all other fields are optional."
,"The expected stdout (>>>) and expected stderr (>>>2) fields can have either"
,"a regular expression match pattern, in which case the test passes if the"
,"output is matched, or 0 or more data lines, in which case the output"
,"must match these exactly. The expected exit status (>>>=) field can have"
,"either a numeric exit code or a /regexp/. A ! preceding a /regexp/ or exit"
,"code negates the match."
,"The command line is required. It runs in your current directory, or with"
,"--execdir, in the test file's directory. The other fields are optional, except"
,">>>= may be required as a delimiter if there are multiple tests in the file."
,""
,"By default there is an implicit test for exit status=0, but no implicit test"
,"for stdout or stderr. You can change this with -i/--implicit-tests."
,">>> and >>>2 can be followed by 0 or more lines of expected output,"
,"in which case the test passes if the command's output matches exactly."
,"Or, they can have a regular expression in forward slashes on the same line,"
,"in which case the test passes if any part of the command's output is matched."
,"Regexp syntax is regex-tdfa's. A preceding ! negates the match."
,""
,"The command runs in your current directory unless you use --execdir."
,"You can use --with/-w to replace the first word of command lines"
,"(everything up to the first space) with something else, eg to test a"
,"different version of your program. To prevent this, start the command line"
,"with a space."
,">>>= is followed by a numeric exit code or a /regexp/ on the same line;"
,"the test passes if the command's exit status is matched. A preceding ! negates"
,"the match."
,""
,"-i/--implicit-tests controls which fields should have default tests applied"
,"when omitted. By default there is an implicit test for exit status=0 but"
,"none for stdout or stderr."
,""
,"--with/-w replaces the first word of each test's command line (up to the first"
,"space) with something else - useful to test different versions of a program."
,"Command lines beginning with a space are left alone."
,""
,"Unrecognised options are passed to test-framework (use no space between"
,"flags and values.) To run just the third test, try -t3."
,"To run tests in parallel for a big speedup, try -j8."
,""
,"Any unrecognised options will be passed through to test-framework's runner."
,"You may be able to get a big speedup by running tests in parallel: try -j8."
]
]

@@ -4,74 +4,39 @@ version: 0.9.98
category: Testing
synopsis: A tool for testing command-line programs.
description:
This tool aims to help with repeatable testing of a command-line program
(or any shell command), via declarative \"shell tests\" which are easier
to write than procedural shell scripts. Tests are defined in one or more
test files, which typically have the .test suffix and live in a tests/
subdirectory. Each test specifies a command line, optional standard input,
and expected standard output, error output and/or exit status. Tests can
be run in parallel for greater speed. shelltestrunner was inspired by the
tests in John Wiegley's ledger project.
A tool for doing repeatable testing of command-line programs (or any shell command),
using declarative \"shell tests\" which specify
a command, optional standard input, and expected standard output, error output and/or exit status.
.
Tests are defined in one or more test files, which typically have the @.test@ suffix
and often live in a @tests/@ subdirectory.
Tests can be run selectively or in parallel for greater speed.
shelltestrunner uses test-framework and was inspired by the tests in John Wiegley's ledger project.
.
Compatibility: should work on microsoft windows as well as unix; not well
tested on windows. Should build with ghc 6.10; for unicode support,
requires ghc 6.12.
Compatibility: tested on unix, should work on microsoft windows.
Should build with ghc 6.10; supports unicode with ghc 6.12 or greater.
.
Here's the basic format of a shell test (one or more per file):
.
Usage:
@
# optional comment
a shell command line
<<<
0 or more lines of input
\>>>
0 or more lines of expected output (or a \/regexp\/ on the above line)
\>>>2
0 or more lines of expected stderr output (or a \/regexp\/ on the above line)
\>>>= expected numeric exit status or a \/regexp\/
@
.
> shelltest [FLAG] [TESTFILES|TESTDIRS]
>
> -? --help[=FORMAT] Show usage information (optional format)
> -V --version Show version information
> -v --verbose Higher verbosity
> -q --quiet Lower verbosity
> -d --debug show debug messages
> --debug-parse show parsing debug messages and stop
> -c --color display with ANSI color codes
> --execdir run tests in same directory as test file
> --extension=EXT extension of test files when dirs specified (default=.test)
> -i --implicit=none|exit|all provide implicit tests (default=exit)
> -w --with=EXECUTABLE alternate executable, replaces the first word of test commands
> =OTHER FLAGS any other flags are passed to test runner
>
> A test file contains one or more shell tests, which look like this:
>
> # optional comment lines
> a one-line shell command to be tested
> <<<
> stdin lines
> >>> [/regexp to match in stdout/]
> [or expected stdout lines
> >>>2 [/regexp to match in stderr/]
> [or expected stderr lines]
> >>>= expected exit status or /regexp/
>
> The command line is required; all other fields are optional.
> The expected stdout (>>>) and expected stderr (>>>2) fields can have either
> a regular expression match pattern, in which case the test passes if the
> output is matched, or 0 or more data lines, in which case the output
> must match these exactly. The expected exit status (>>>=) field can have
> either a numeric exit code or a /regexp/. A ! preceding a /regexp/ or exit
> code negates the match. The regular expression syntax is that of the
> pcre-light library with the dotall flag.
>
> By default there is an implicit test for exit status=0, but no implicit test
> for stdout or stderr. You can change this with -i/--implicit-tests.
>
> The command runs in your current directory unless you use --execdir.
> You can use --with/-w to replace the first word of command lines
> (everything up to the first space) with something else, eg to test a
> different version of your program. To prevent this, start the command line
> with a space.
>
> Any unrecognised options will be passed through to test-framework's runner.
> You may be able to get a big speedup by running tests in parallel: try -j8.
See @shelltest --help@ for more details.

license: GPL
license-file: LICENSE
author: Simon Michael <simon@joyful.com>
maintainer: Simon Michael <simon@joyful.com>
homepage: http://joyful.com/darcsweb/darcsweb.cgi?r=shelltestrunner
homepage: http://hackage.haskell.org/package/shelltestrunner
bug-reports: mailto:simon@joyful.com
stability: beta
tested-with: GHC==6.12

0 comments on commit d80c0f2

Please sign in to comment.