Clone this wiki locally
Upgrading to the new test interface
If you wrote a package with test suites before Cabal made it easy, you're probably now seeing a message like this:
Warning: In the use of `runTests' (imported from Distribution.Simple, but defined in Distribution.Simple.UserHooks): Deprecated: "Please use the new testing interface instead!"
This guide will help you upgrade your test suites to use the new interface. The improved test interface eliminates the need for common boilerplate code and assures users of software quality by allowing Cabal to automatically run test suites and correctly interpret their results. Eventually, the test interface will be integrated with Hackage to make it easier for users to recognize high-quality libraries.
This guide will show the most common patterns from the old interface and demonstrate their equivalents in the new interface. Inevitably, some patterns will be missed; for that reason, contributions to this wiki are encouraged.
What Cabal expects
Cabal expects that each of your test suites compiles to a single executable that:
- Runs one or more test cases
- (Optionally) prints human-readable information about their results to the terminal
- Returns a non-zero exit code if the tests failed or zero otherwise.
Your test suites likely already fit this model. However, if you are
using an old version of
executable may not be returning the correct error code. Other test
libraries, such as
test-framework, handle this correctly; check the
documentation if you're unsure.
What Cabal does for you
So far, it sounds like business as usual. Why upgrade the test interface? Here's what Cabal will do for you:
- Conditional compilation: no more gymnastics with flags to enable and disable your test suites.
- Automatic testing: no need for user hooks or custom setup files.
- Automatic logging: Cabal can summarize test results, so there's no need to clutter up your terminal with output from successful tests.
As you can see, Cabal can manage the overhead of maintaining test suites for you.
How to upgrade
Remove user hooks
The old interface required you to implement user hooks telling
Cabal how to run your test suites. In turn, this required you to
use a custom setup script. All the information Cabal needs about
your test suites is now specified in the package description
.cabal) file, so you should remove the
runTests hook from your
If this was the only user hook you implemented, you can actually go back to using the default Setup.hs:
import Distribution.Simple main = defaultMain
Then, you can use the
Simple build type instead of
Custom. In your
.cabal file, the field
Don't do this if you are still using a custom
Setup.hs because you
have other user hooks!
Convert executable sections
Your test suite is likely included in your
.cabal file as an
executable section such as:
executable test-foo main-is: test-foo.hs build-depends: ... ...
Your executable may vary; in particular, it might have some
conditional blocks and a flag called something like
situation will be addressed shortly, but first, you must tell Cabal
that this executable is actually a test. The section above would
test-suite test-foo type: exitcode-stdio-1.0 main-is: test-foo.hs build-depends: ... ...
The only thing that changes is the name of the section and the
addition of the field type. Any build information field that is permitted in an
executable section is permitted here.
If you were using a flag, such as
tests, to control compilation of
your test suites, you should remove the flag and the conditional
blocks it controlled. The
test-suite sections should be written as
if tests are always enabled; Cabal will enable or disable them as
That's it! If you use the
cabal tool, you can install your test's
$ cabal install --enable-tests --only-dependencies
To test your package, just do
$ cabal test
cabal will automatically enable tests, rebuild your package
(if required) and run the test suites. If you need to configure
your package manually, you can also enable tests with
$ cabal configure --enable-tests