pip install checkpy
Besides installing checkPy, you might want to download some tests along with it. Simply run checkPy with the
-d arg as follows:
checkpy -d YOUR_GITHUB_TESTS_URL
usage: checkpy [-h] [-m MODULE] [-d GITHUBLINK] [-clean] [file] checkPy: a simple python testing framework positional arguments: file name of file to be tested optional arguments: -h, --help show this help message and exit -m MODULE provide a module name or path to run all tests from the module, or target a module for a specific test -d GITHUBLINK download tests from a Github repository and exit -clean remove all tests from the tests folder and exit
To simply test a single file, call:
If you are unsure whether multiple tests exist with the same name, you can target a specific test by specifying its module:
checkpy YOUR_FILE_NAME -m YOUR_MODULE_NAME
If you want to test all files from a module within your current working directory, then this is the command for you:
checkpy -m YOUR_MODULE_NAME
- Support for ordering of tests
- Execution of tests can be made dependable on the outcome of other tests
- The test designer need not concern herself with exception handling and printing
- The full scope of Python is available when designing tests
- Full control over displayed information
- Support for importing modules without executing scripts that are not
if __name__ == "__main__"
- Support for overriding functions from imports in order to for instance prevent blocking function calls
- Support for grouping tests in modules, allowing the user to target tests from a specific module or run all tests in a module with a single command.
Tests in checkPy are collections of abstract methods that you as a test designer need to implement. A test may look something like the following:
0| @t.failed(exact) 1| @t.test(1) 2| def contains(test): 3| test.test = lambda : assertlib.contains(lib.outputOf(_fileName), "100") 4| test.description = lambda : "contains 100 in the output" 5| test.fail = lambda info : "the correct answer (100) cannot be found in the output"
From top to bottom:
- The decorator
failedon line 0 defines a precondition. The test
exactmust have failed for the following tests to execute.
- The decorator
teston line 1 prescribes that the following method creates a test with order number
1. Tests are executed in order, lowest first.
- The method definition on line 2 describes the name of the test
contains), and takes in an instance of
test.py. This instance is provided by the decorator
teston the previous line.
- On line 3 the
testmethod is bound to a lambda which describes the test that is to be executed. In this case asserting that the print output of
_fileNamecontains the number
_fileNameis a magic variable that refers to the to be tested source file. Besides resulting in a boolean indicating passing or failing the test, the test method may also return a message. This message can be used in other methods to provide valuable information to the user. In this case however, no message is provided.
- On line 4 the
descriptionmethod is bound to a lambda which when called produces a string message describing the intent of the test.
- On line 5 the
failmethod is bound to a lambda. This method is used to provide information that should be shown to the user in case the test fails. The method takes in a message (
info) which comes from the second returned value of the
testmethod. This message can be used to relay information found during execution of the test to the user.
Test methods are discovered in checkPy by filename. If one wants to test
foo.py, the corresponding test must be named
checkPy assumes that all methods in the test file are tests, as such one
should not use the
from ... import ... statement when importing
A test minimally consists of the following:
import check.test as t @t.test(0) def someTest(test): test.test = lambda : False test.description = lambda : "some description"
Here the method
someTest is marked as test by the decorator
test. The abstract methods
implemented as these are the only methods that necessarily require
implementation. For more information on tests and their abstract methods
you should refer to
test.py. Note that besides defining the
class and its abstract methods,
test.py also provides several
decorators for introducing test dependencies such as
When providing a concrete implementation for the test method one should
take a closer look at
provides a collection of useful functions to help implement tests. Most
outputOf. These provide the tester with
a function from the source file and the complete print output
getFunction makes checkPy evaluate only import
statements and code inside definitions of the to be tested file.
Effectively all other parts of code are wrapped by
if __name__ == "__main__" and thus ignored.
provides a collection of assertions that one may find usefull when
For inspiration inspect some existing collections of tests like the tests for progNS2016.
CheckPy can download tests directly from Github repos.
The only requirement is that a folder called
tests exists within the repo that contains only tests and folders (which checkpy treats as modules).
Simply call checkPy with the optional
-d argument and pass your github repo url.
Tests will then be automatically downloaded and installed.