Testing

Several sets of unit tests are distributed with Logtalk. These include the tests in the tests, tools, examples, and contributions subdirectories. You can run these unit tests or your own unit tests by using the provided scripts/logtalk_tester.sh shell script.

Note that some tests require specific backend Prolog compiler features such as constraints, tabling, and threads. These tests are skipped when using backend Prolog compilers without native support for those features.

For information about the unit test framework, see its documentation.

Automating running unit tests on POSIX systems

In a POSIX system, you can run the provided unit tests by typing:

$ cd $LOGTALKUSER/tests/logtalk
$ logtalk_tester -p <back-end Prolog compiler>
...
$ cd ../../tools
$ logtalk_tester -p <back-end Prolog compiler>
...
$ cd ../examples
$ logtalk_tester -p <back-end Prolog compiler>
...
$ cd ../contributions
$ logtalk_tester -p <back-end Prolog compiler>
...

If you didn't use one of the provided Logtalk installers or the installation script, you may need to type logtalk_tester.sh instead of just logtalk_tester.

To identifiers for the supported back-end Prolog compilers can be listed by typing:

$ logtalk_tester -h
...

Logtalk source code can be compiled in three different modes: optimal, normal, and debug. The logtalk_tester accepts an option, -m, to set the mode to be used to run the unit tests. Be aware, however, that running the unit tests in debug mode will be slower compared with the other two modes. Also, running the tests in optimal mode may flag errors not reported when running in normal mode due to e.g. the more extended use of static binding.

Note that, when running the unit tests using stable Logtalk releases, failed tests usually result from bugs in the backend Prolog compilers or from their lack of compliance with official and de facto standards. The most common issues are non-standard exception handling and syntax errors due to parser bugs. These issues can only be fixed by the developers of those Prolog compilers.

Testing backend Prolog compilers for standards conformance

Starting with Logtalk 3.00.0-rc4, a set of unit tests for testing backend Prolog compilers for conformance with official and de facto standards is also provided. You can run them by typing:

$ cd $LOGTALKUSER/tests/prolog
$ logtalk_tester -p <back-end Prolog compiler>
...

Running these tests is advisable when developing portable Logtalk applications to ensure that all targeted backend Prolog compilers work as expected and that relevant differences between systems are accounted for. For more details about these tests see their documentation

Automating running unit tests on Windows systems

The logtalk_tester.sh script can also be used on Windows operating-systems by installing Git for Windows, which includes a bash shell implementation. After installation, you can start the bash shell by selecting Git Bash from the context menu. You will also need to add the $LOGTALKHOME/scripts and $LOGTALKHOME/integration directories plus the backend Prolog compiler executable directories to the system path environment variable. For example, assuming that you will be using YAP as backend Prolog compiler, the contents of your ~/.profile file would contain something like:

# YAP
export PATH="/C/Program Files/Yap64/bin":$PATH
# Logtalk
export PATH="$LOGTALKHOME/scripts":"$LOGTALKHOME/integration":$PATH

When calling the scripts, you will need to use the .sh extension (e.g. use yaplgt.sh instead of simply yaplgt).

Using a continuous integration server

The Logtalk unit test framework supports both TAP and xUnit output formats. Most CI servers support one or both formats for reporting and summarizing test results. In addition, the logtalk_tester script returns a non-zero exit value in case of failed tests, accepts user-defined arguments that are passed to application being tested, and traverses directories recursively looking for test sets to execute.

As an example, a CI server build script could contain:

# change directory to the test sets root directory
cd tests
# run all tests using the SWI-Prolog Logtalk pack, results in TAP
# format, and passing arguments foo, bar, and baz to the tests
logtalk_tester -p swipack -tap -- foo bar baz

By configuring the CI server TAP support to look for tap_report.txt files recursively inside the tests directory, the build report will summarize and list the test results. By making the build script fail when the logtalk_tester script returns a non-zero value, the build will be marked as failed when there are failed tests. Consult your CI server documentation for details.