Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
doc
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
sis
 
 
 
 

README.mdwn

Sisyphus

A compiler regression testing tool.

How to Use

We assume you have a directory full of test programs. To configure sisyphus for your test programs, you have to write a plugin, unless our included plugins are sufficient. Put the plugin into the program directory as well, sisyphus will find it automatically.

Now let sisyphus run. It will test all your programs according to your plugin.

Why

A simple shell script can do the job just as well, does it? Dependending on your requirements that might be true. We also started like that. Then the script got more and more complicated until it was rewritten in Python. A few more extensions and rewrites and sisyphus emerged. A few features your shell script might not have:

  • An expectation file, possibly from a URL, to distuingish between expected and unexpected failures. This means you notice, when you accidentally fix a long standing bug.
  • Parallel execution of test, but limited to the number of cores on the machine.
  • Options for cross-compilation, qemu, valgrind, and performance checks.
  • Allows to retry to accomodate tests which sometimes fail.

Concepts

A single test is represented by a file. Test factories generate a test for each file. Those factories also associate a list of steps to perform with the file. Each step in turn has a list of checks associated.

There might be additional files (reference output, header files, etc), which should be ignored by the factories.

Licence

MIT, see LICENCE file for full text

About

Test runner for file-based regression testing

Topics

Resources

License

Releases

No releases published

Packages

No packages published

Languages