This is a COMP 412 student made test suite for lab1, lab2, and probably lab3!
Implementation contributors:
- Bill Qian (@billqian06)
- Adam "Zawie" Zawierucha (@zawie)
- Alison Qiu (@alisonqiu)
- Shreyas Minocha
- Stephen Xu
Simply clone this repo into your project directory and execute the bash script 'test' located in the folder:
git clone https://github.com/zawie/zill412
./zill412/lab1 >> test_output.log
./zill412/lab2 >> test_output.log
./zill412/lab3 >> test_output.log
The file piping (>> test_output.log
) is optional, but the output is very long.
These commands should be run in the root of your project directory. This needs to be ran on the clear system.
If your executable is not located in the root of the directory, you can call a different shell script:
./zill412/test lab1 <path 412fe>
./zill412/test lab2 <path 412alloc>
./zill412/test lab3 <path 412schedule>
It loops through a set of ILOC of files and compares your implementation's output to the references output. It only expects that you flag the erroneous lines at least once. Moreover, it will also indicate failure if you flag an correct line as erroneous.
runner.py
(TIME_LIMIT
). Most of the test files (excluding the scalability ones) should run very vast. If you keep getting timed out, it is likely because your implementation has entered a infinite loop.
This checks if you emit a success message when the reference solution emits a success message. A line is considered a success message if it matches the following pattern:
(^|\s)(((S|s)ucce((ss)|(eded)))|(SUCCE((SS)|(EDED))))
(basically any line containing the word "success" or "succeeded" but not "unsuccessful")
//TODO: Write this section
//TODO: Write this section
If you're using the original script, you might run into this issue: "ModuleNotFoundError: No module named 'commands'". To fix this, you can either use the updated script from this commit, or use the following command:
python2 <path runner.py> <lab number> <path 412fe/412alloc/412schedule>>>test_output.log
For example:
python2 ./zill412/runner.py lab3 ./schedule>>test_output.log
The biggest thing this needs is MORE ILOC FILES. Please chip in and contribute some test files in the /blocks
directory.
You don't even need to indicate which lines are failure as this script automatically figures out which lines are bad via the reference solution!