Added test suite, make check and openblas support

[pjotrp]

This is a simple testsuite which is easy to install. We can always change to something else later.

Added OpenBLAS support in makefile, though it does not make a difference in performance on my laptop.

[pcarbo]

This looks great @pjotrp. I'd like to make sure the tests work on my end before merging the pull request. Is there anything else I need to do besides running run_tests.sh after building the gemma executable?

[pjotrp]

You need to download shunit2 using the link in run_tests.sh into the source tree. Then it should just pick it up.

[pcarbo]

@pjotrp Sounds good, I will take a look at your tests on Thursday, after my workshop.

[pcarbo]

@pjotrp The link you provided for shunit2 doesn't work for me; I get a "404 - file not found" error on Dropbox. Maybe you need to create a shared link in Dropbox? (Or create a shunit2 github repo?)

[pjotrp]

it is on https://github.com/genenetwork/shunit2

[pcarbo]

@pjotrp Should all the tests pass? This is the output I got from running ./run_tests.sh:

#
# Test report
#
tests passed: 12
tests failed: 1
tests total:  13
success rate: 92%

I don't see a log file. Is there a way to record the output of the tests? How can I find out which of the tests failed?

[pjotrp]

I have committed some instructions.

run_tests.sh and make check do:

check: all
        cd test && ./test_suite.sh | tee ../test.log
        grep -q 'success rate: 100%' test.log

I.e. stdout get put in test.log, errors get shown on stderr.

My sample errors are:

make check
cd test && ./test_suite.sh | tee ../test.log
#
# Performing tests
#
testCenteredRelatednessMatrixK
ASSERT:expected:<0> but was:<127>
## total computation time = 0.35415 min 
ASSERT:expected:<29.691> but was:<24.9799>
testUnivariateLinearMixedModel
ASSERT:expected:<0> but was:<127>
## total computation time = 0.489185 min 
testMultivariateLinearMixedModel
ASSERT:expected:<0> but was:<127>
## total computation time = 0.537555 min 

#
# Test report
#
tests passed: 9
tests failed: 4
tests total:  13
success rate: 69%
grep -q 'success rate: 100%' test.log
Makefile:110: recipe for target 'check' failed
make: *** [check] Error 1

so the test name is there and the assertion. Send output to me, so we can fix it.

[pcarbo]

@pjotrp In the log, don't you want to save both standard output and standard error? i.e., do something like ./test_suite.sh >> test.log 2>&1? Attached is the output from running this command. The only test that fails is testCenteredRelatednessMatrixK.

Note that I am seeing ^M characters is my output (which I know you make an effort to eliminate).

test.log.gz

[pjotrp]

Sure, make it work for you. As long as the tests run and can be fixed ;) Easier than trying to go back and forth here. I suspect the failing test is the same as #55

[pcarbo]

@pjotrp Sure, I can do that. Before I do that, can you please add steps in each of the tests to clean up the output files? This should resolve the problem you observed earlier, and should make the tests less error-prone. In other words, gemma should be outputting to an empty directory each time.