running tests after installation

[wwood]

Hi,

I'm attempting to package vegan for GNU Guix. It seems all it all works, but to be more confident it would be good to run the test code, to see that it compiles correctly etc etc. I see that the tests are not included in the CRAN release, and the 2.3 branch has seen many commits and much time since the tests were deleted (I believe they were deleted in c67ae3d).

Is it possible to run tests on released code?

Thanks,
ben

[jarioksa]

No. The tests were removed. The tests are only available in the master branch (to be release 2.4-0).

[gavinsimpson]

Now that there are reasonable ways to skip tests on CRAN and on various continuous integration services that we utilise, we could reconsider this and have tests (if appropriate) in the released version too.

I'm not suggesting we switch to testthat or anything but we can borrow their ideas to implement the check for one or more environmental variables (for CRAN, Travis-CI, and Appveyor) and just throw an if() clause around the entire set of tests.

[jarioksa]

If you (or anybody) knows how to do this, and have the time and patience to implement the change, then we can keep tests in the CRAN release of vegan_2.4-0. The plan so far is to remove tests from the cran-2.4 branch as soon as we make the release and the master becomes 2.5-0. I just want to avoid the test mess in CRAN.

[gavinsimpson]

I've only used it via testthat but I understand the general principle. I won't get chance to do this before the 2.4-0 release date though, but I will look to implement it in the master branch and then move the tests back to the 2.5-0 branch at some later point in time. Ideally the end result would be having no need to remove tests from a release branch.

[wwood]

OK, thanks. Are the tests that were deleted when 2.3 was branched off still applicable t 2.3-5?

[jarioksa]

The deleted tests may not be valid any longer. The tests have changed a lot in master during 2.4-0 development:

$ git diff --stat c67ae3df2^ tests/
 tests/Examples/vegan-Ex.Rout.save | 1159 +++++++++++++++++++++++++------------
 tests/oecosimu-tests.Rout.save    |  376 ++++++------
 tests/vegan-tests.R               |   32 +
 tests/vegan-tests.Rout.save       |  280 ++++++++-
 4 files changed, 1267 insertions(+), 580 deletions(-)

The Examples/vegan-Ex.Rout.save changed mostly because the examples in man/*.Rd changed, and because we have changed the functions and their output. The oecosimu-tests.Rout.save changed because "quasiswap" null model uses now thin. The tests in vegan-tests.R were changed because we added dbrda function, and its reference output .Rout.save because of this and associated changes in capscale plus changes in random start strategies in metaMDS.

Most of these changes were not ported to the cran-2.3 branch so that you may be able to use oecosimu-tests and vegan-tests of c67ae3d^, but Examples/vegan-Ex.Rout.save will probably have some changes.

The pain of maintaining two parallel branches of tests is another reason of dropping them from the release branch.

[jarioksa]

We are planning to release vegan 2.4-0 within a couple of days and hard working on 2.3-5 may not be rewarding. The plan is to remove tests from the CRAN branch, but the current tests apply for the future dash-zero release.

[wwood]

OK, that sounds like a much better plan. Thanks. I'm pretty much through getting the package together minus tests, but I'll wait until 2.4 now.

One thing you may not be aware of while going through the packaging process. vegan requires permute at runtime but during testing permute requires vegan. This circular dependency makes testing permute hard, hard enough that I won't bother. I'm not sure if that is easily avoided?

[jarioksa]

We are aware of circular dependence between permute and vegan and we have solved the issue: the tests in permute are conditional on the availability of vegan (see if(require("vegan")){...} blocks in examples).

This was discussed in gavinsimpson/permute#4

[jarioksa]

vegan 2.4-0 was released today and it is available on CRAN. We removed the tests again from the release version, but the 2.4-0 release is identical to the current master branch and the same tests can be used.

[wwood]

Thanks for the heads up. I just pushed a vegan package to the master branch of Guix which reverts the test deletion patch.