tests run in R session, but "R CMD check" fails

[unDocUMeantIt]

tests for a package (see below) fail when run by "R CMD check", but if i run them from an R session, everything is fine. the package tested creates some files in a tempdir and i want to check for their existance.

i've noticed errors like
"Error in file(filename, "r", encoding = encoding) : cannot open the connection
Calls: local ... eval.parent -> eval -> eval -> eval -> eval -> source -> file
In addition: Warning message:
In file(filename, "r", encoding = encoding): cannot open file 'startup.Rs': No such file or directory"

my package functions use some setwd() calls. when i create a dummy "startup.Rs" file in the directory the functions jump to, these errors disappear. i have no clue what this file is supposed to do or why it's only missing when "R CMD check" is running the tests. anyway, the tests still fail after all.

in addition, testthat didn't load all dependencies, i had do manually add a require() call to the test suite. here's the package, if you want to try to replicate the problem: http://reaktanz.de/stuff/R/roxyPackage_0.03-11.tar.gz

[hadley]

Hmmm, calls to setwd() are usually a bad idea. The tests are probably being run in a different environment that what you expect - I've only ever been able to debug problems by painfully adding lots of print statements :(

[yoni]

I'm seeing similar issues, though I'm not calling setwd(). I'm able to run my test suite using testthat:::test_dir and devtools:::test(fresh=TRUE), but not devtools:::test() (i.e. with fresh=FALSE) or devtools:::check().

I'll dig deeper into the issue, but figured I'd share the strange behavior while I work through this.

[lhsego]

I'm finding the same issue, using R 3.2.2 on OSX Mavericks. Per the instructions in the README.md, I put the following into tests/testthat.R with test files in tests/testthat:

library(testthat)
library(qFeature)
test_check("qFeature")

Upon running R CMD CHECK qFeature, it hangs when it starts the tests. Breaking it with Ctrl-C gives the following:

    Running ‘testthat.R’^C
 ERROR
Running the tests in ‘tests/testthat.R’ failed.
Last 13 lines of output:
  > library(testthat)
  > library(qFeature)
  > 
  > test_check("qFeature")
  Error in file(filename, "r", encoding = encoding) : 
    cannot open the connection
  Calls: local ... eval.parent -> eval -> eval -> eval -> eval -> source -> file
  In addition: Warning message:
  In file(filename, "r", encoding = encoding) :
    cannot open file 'startup.Rs': No such file or directory
  Execution halted

However, all things being equal, testing in the R console works fine:

 > library(testthat)
 > test_package("qFeature")
ddply_getFeatures() - Output Values : .
discFeatures() - Output Values : .........
fitQ() - Summary Values : Loading required package: qFeature
................................
fitQ() - Linear Output Values : ....
fitQ() - Quadratic Output Values : ....
getFeatures() - Output Values : .

DONE 

[lhsego]

PS: devtools::check("~/path_to_package/qFeature") produces the same behavior as running R CMD CHECK: it hangs, and once broken with Cntrl-C, the same error appears.

[lhsego]

Found a workaround

This issue is related to these others: #144 and zoonproject/zoon#150.

A workaround is here: luckyrandom/cmaker@b85813a

Simply add Sys.setenv("R_TESTS" = "") as the first line in tests/testthat.R