Difference in interactive versus package checking

[BartJanvanRossum]

Hi Mark,

Not quite sure if this is a bug in tinytest or just wrong expectations from my side, but since it took me quite some time to figure out what was going on, I'm posting it anyway.

The test below runs fine interactively and also when I use run_test_file or run_test_dir. However if I use R CMD check to check the package it is in, it fails. This has to do with LC_COLLATE. In my local R installation that is "English_United States.1252" and that is used by the interactive functions. However when using R CMD check LC_COLLATE is set to "C", which affects sorting.

expect_equal(sort(c("a", "B")), c("a", "B"))

These different settings cause tests to pass when run interactively and fail when run in R CMD check, which was unexpected for me. Is this the intended behavior?

For what it's worth, for comparison I ran the same test in testthat, and there it fails both interactively and in R CMD check.

[markvanderloo]

Hi Bart-Jan,

Yes, this is an interaction between local tests and R CMD check settings. You can set any environment variable of your choice in any test runner. For example, you can do

test_package("mypkg", set_env=list(LC_COLLATE="C"))

and all tests will run with LC_COLLATE set. And this is probably what you want because that is what R CMD check does and also what CRAN does.

Now, you could also set LC_COLLATE="English_United States.1252", but this is specific to Windows. That can be circumvented by first detecting the OS (Sys.info()) and choosing the appropriate annotation. But even then you cannot be sure that this will be used on CRAN, because in order for those settings to take effect, the machine running the tests must have the corresponding collation charts installed.

testthat probably sets LC_COLLATE="C" for all tests. But that may be undesirable for packages that never go to CRAN. I might make it default with an option to switch it off.

[BartJanvanRossum]

Thanks for you reply.

It seems I definitely don't want the windows locale for my tests, so using LC_COLLATE="C" seems the way to go.

Unfortunately the solution you propose doesn't work. The tests still fail. I had to put this in the relevant test files to make them run without fails:

Sys.setlocale("LC_COLLATE", "C")

This is a bit ugly, but it has the added advantage the run_test_file() is working as well now for that file without having to add extra parameters to the functions.

My guess for why it doesn't work is that the Locale isn't really an environment variable, it doesn't show up in Sys.getenv() for example.