Problems with encoding (latin1 vs. UTF-8) when running devtools:test() #1306
UTF-8 encoded files with non-ASCII strings are not correctly represented when running devtools::test()
Description of issue
I am writing a package where strings containing non-ASCII characters need to be sent to a SQL server through RODBC.
By saving a query string with non-ASCII characters and running this in RStudio, R-terminal or using source from RStudio I am able to get the correct result.
However if I use devtools:test() my code fails since it seems that it does not source the functions using the correct encoding.
I have made a minimal example test functions (not using the SQL server). Here is the relevant code:
Running this code in RStudio (using Ctrl-R) returns:
Pressing the source button in RStudio runs:
From this I conclude that the files are saved as UTF-8 and should therefore be sourced as UTF-8 (also confirmed using external program and by inspecting the RStudio settings).
If I save this script in a testthat folder in an otherwise empty package and run
I get the following result:
This means that devtools is sourcing the files as latin1 instead of the UTF-8 files that they are. This happens irrespective of the
I have attempted to change to a UTF-8 encoding for my locale using
When using devtools::test() the encoding is not properly handled (at least on Windows). It is not possible to change to a UTF-8 encoding for the locale on Windows (as far as I can tell), even if this is the suggested encoding to use.
I would prefer solution 1
The text was updated successfully, but these errors were encountered:
I traced the calling functions from
and finally to
This uses readLines to get the lines from the test-R-file. I can confirm that using:
reads the file correctly.
So propagating the encoding to this function would solve the problem.
@hansharhoff You seem to have a good grip on the problem and solution, do you want to try submitting pull requests to devtools and testthat with the above fixes? See https://github.com/hadley/devtools/blob/master/CONTRIBUTING.md#pull-requests for information on contributing to devtools.
…ompatability on Windows (see e.g. r-lib/devtools#1306)
I have made two pull requests in regards to this issue. They should fix the issue without disturbing any functionality. I also to add a test to test the filter parameter of test(), but I did not manage to make a consistent test since the indirection of pointing at another set of tests confused me.
I also did not add a test of the encoding issue, in part because it only occurs on some platforms.
…ompatability on Windows (see e.g. r-lib/devtools#1306) Add NEWS.md line about the change.
* Add encoding parameter to source_file and test_file to ensure UTF-8 compatability on Windows (see e.g. r-lib/devtools#1306) * Add encoding parameter to source_file and test_file to ensure UTF-8 compatability on Windows (see e.g. r-lib/devtools#1306) Add NEWS.md line about the change. * Simplify documentation of encoding parameter. Remove ... from call/definition of test_files