Encoding fixes

[JanMarvin]

On non unicode systems, encoded strings must be passed through Rcpp::String to avoid broken encoding. On unicode systems there is no issue.

Our previous attempt was not sufficient, this should be fixed now and the code now also a bit cleaner. Previously we checked our R locale and passed that along.

Obviously handling non unicode encoding is a major pita, but non unicode R and OSes will likely be around for a long time. We should also add some non unicode CI.

[JanMarvin]

The failing R 4.1 check complicates things a bit to much 😆 I'm not aware if it should work in whatever windows locale is selected. Maybe it works fine, because these characters are not available in the default "C" locale?

[JanMarvin]

hmmm, that does not work as expected :(

[JanMarvin]

Could you please have a look @jmbarbone ? Obviously I'm missing something here. It works as intended (that much I was able to confirm with a German locale R 4.2), I'm just unable to make the test work.

[JanMarvin]

Okay, the remaining error either seems to have nothing to do with us (something in testthats mclapply), or at least it only occurs in x32-R. I can live with that. Might poke it a few more times tomorrow, but if it cannot be fixed, I wont hesitate merging this either way and disable the oldrel Windows-CI. Somehow doubt that anybody was working on 32-bit R on Windows in the last decade.

[JanMarvin]

After a restful sleep, I realized that I don't want to invest any more time in this problem and that it won't be an obstacle to the encoding PR I care about. We may need to investigate this once we roll out to CRAN. We may have to disable some testing on Windows. But right now there is a lot more to do and multiarch builds on Windows are something I am not that interested in.

[jmbarbone]

Noting that #244 is related to the test failures from this PR

[JanMarvin]

Thanks, still strange though. The x64 tests finished and locally I didn't recall issues either. There isn't anything I would suspect in this pull request either.

[JanMarvin]

Wasted another 2-3 hours on this, but to no avail. I dislike Windows for a reason and R on Windows makes no exception :)

What I tried:

• disable the newly introduced tests
• test in 32 bit and 64 bit
• test with R 4.1.0 and R 4.1.2
• test with devtools::check() in the R console (this creates strange warnings regarding exit, abort, and printf
• couldn't make R CMD check work in the CMD 😭
• created Makevars.in, Makevars.win and Makevars.urct

What I will try next:

• forget that the issue exists

[jmbarbone]

I tried playing around with a few things but couldn't get any error to pop up. Maybe not as comprehensive as you've checked.

forget that the issue exists

So backlog then? We can wait until it pops up again.

[JanMarvin]

Not the ideal way to proceed, but I have no other idea. At least it works with x64 and on R 4.2+