Add RCPP_USING_UTF8_ERROR_STRING macro to use UTF-8 encoding exception string in R.

[qinwf]

Hi, I posted on Rcpp mailing list last week about Rcpp exception with UTF-8 strings on Windows.

This PR adds RCPP_USING_UTF8_ERROR_STRING to use UTF-8 encoding exception string in R.

Here is an example for this on Windows.

By default, not using RCPP_USING_UTF8_ERROR_STRING,

sourceCpp(code = "
#include <Rcpp.h>

// [[Rcpp::export]]
void error_with_unicode(std::string input){
    Rcpp::stop(input);
}
")
res = try(error_with_unicode(enc2utf8("测试")))
#> Error : 娴嬭瘯
Encoding(res)
#> [1] "unknown"

Using RCPP_USING_UTF8_ERROR_STRING,

sourceCpp(code = "
#define RCPP_USING_UTF8_ERROR_STRING
#include <Rcpp.h>

// [[Rcpp::export]]
void error_with_unicode(std::string input){
    Rcpp::stop(input);
}
")
res = try(error_with_unicode(enc2utf8("测试")))
#> Error : 测试
Encoding(res)
#> [1] "UTF-8"

[eddelbuettel]

Thanks. The PR is nicely minimal, and even contains ChangeLog and NEWS. Tests would be nice too.

As the Contributing.md file suggests, we generally prefer discussion prior to submission but we'll try to review in due course.

[kevinushey]

I feel like we shouldn't even use this macro define and always create our strings with UTF-8 encoding. (Ie, we shouldn't use mkChar(); rather, always mkCharLenCE() with UTF-8 encoding as in this PR.

[eddelbuettel]

Not a bad idea. Any possible side effects? Sounds like I may need a rev.deps run for this.

[thirdwing]

I agree with Kevin. I will try to replace mkChar this weekend.

BTW, it seems the error fixed by this PR can only be reproduced on Windows.

[eddelbuettel]

Right on.

But ... didn't we once try this and failed with a different / older PR?

[ Goes off looking ... ]

What about #280 by @thirdwing ?

[thirdwing]

In the old PR, I try to convert strings into UTF-8 when passing strings from R to C++.

Now, this error happens when we throw strings from C++ to R...

A good solution is needed for encoding problems...

[kevinushey]

FWIW, this PR is a bit more targeted in that we only change the representation of error strings returned to R, not all Rcpp String objects (which may or may not be UTF-8 encoded). Because of that, I think this is safer than #280.

(Ie, this PR doesn't change any of the Rcpp internals; just how R error objects are constructed / thrown)

[eddelbuettel]

Yep, yep. And that we could indeed make unconditionally UTF-8.

[qinwf]

We can remove the macro and make it unconditionally UTF-8, and this may change error messages of some existing packages.

Or leave the macro as now, and developers will only be able to change the result with the macro in the new code.

[eddelbuettel]

That is a good point.

[eddelbuettel]

I think I'll merge as-is. We can still discuss

• actually setting the #define ourselves (either now or later) to make it a default
• revert the logic, ie have UTF-8 the default with an opt-out for packages that cannot/will not change

but the current suggestion is good as it is pretty minimal.

[kevinushey]

Sounds good to me.