-
-
Notifications
You must be signed in to change notification settings - Fork 209
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
String class needs someway to declare encoding of string #263
Comments
Fixed in #310 |
How do you create a String from a std::string() with a specified encoding? I didn't see a new constructor for that. |
I think I misunderstood why you opened this issue. Right now we have If you think |
It sounds like we want the whole matrix for: Inputs: |
Yeah, exactly (except I don't believe that representing the encoding as a string is really that useful) |
Also, personally, setting the default encoding that doesn't vary from platform to platform would reduce a huge number of bugs in my own code. |
OK. I will update this. (1) the ctor matrix; (2) when constructing from Any more suggestions? |
I also worry a little that |
Can I have more info? |
I made a typo - I should have done |
I won't pretend to speak with authority here, but one issue I think we will I think it's definitely desirable if self-contained libraries try to get On Tue, Aug 2, 2016 at 1:00 PM, Qiang Kou (KK) notifications@github.com
|
@hadley I know what you mean. Let's remove the string representation of encoding or add an @jjallaire What about using macros? Using |
Most R APIs will attempt to translate from UTF-8 to the system encoding when making system calls; the main issue is that translation can often fail for UTF-8 characters not representable in the system encoding. R typically handles these errors by rendering the unicode code point for the non-translatable UTF-8 character, e.g. converting The other commonly seen issue is that many R routines that use system calls will attempt to round-trip strings through the system encoding, e.g. 'UTF-8' => 'system' => 'UTF-8', and since the translation to the system encoding can be lossy (mainly on Windows) you get incorrect results. (One example of this I bumped into recently was with For what it's worth, I think in RStudio the primary issue we have is that we traffic around UTF-8 strings by default, but when passing those strings to R functions we don't mark the encoding. This means that, on Windows, we might end up with strings with contents that are UTF-8 encoded, but the CHARSXP itself has |
So it sounds like R will mostly do the right thing if the SEXP is marked I still wonder if a macro might be the way to activate the UTF8 by default On Tue, Aug 2, 2016 at 1:17 PM, Qiang Kou (KK) notifications@github.com
|
@jjallaire I think that change is relatively unlikely to affect much code because few people explicitly construct CharacterVector x(1);
x[0] = "abc" I don't know whether that implicitly constructs a |
It seems quite likely it's handled internally by CharacterVector. If String is indeed not used that much it wouldn't be much of a band-aid to On Tue, Aug 2, 2016 at 2:56 PM, Hadley Wickham notifications@github.com
|
Also the current behaviour makes it very easy to introduce bugs that only appear on Windows with non-English text. So if things do break with the change, it's likely to be just making existing breakage less silent. |
After defining #include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
void test(){
Rcout << "No String ctor" << std::endl;
CharacterVector x(1);
x[0] = "abc";
Rcout << "String ctor" << std::endl;
String y("abc");
String z = "abc";
}
/*
> Rcpp::sourceCpp("test.cpp")
> test()
No String ctor
String ctor
Rcpp/String.h: 145 String(const char*, cetype_t)
Rcpp/String.h: 145 String(const char*, cetype_t)
*/ |
Reopening at @coatless 's suggestion. |
e.g. instead of
Rf_mkCharCE("abc", CE_UTF8)
The text was updated successfully, but these errors were encountered: