Handle Turkish dotted and dotless i properly

[yutannihilation]

Fix #3000

Turkish has two versions of i; dotted and dotless. Accordingly, the capital of i is İ, not I, which make ggplot2 to fail to find PositionIdentity and StatIdentity. The same thing can happen on dotless ı and I (c.f. http://www.i18nguy.com/unicode/turkish-i18n.html)

This seems a well-known problem, and ICU has a flag for this, but unfortunately, there's no way to use this via R's toupper().

It is also not language-sensitive, although there is a flag for whether to apply special mappings for use with Turkic (Turkish/Azerbaijani) text data.
(http://userguide.icu-project.org/transforms/casemappings)

After googling around, I found several options to fix this:

1. Use chartr() instead of toupper() and tolower() (This PR)
2. Use stringi::stri_trans_toupper() and stringi::stri_trans_tolower(), where we can specify locale.
3. Modify the locale temporarily

For 2., stringi is cool, but adding it to the dependency is a bit heavy. For 3., it seems dangerous to mess the user's locale. So, from these, I think option 1 is rather reasonable, although not very cool. chartr() is around 4x slower than toupper(), but I hope this is acceptable considering that people in Turkish locale probably almost cannot use ggplot2... (I'm not familiar with this problem, so I might be wrong.)

bench::mark(
  chartr("abcdefghijklmnopqrstuvwxyz", "ABCDEFGHIJKLMNOPQRSTUVWXYZ", "a"),
  toupper("a")
)
#> # A tibble: 2 x 10
#>   expression     min   mean median   max `itr/sec` mem_alloc  n_gc n_itr
#>   <chr>      <bch:t> <bch:> <bch:> <bch>     <dbl> <bch:byt> <dbl> <int>
#> 1 "chartr(\~   3.3us 7.03us  3.8us 732us   142357.        0B     0 10000
#> 2 "toupper(~   700ns 2.24us    1us 291us   446879.        0B     0 10000
#> # ... with 1 more variable: total_time <bch:tm>

[hadley]

Approach seems good. It would be nice to have a unit test, but I can't think of an obvious way to simulate it.

[hadley]

Can we make this to_lower_ascii() and to_upper_ascii()?

And I think we should wrap to_lower() and to_upper() to throw an error, to make sure that this error doesn't arise again in the future.

[yutannihilation]

Thanks, I added some unit tests. Do you think it's overkill to install language-pack-tr-base just for several test cases? If so, I'm OK to remove this. Anyway, I believe the fix is confirmed as valid since this commit passsed checks:

5f1bb92
https://travis-ci.org/tidyverse/ggplot2/jobs/457186678

[yutannihilation]

Ah, sorry, it seems I'm doing wrong about detecting the available locales... Let me fix.

any(x == "tr_TR") isn't true.
https://travis-ci.org/tidyverse/ggplot2/builds/457214479#L5545

[hadley]

Sorry my comment was supposed to reinforce not having a unit test. I don't think it gets us much here as it's likely to be fragile, and we've reduced the chance by overwriting toupper()

[hadley]

"in Turkish locale" -> "in all locales"

[yutannihilation]

Oh, I got it wrong, sorry. Agreed.

[yutannihilation]

Thanks for confirmation, @cystein!

[lock]

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/