non-ASCII characters

[beanumber]

Version: 0.1.4
Check: data for non-ASCII characters
Result: NOTE
     Note: found 2698 marked UTF-8 strings
Flavors: r-devel-linux-x86_64-fedora-clang, r-devel-linux-x86_64-fedora-gcc, r-patched-solaris-x86, r-release-osx-x86_64, r-oldrel-osx-x86_64

Ugh.

[beanumber]

See #17

[nicholasjhorton]

There are pedagogical reasons why we would want to have non-ASCII characters. Any idea why this is now a requirement for CRAN?

[beanumber]

I have no idea. Cultural imperialism?

Somehow we fixed this before, but I'm running the same code now and it doesn't seem to be working.

> showNonASCII(WorldCities$name)
511: Xank<c2><82><c2><9d>ndi
518: L<c2><82><c2><9d>nk<c2><82><c2><9d>ran
523: Beyl<c2><83><c2><aa>qan
> x <- WorldCities$name
> Encoding(x) <- "UTF-8"
> y <- iconv(x, from = "UTF-8", to = "latin1", sub = "")
> showNonASCII(y)
511: Xank<82><9d>ndi
518: L<82><9d>nk<82><9d>ran
523: Beyl<83><aa>qan
> z <- stringi::stri_trans_general(WorldCities$name, "latin-ascii")
> showNonASCII(z)
511: Xank<c2><82><c2><9d>ndi
518: L<c2><82><c2><9d>nk<c2><82><c2><9d>ran
523: Beyl<c2><83><c2><aa>qan

Ugh, I am at a loss here. Any help would be appreciated!

[nicholasjhorton]

How about: iconv(x, "latin1", "ASCII", "")

On Dec 15, 2017, at 10:30 AM, Ben Baumer ***@***.***> wrote: I have no idea. Cultural imperialism? Somehow we fixed this before, but I'm running the same code now and it doesn't seem to be working. > showNonASCII(WorldCities$name) 511: Xank<c2><82><c2><9d>ndi 518: L<c2><82><c2><9d>nk<c2><82><c2><9d>ran 523: Beyl<c2><83><c2><aa>qan > x <- WorldCities$name > Encoding(x) <- "UTF-8" > y <- iconv(x, from = "UTF-8", to = "latin1", sub = "") > showNonASCII(y) 511: Xank<82><9d>ndi 518: L<82><9d>nk<82><9d>ran 523: Beyl<83><aa>qan > z <- stringi::stri_trans_general(WorldCities$name, "latin-ascii") > showNonASCII(z) 511: Xank<c2><82><c2><9d>ndi 518: L<c2><82><c2><9d>nk<c2><82><c2><9d>ran 523: Beyl<c2><83><c2><aa>qan Ugh, I am at a loss here. Any help would be appreciated! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Nicholas Horton Professor of Statistics Department of Mathematics and Statistics, Amherst College PO Box 5000, AC #2239 Amherst, MA 01002-5000

[beanumber]

OK, that seems to have worked. Thanks @nicholasjhorton !