Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better support for encoding #1293

Closed
arunsrinivasan opened this issue Aug 26, 2015 · 2 comments
Closed

Better support for encoding #1293

arunsrinivasan opened this issue Aug 26, 2015 · 2 comments
Assignees
Milestone

Comments

@arunsrinivasan
Copy link
Member

One way to provide better Encoding support is to provide an argument while creating/converting a data.table. That is,

data.table(..., encoding = c("unknown", "UTF-8", "Latin-1"))
as.data.table(..., encoding = c("unknown", "UTF-8", "Latin-1"))
setDT(..., encoding = c("unknown", "UTF-8", "Latin-1"))

Since we convert the i argument of joins to data.tables, even scenarios of vectors/lists are handled. This should drastically reduce the occurrence of encoding warnings.

@izahn
Copy link

izahn commented Nov 7, 2015

It would be good to improve the situation with respect to merging on columns with marked encodings. The current warning message is leading to nonsense like http://stackoverflow.com/q/33482308/189946

@shrektan
Copy link
Member

shrektan commented Nov 8, 2015

+1

发自我的 iPhone

在 2015年11月8日,01:25,Ista Zahn notifications@github.com 写道:

It would be good to improve the situation with respect to merging on columns with marked encodings. The current warning message is leading to nonsense like http://stackoverflow.com/q/33482308/189946


Reply to this email directly or view it on GitHub.

arunsrinivasan added a commit that referenced this issue Jan 20, 2016
@arunsrinivasan arunsrinivasan self-assigned this Jan 20, 2016
arunsrinivasan added a commit that referenced this issue Jan 27, 2016
When marked non-utf8 encodings are detected, they are internally converted to utf-8 before comparing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants