Turn on data.table support in h2o R package by default

[exalate-issue-sync]

Let's make the h2o R package use data.table by default (if installed) for the as.h2o() and as.data.frame() functions, rather than forcing user to set options("h2o.use.data.table"=TRUE) in their code. This gives a huge speed up!

[exalate-issue-sync]

Erin LeDell commented: We are already forcing it properly here: https://github.com/h2oai/h2o-3/blob/master/h2o-r/h2o-package/R/frame.R#L3232

But here are two places where it relies on user options (which will not be set unless the user explicitly knows to do so).
https://github.com/h2oai/h2o-3/blob/master/h2o-r/h2o-package/R/frame.R#L3203
https://github.com/h2oai/h2o-3/blob/master/h2o-r/h2o-package/R/frame.R#L3367

Should we always force the use, even if the user has manually set it to false? I guess we should honor that too.

[exalate-issue-sync]

Hugh Parsonage commented: The option should be TRUE or NULL by default, otherwise it causes performance regressions (e.g. as.h2o will not work unless a global option is set, which is opaque).

I don't think it makes much sense to turn off e.g. fwrite when it's available.

[exalate-issue-sync]

Jan Sterba commented: Was possoble to do this for Matrix conversion, however using data.table for csv parsing needs to remain optional because of data.table buggy handling of quote-escaping.

[exalate-issue-sync]

Matt Dowle commented: Jan - I find the phrase “data.table buggy handling of” a bit flippant and unhelpful. I thought data.table is one of the best file readers at handling quoting. It would be more helpful if you could point me to a specific issue you have faced. There are bugs yes but I don’t think it warrants that phrase.

[exalate-issue-sync]

Jan Sterba commented: [~accountid:557058:393936ef-8683-427b-babb-14ffad4bb6d7] I am sorry you find my comment flippant, I did not mean to offend anyone.

I have spend plenty of time trying to make data.table#fread handle escaped quotes in CSV and was unable to, hence I concluded its simply impossible.

• backslash-escaped quotes are parsed including the back-slash
• double-double-qoute escaped quotes are also read as-is

This makes parsing single column csv with values with one quote inside impossible to parse. For more details see my comment in the pull request please.

[hasithjp]

JIRA Issue Migration Info

Jira Issue: PUBDEV-4639
Assignee: Jan Sterba
Reporter: Erin LeDell
State: Resolved
Fix Version: 3.30.0.1
Attachments: N/A
Development PRs: Available

Linked PRs from JIRA

#1409
#1453
#4265