Disconnecting Constantly

[exalate-issue-sync]

I am experiencing a recurrent problem when automating creation of h2o models in R.

I have a script in R (using RStudio) that contains a for loop, and each iteration of that loop creates a subset of a larger data frame, uses h2o.automl to train and evaluate a model on that data frame, and then writes some statistics on that model and it’s output into a table. When I run this loop, it will work perfectly for multiple cycles. Usually between about 5 and 20 (it's only training a few default XGBoost models, so each iteration completes in only a minute or so). Then R loses its connection to the h2o instance, and usually RStudio also loses its connection to the R session, forcing me to quit and restart. This happens every single time, but only after a variable number of iterations are completed successfully, which makes tracking down the problem difficult. Since the entire loop can be completed multiple times successfully before it happens, it doesn’t seem like my script is the problem.

Some addition information on the errors occurring. When the problem occurs, RStudio CPU usage stays steady at about 100%. Shortly after, this error message prints in the RStudio console:

“Error in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = page, :
Unexpected CURL error: Connection timeout after 300000 ms“

The RStudio log file has numerous entries that say:

“ERROR asio.misc error 4 (The descriptor does not fit into the select call's fd_set); OCCURRED AT void rstudio::session::HttpConnectionImpl<rstudio_boost::asio::ip::tcp>::handleRead(const boost::system::error_code &, std::size_t) [ProtocolType = rstudio_boost::asio::ip::tcp] src/cpp/session/http/SessionPosixHttpConnectionListener.cpp:368; LOGGED FROM: void rstudio::session::HttpConnectionImpl<rstudio_boost::asio::ip::tcp>::handleRead(const boost::system::error_code &, std::size_t) [ProtocolType = rstudio_boost::asio::ip::tcp] src/cpp/session/http/SessionPosixHttpConnectionListener.cpp:370”

h2o Flow remains accessible via browser, and appears to function normally.

I am initiating h2o through R using the default settings. R version 4.1.3. RStudio 2022.02.1 Build 461. h2o 3.36.0.3. java version "18" 2022-03-22 (also had issue when running version 17). macOS Monterey 12.3 running on Intel MacBook Pro.

Please let me know if more details are needed. I love h2o, but this is preventing me from being able to really take full advantage of it.

[exalate-issue-sync]

Michael N commented: After several more days of searching, I think I found the solution to my problem here:

[https://stackoverflow.com/questions/69485936/error-in-h2o-dosaferesth2orestapiversion-h2orestapiversion-urlsuffix-urls|https://stackoverflow.com/questions/69485936/error-in-h2o-dosaferesth2orestapiversion-h2orestapiversion-urlsuffix-urls|smart-link]

It appears this is not an h2o bug, but rather a problem with the version of CURL that’s included with MacOS. By installing an updated version of CURL using Homebrew with the --disable-socketpair argument set, it looks like I have been able to stop the problem from occurring. Perhaps MacOS users should be warned of this when downloading h2o? I’m copying the instructions in the linked post below just in case:

{noformat}$ brew edit curl # add --disable-socketpair to args list
$ brew install --build-from-source curl # using reinstall might be needed instead of install

$ export RCURL_PATH="usr/local/opt/curl@7.81.0" # can be found using brew info curl
$ export PATH="$RCURL_PATH/bin:$PATH" # for curl-config
$ export LDFLAGS="-L$RCURL_PATH//lib"
$ export CPPFLAGS="-I$RCURL_PATH/include"
$ export PKG_CONFIG_PATH="$RCURL_PATH/lib/pkgconfig"

$ R -e "chooseCRANmirror(graphics=FALSE, ind=1);install.packages('RCurl', type = 'source')"
$ R -e "RCurl::curlVersion()$version" # check if RCurl is using the proper version of curl
{noformat}

[exalate-issue-sync]

Tomas Fryda commented: [~accountid:6244bbe6f6a26900695cb5a2] Thank you for reporting this issue. I added an additional warning when the issue could be caused by this {{RCurl}} issue. In the next release we also detect if the user has a recent version of R package {{curl}} (4.3.0 and above) and in that case we use it. The development of {{curl}} package is much more active than {{RCurl}} and doesn’t have this issue.

[h2o-ops]

JIRA Issue Details

Jira Issue: PUBDEV-8643
Assignee: Tomas Fryda
Reporter: Michael N
State: Resolved
Fix Version: 3.36.1.1
Attachments: N/A
Development PRs: Available

[h2o-ops]

Linked PRs from JIRA

#6141