Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reset threads after fork #2885

Closed
renkun-ken opened this issue May 17, 2018 · 1 comment
Closed

Reset threads after fork #2885

renkun-ken opened this issue May 17, 2018 · 1 comment
Labels
Milestone

Comments

@renkun-ken
Copy link
Member

@renkun-ken renkun-ken commented May 17, 2018

I understand that using OpenMP multi-threading in a fork of R session may end up in a deadlock as noted in this code. And threads are down limit to 1 in a fork. For safety concerns, the limit continues after fork.

This behavior makes RStudio Server users harder to leverage the multi-threading capabilities data.table has to offer since rstudio/rstudio#1827 notes that a refresh of RStudio server web page could trigger the behavior and since then data.table is limited to be single-threaded.

The recent release of fst package adopts the reset-after-fork behavior unless an option is toggled, as discussed at fstpackage/fst#110, fstpackage/fst#112. I'm not sure if it makes sense that data.table adopts this behavior in future if this is thoroughly tested?

@MarcusKlik
Copy link
Contributor

@MarcusKlik MarcusKlik commented May 17, 2018

To be on the safe side of things, the default behavior for data.table could also be set to switch back to single-threaded mode after a fork, but to allow overriding that behavior with an option.

Once that proves stable, that logic could be reversed.

Perhaps an additional argument reset_after_fork could also be added to setDTthreads() to set this behavior after the package has been loaded (like in fst::threads_fst()), so:

data.table::setDTthreads(threads = 4, reset_after_fork = FALSE)

to set the threads and disable switching back to single-threaded mode at the same time.

thanks for a great package by the way ! :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants