I understand that using OpenMP multi-threading in a fork of R session may end up in a deadlock as noted in this code. And threads are down limit to 1 in a fork. For safety concerns, the limit continues after fork.
This behavior makes RStudio Server users harder to leverage the multi-threading capabilities data.table has to offer since rstudio/rstudio#1827 notes that a refresh of RStudio server web page could trigger the behavior and since then data.table is limited to be single-threaded.
The recent release of fst package adopts the reset-after-fork behavior unless an option is toggled, as discussed at fstpackage/fst#110, fstpackage/fst#112. I'm not sure if it makes sense that data.table adopts this behavior in future if this is thoroughly tested?
The text was updated successfully, but these errors were encountered:
To be on the safe side of things, the default behavior for data.table could also be set to switch back to single-threaded mode after a fork, but to allow overriding that behavior with an option.
Once that proves stable, that logic could be reversed.
Perhaps an additional argument reset_after_fork could also be added to setDTthreads() to set this behavior after the package has been loaded (like in fst::threads_fst()), so:
I understand that using OpenMP multi-threading in a fork of R session may end up in a deadlock as noted in this code. And threads are down limit to 1 in a fork. For safety concerns, the limit continues after fork.
This behavior makes RStudio Server users harder to leverage the multi-threading capabilities data.table has to offer since rstudio/rstudio#1827 notes that a refresh of RStudio server web page could trigger the behavior and since then data.table is limited to be single-threaded.
The recent release of fst package adopts the reset-after-fork behavior unless an option is toggled, as discussed at fstpackage/fst#110, fstpackage/fst#112. I'm not sure if it makes sense that data.table adopts this behavior in future if this is thoroughly tested?
The text was updated successfully, but these errors were encountered: