Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows 10 parallelization can be super slow #94

Closed
avehtari opened this Issue Dec 30, 2018 · 1 comment

Comments

Projects
None yet
2 participants
@avehtari
Copy link
Collaborator

avehtari commented Dec 30, 2018

Lawrence Hunsicker reported that Windows 10 is super slow in parallelization if .rprofile has options(mc.cores = parallel::detectCores()). The same line in a script is not a problem,
https://discourse.mc-stan.org/t/problem-parallelizing-rstanarm-loo/6433/10

I don't know what we can do for this in loo package, but I add this here if someone is hit by the same Windows bug.

@jgabry

This comment has been minimized.

Copy link
Member

jgabry commented Jan 3, 2019

We could mention this in the package documentation, but if this is very severe then maybe it's worth printing a message about it to Windows users when the package is loaded?
I'm not sure what the message would be precisely, but it's possible by adding to the .onAttach() function. Currently it's this:

loo/R/zzz.R

Lines 1 to 10 in e147fd3

.onAttach <- function(...) {
ver <- utils::packageVersion("loo")
packageStartupMessage(
"This is loo version ", ver, ".\n",
"**NOTE: As of version 2.0.0 loo defaults to 1 core ",
"but we recommend using as many as possible. ",
"Use the 'cores' argument or set options(mc.cores = NUM_CORES) for an entire session. ",
"Visit mc-stan.org/loo/news for details on other changes."
)
}

avehtari added a commit that referenced this issue Feb 3, 2019

jgabry added a commit that referenced this issue Feb 27, 2019

Merge pull request #99 from stan-dev/add-a-warning-issue-94
add warning about Windows 10 core problem #94

@avehtari avehtari closed this Mar 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.