Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support mc.cores greater than number of physical cores #57

Merged
merged 2 commits into from
Oct 23, 2020
Merged

Support mc.cores greater than number of physical cores #57

merged 2 commits into from
Oct 23, 2020

Conversation

benfasoli
Copy link
Contributor

This adds a processes_per_node argument which is passed to parallel::mcmapply(mc.cores = processes_per_node). Specifying processes_per_node > cpus_per_node can more efficiently utilize nodes supporting hyperthreading.

The default behavior is processes_per_node = cpus_per_node for backwards compatibility.

Closes #56

@qdread
Copy link
Contributor

qdread commented Oct 19, 2020

Thanks @benfasoli ! I will take a look shortly.

@qdread
Copy link
Contributor

qdread commented Oct 22, 2020

Hey @benfasoli I'm testing out the PR but I think there is a bug still: check out for example line 173 in slurm_apply.R. There you need to pass the processes_per_node argument to whisker.render(). Otherwise, the slurm_run_R.txt template generates a slurm_run.R file with an empty argument for mc.cores. That would need to be fixed in a few places, I think. Do you mind going through and fixing that in the relevant places? thanks!

@benfasoli
Copy link
Contributor Author

Nice catch. processes_per_node was being passed to the SLURM submission script template (where it is unused) and not to the R script template.

@qdread
Copy link
Contributor

qdread commented Oct 22, 2020

Wow that was quick! Thanks! 💯 ... I probably could have done it but I thought it would be fresher in your mind where all it was needed to fix.

@benfasoli
Copy link
Contributor Author

Happy to help! Looks like this passes devtools::test() now.

@qdread qdread merged commit bb55479 into earthlab:master Oct 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support mc.cores greater than number of physical cores
2 participants