Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support basic core affinities in LocalManager and SSHManager #10733

Closed
wants to merge 1 commit into from

Conversation

amitmurthy
Copy link
Contributor

Allows pinning workers to cores. Linux only feature, requires the taskset command to be installed on the system.

Closes #10645

@amitmurthy
Copy link
Contributor Author

This branch has been tested on julia.mit.edu with the same results as #10427 (comment) .

Some things to be noted:

  • It is currently a linux only feature - which is fine I guess, since most of the larger HPC type machines where this feature is required are linux.
  • The current PR automatically associates julia workers with cores. affinity=:compact associates cores serially, while affinity=:balanced spreads out the julia workers across available cores.
  • It is a basic feature, in the sense that it is not aware of the exact topology like that provided by https://github.com/JuliaParallel/hwloc.jl . But that could be provided by a custom cluster manager that uses hwloc.jl for more intelligent core-julia process mapping taking into account processor sockets, L1/L2 caches, etc.

I would like to merge this in a few days if there are no specific concerns.

@ViralBShah
Copy link
Member

LGTM.

@amitmurthy
Copy link
Contributor Author

I have changed my mind, and on further thinking, I am closing this PR in favor of adding this functionality to ClusterManagers.jl in a new LocalAffinityManager. Mainly because LocalAffinityManager can be more easily developed outside of Base to work with L1/L2 cache information, cores to socket mapping, etc using hwloc.jl.

@amitmurthy amitmurthy closed this Apr 16, 2015
@amitmurthy amitmurthy deleted the amitm/cpuaffinity branch June 14, 2015 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

option to set processor affinity in addprocs
2 participants