Add setaffinities and setaffinities_cpuids#116
Merged
carstenbauer merged 14 commits intomainfrom Mar 26, 2026
Merged
Conversation
Add `setaffinities(masks; threadpool)` and `setaffinities_cpuids(cpuids_vec; threadpool)` as multi-thread pendants of `setaffinity` / `setaffinity_cpuids`, analogous to the relationship between `pinthread` and `pinthreads`. Fixes #115. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- setaffinities(mask) and setaffinities_cpuids(cpuids) now accept a single mask/cpuid-vector and broadcast it to all threads in the pool - Add unit tests covering vector and broadcast variants of both functions, as well as ArgumentError on length mismatch Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The kernel may normalize affinity masks (e.g. strip trailing zeros) after a setaffinity syscall, so comparing raw mask bytes before and after is unstable across systems. Follow the existing setaffinity test style and only assert isnothing(...). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of only checking isnothing(...), pin to single-CPU masks/cpuid vectors and verify getcpuid() returns the expected CPU for each thread — the same approach used by pinthreads tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ThreadPinningCore.setaffinity(mask) bypasses the virtual CPU state that faking maintains for pinthreads/getcpuid. Under faking, getcpuid() returns the virtual fake CPU ID (unchanged by setaffinity), so exact-CPU checks after setaffinities/setaffinities_cpuids produce false failures on fake systems like FUGAKU. Guard those checks with !Faking.isfaking(), matching the existing precedent for checks that don't apply under faking. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
cpuids2affinitymask maps CPU IDs to positions in ThreadPinning.cpuids(). For systems with non-contiguous IDs (FUGAKU: 0,1,12,...,59), position 10 corresponds to CPU 19, not CPU 9. Under faking on a host with contiguous IDs, ThreadPinningCore.setaffinity interprets position 10 as real CPU 9, causing the mismatch (9 == 19 on FUGAKU, passes on other fake systems). Fix: get masks via getaffinity() after a pinthreads call instead, so the masks are in the real-host format that ThreadPinningCore.setaffinity expects. Keep !isfaking() guard on getcpuid() checks since setaffinity does not update the faking virtual state that getcpuid() reads from. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- setaffinities (mask) tests: after applying saved masks, verify via getaffinity() that the real OS masks match (getaffinity is not faked, so this works on both real and fake systems). getcpuids() check remains guarded by !isfaking() since getcpuid is faked. - setaffinities_cpuids tests: add ispinned() checks (reads real OS) to confirm each thread was actually pinned to a single CPU, even under faking where we can't verify which CPU via getcpuid. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Owner
Author
|
Specifically, you get the behavior desired in #115 by calling e.g. setaffinities_cpuids(numa(1)) # will set the affinities of all Julia threads to the first NUMA domain |
ThreadPinningCore.setaffinity(mask) extends the mask in-place via append! when mask length < uv_cpumask_size(). This caused setaffinities to mutate the caller's mask objects, leading to test failures when comparing the mutated masks (now extended to uv_cpumask_size) against fresh getaffinity() results (truncated to cpuidlimit/Sys.CPU_THREADS). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The faking layer is self-consistent for mask-based operations: setaffinity updates fake cpuid from mask bit position, and getaffinity reconstructs the mask from the fake cpuid. So the mask round-trip and getcpuid checks work correctly on all fake systems. The cpuid-based tests (setaffinities_cpuids) still need the guards because cpuids2affinitymask maps by list position, which diverges from the faking convention (cpuid = bit position - 1) on systems with non-contiguous CPU IDs like FUGAKU. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Julia 1.13 no longer accepts a function call as the library expression in @CCall (TypeError: expected Symbol, got Expr). Replace with explicit Libdl.dlsym/dlopen using a cached library handle to avoid repeated lookups. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Closed
Contributor
There was a problem hiding this comment.
Pull request overview
Adds multi-thread variants of the existing affinity-setting APIs so callers can set the affinity masks (or CPU ID sets) for an entire Julia threadpool at once, addressing the thread-affinity inheritance problem described in #115.
Changes:
- Introduces
setaffinities(masks; threadpool)andsetaffinities_cpuids(cpuids_vec; threadpool)APIs (with “single value broadcast” behavior). - Adds unit tests and API reference docs for the new functions.
- Updates MKL symbol lookup to use an explicit cached
dlopenhandle; bumps version/changelog.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
src/pinning.jl |
Adds public API declarations + implementations for setaffinities / setaffinities_cpuids. |
test/tests_pinning.jl |
Adds test coverage for vector-per-thread and broadcast forms + error cases. |
src/ThreadPinning.jl |
Exports the new APIs and defines non-Linux no-op fallbacks. |
docs/src/refs/api_pinning.md |
Adds the new APIs to the generated pinning API reference list. |
src/mkl.jl |
Switches MKL calls to dlsym off a cached dlopen handle; invalidates cache on path refresh. |
Project.toml |
Bumps version to 1.1.0-DEV. |
CHANGELOG.md |
Adds a 1.1 entry describing the new APIs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ector{<:Integer}}
Agent-Logs-Url: https://github.com/carstenbauer/ThreadPinning.jl/sessions/699ce3ff-9636-490c-a773-517e3f193554
Co-authored-by: carstenbauer <187980+carstenbauer@users.noreply.github.com>
Contributor
|
Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
setaffinities(masks; ...)signature toAbstractVector{<:AbstractVector{<:Integer}}⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.