Skip to content

Use dynamic CPU count for cmake --build -j in docs and test scripts - #20436

Merged
JakeStevens merged 1 commit into
pytorch:mainfrom
ShamSaleem:fix-10887-portable-cmake-jobs
Jul 22, 2026
Merged

Use dynamic CPU count for cmake --build -j in docs and test scripts#20436
JakeStevens merged 1 commit into
pytorch:mainfrom
ShamSaleem:fix-10887-portable-cmake-jobs

Conversation

@ShamSaleem

@ShamSaleem ShamSaleem commented Jun 22, 2026 •
edited by pytorch-bot Bot
Loading

Copy link
Copy Markdown
Contributor

Summary

Several build docs and test/ scripts hardcode the cmake --build -j
parallelism (-j9, -j10), which assumes a fixed machine. This replaces
them with a portable expression that derives "core count + 1" at runtime —
nproc on Linux, sysctl -n hw.ncpu on macOS:

-j$(( $(nproc 2>/dev/null || sysctl -n hw.ncpu) + 1 ))

"core count + 1" matches the guidance already documented in
docs/source/using-executorch-building-from-source.md. The nproc → sysctl
fallback keeps the commands working on both Linux and macOS, and the
arithmetic degrades gracefully to -j1 if neither tool is available.

Partial fix for #10887. Scope is limited to general (non-vendor) docs and
contributor-facing test/ build scripts (9 files). Vendor-backend scripts
(cadence, vulkan, coreml, qualcomm, mediatek, samsung, mps, nxp), CI scripts
under .ci/, and non-cmake -j flags are intentionally left for follow-ups.

Test plan

  • lintrunner passes on all changed files.
  • bash -n passes on the three modified shell scripts.
  • Verified the expression evaluates to a valid integer on Linux, e.g. 17
    on a 16-core machine.

cc @GregoryComer @digantdesai @cbilgin @JakeStevens @larryliu0820

@pytorch-bot

pytorch-bot Bot commented Jun 22, 2026 •
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20436

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ You can merge normally! (1 Unrelated Failure)

As of commit 3e079a1 with merge base 266e0dc (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 22, 2026
@linux-foundation-easycla

linux-foundation-easycla Bot commented Jun 22, 2026 •
edited
Loading

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: ShamSaleem / name: ShamSaleem (b340e88)

@github-actions

github-actions Bot commented Jun 22, 2026

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

digantdesai
digantdesai reviewed Jul 7, 2026
View reviewed changes
Comment thread backends/xnnpack/README.md

```bash
cmake --build cmake-out -j9 --target install --config Release
cmake --build cmake-out -j$(( $(nproc 2>/dev/null || sysctl -n hw.ncpu) + 1 )) --target install --config Release

@digantdesai digantdesai Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this work on a macos?

@nil-is-all nil-is-all Jul 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ShamSaleem, could you clarify on this review comment?

@ShamSaleem ShamSaleem Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late reply, I was on vacation. But yeah, it should work on macOS. nproc isn't available there by default, so nproc 2>/dev/null fails quietly and it falls back to sysctl -n hw.ncpu, which is the macOS equivalent. The $(( ... + 1 )) is POSIX arithmetic, so it's fine in both bash and zsh.

I don't have a Mac to test on though — if someone can confirm, that'd be great.

@nil-is-all nil-is-all added module: xnnpack Issues related to xnnpack delegation and the code under backends/xnnpack/ module: build/install Issues related to the cmake and buck2 builds, and to installing ExecuTorch labels Jul 14, 2026
@ShamSaleem
Use dynamic CPU count for cmake --build -j in docs and test scripts
3e079a1 
Replace hardcoded -j values (-j9, -j10) in the general build
documentation and the test/ build scripts with a portable expression
that derives "core count + 1" at runtime: nproc on Linux and
sysctl -n hw.ncpu on macOS. This matches the recommendation already
stated in docs/source/using-executorch-building-from-source.md and
avoids machine-specific job counts that don't fit the user's hardware.

Scope is limited to general (non-vendor) docs and contributor-facing
test/ scripts. Vendor-backend scripts, CI scripts under .ci/, and
non-cmake -j flags are intentionally left for a follow-up.

Partially addresses pytorch#10887.
@ShamSaleem
ShamSaleem force-pushed the fix-10887-portable-cmake-jobs branch from b340e88 to 3e079a1 Compare July 22, 2026 11:17
@JakeStevens
JakeStevens merged commit 7f86746 into pytorch:main Jul 22, 2026
194 of 195 checks passed
@ShamSaleem ShamSaleem mentioned this pull request Jul 29, 2026
Open
@nil-is-all nil-is-all added enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: cleanup Issues/PRs which cleanup code across the repository labels Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@digantdesai digantdesai digantdesai left review comments
@nil-is-all nil-is-all nil-is-all left review comments
@JakeStevens JakeStevens JakeStevens approved these changes
@larryliu0820 larryliu0820 Awaiting requested review from larryliu0820 larryliu0820 is a code owner
@kirklandsign kirklandsign Awaiting requested review from kirklandsign kirklandsign is a code owner
@manuelcandales manuelcandales Awaiting requested review from manuelcandales manuelcandales is a code owner
@JacobSzwejbka JacobSzwejbka Awaiting requested review from JacobSzwejbka JacobSzwejbka is a code owner
@mergennachin mergennachin Awaiting requested review from mergennachin mergennachin is a code owner

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: build/install Issues related to the cmake and buck2 builds, and to installing ExecuTorch module: cleanup Issues/PRs which cleanup code across the repository module: xnnpack Issues related to xnnpack delegation and the code under backends/xnnpack/

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ShamSaleem @digantdesai @JakeStevens @nil-is-all