Set default value for nccl make MAX_JOBS if ProcessorCount returns 0

[mrshenli]

Stack from ghstack (oldest at bottom):

• -> Set default value for nccl make MAX_JOBS if ProcessorCount returns 0 #84231
• [reland] Make allreduce compatible with make_fx #84221

[facebook-github-bot]

🔗 Helpful links

• 🧪  See artifacts and rendered test results at hud.pytorch.org/pr/84231
• ✖️ Python docs build was skipped
• ✖️ C++ docs build was skipped
• ❓Need help or want to give feedback on the CI? Visit our office hours

✅ No Failures (0 Pending)

As of commit 0c57d84 (more details on the Dr. CI page):

Expand to see more

---

💚 💚 Looks good so far! There are no failures yet. 💚 💚

---

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[mrshenli]

The error message I got is:

[108/6838] Performing build step for 'nccl_external'                                                                                                              
FAILED: nccl_external-prefix/src/nccl_external-stamp/nccl_external-build nccl/lib/libnccl_static.a /fsx/users/shenli/pytorch/build/nccl_external-prefix/src/nccl_e
xternal-stamp/nccl_external-build /fsx/users/shenli/pytorch/build/nccl/lib/libnccl_static.a                                                                       
cd /fsx/users/shenli/pytorch/third_party/nccl/nccl && env CCACHE_DISABLE=1 SCCACHE_DISABLE=1 make -j0 -l0 CXX=/data/shared/bin/c++ CUDA_HOME=/usr/local/cuda-11.1
NVCC=/data/shared/bin/cuda/nvcc NVCC_GENCODE=-gencode=arch=compute_80,code=sm_80 BUILDDIR=/fsx/users/shenli/pytorch/build/nccl VERBOSE=0 && /fsx/users/shenli/cond
a/envs/compiler/bin/cmake -E touch /fsx/users/shenli/pytorch/build/nccl_external-prefix/src/nccl_external-stamp/nccl_external-build
make: the '-j' option requires a positive integer argument

[malfet]

LGTM if CI is green (why env is needed though)?

[malfet]

Why this is needed?

[mrshenli]

It was there before, but removed by #83696. I thought we need to pass this to pass env vars to make if necessary?

[peterbell10]

No env vars are being passed though, it was left over from the CCACHE_DISABLE=1 that was there before.

[mrshenli]

Oh I see, let me remove that then.

[rohan-varma]

is this change intentional?

[mrshenli]

Yep, it was there before, but removed by #83696. I thought we need to pass this to pass env vars to make if necessary?

[mrshenli]

@pytorchbot merge

[pytorchmergebot]

@pytorchbot successfully started a merge job. Check the current status here and land check progress here.
The merge job was triggered with the land checks (-l) flag. If you did not specify this flag yourself, you are likely enrolled in the land checks rollout. This means that your change will be merged once all checks on your PR and the land checks have passed (ETA 4 Hours). If you need to coordinate lands between different changes and cannot risk a land race, please add the ciflow/trunk label to your PR and wait for signal to complete, and then land your changes in proper order. Having trunk, pull, and Lint pre-run on a PR will bypass land checks and the ETA should be immediate. If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

[pytorchmergebot]

Merge failed

Reason: 2 additional jobs have failed, first few of them are: trunk ,trunk / macos-12-py3-arm64 / test (default, 1, 2, macos-m1-12)

If you believe this is an error, you can use the old behavior with @pytorchbot merge -g (optionally with the ciflow/trunk to get land checks) or use @pytorchbot merge -f "some reason here". For more information, see the bot wiki.

Please reach out to the PyTorch DevX Team with feedback or questions!

Details for Dev Infra team

Raised by workflow job

[mrshenli]

@pytorchbot merge -g

[mrshenli]

@pytorchbot merge -g

[pytorchmergebot]

@pytorchbot successfully started a merge job. Check the current status here.
The merge job was triggered with the green (-g) flag. This means that your change will be merged once all checks on your PR have passed (ETA: 0-4 Hours). If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

[github-actions]

Hey @mrshenli.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.