[ROCm] Use linux.rocm.gpu.2 for 2-GPU and linux.rocm.gpu.4 for 4-GPU runners

[jithunnair-amd]

• Will enable us to target periodic/distributed CI jobs to 4-GPU runners using a different label linux.rocm.gpu.4
• Use 2-GPU runners for trunk, pull and slow (in addition to inductor-rocm) as well (although this currently will not change anything, since all our MI2xx runners have both linux.rocm.gpu and linux.rocm.gpu.2 labels... but this will change in the future: see next point)
• Continue to use linux.rocm.gpu label for any job that doesn't need more than 1-GPU eg. binary test jobs in workflows/generated-linux-binary-manywheel-nightly.yml

cc @jeffdaily @sunway513 @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @naromero77amd

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/143769

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 75 Pending, 3 Unrelated Failures

As of commit 7035cf6 with merge base 2ab698e ():

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• periodic / linux-focal-cuda11.8-py3.10-gcc9-debug / test (default, 2, 7, lf.linux.4xlarge.nvidia.gpu, oncall:debug-build) (gh) (trunk failure)
  test_nestedtensor.py::TestNestedTensorSubclassCUDA::test_linear_backward_memory_usage_cuda_float32
• periodic / linux-focal-cuda11.8-py3.9-gcc9 / test (multigpu, 1, 1, lf.linux.g5.12xlarge.nvidia.gpu, oncall:distributed) (gh) (trunk failure)
  test_cuda_multigpu.py::TestCudaMultiGPU::test_cuda_device_memory_allocated
• periodic / linux-focal-cuda12.4-py3.10-gcc9-bazel-test / build-and-test (default, 1, 1, lf.linux.4xlarge.nvidia.gpu) (gh) (trunk failure)
  Build completed, 1 test FAILED, 3229 total actions

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jithunnair-amd]

Will merge this once we have added the linux.rocm.gpu.4 label on the ROCm runners

[jithunnair-amd]

Successfully able to launch distributed config jobs on linux.rocm.gpu.4 label runners here: https://github.com/pytorch/pytorch/actions/runs/12474788837/job/34820089359

[jithunnair-amd]

@pytorchbot merge -f "Label updates successfully picked up runners. CI failures are not related to this PR"

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here