Skip to content

Conversation

@amdfaa
Copy link
Contributor

@amdfaa amdfaa commented Oct 29, 2025

  • We are separating out the rocm jobs of the periodic workflow
  • We are introducing a new label ciflow/periodic-rocm-mi200 to allow us to run distributed tests only on ROCm runners, without triggering many other jobs on the periodic.yml workflow (via ciflow/periodic)
  • This new workflow will also be triggered via the ciflow/periodic, thus maintaining the old status quo.
  • We are reverting to the linux.rocm.gpu.4 label since it targets a lot more CI nodes at this point than the K8s/ARC-based linux.rocm.gpu.mi250.4 label, as that is still having some network/scaling issues.

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @naromero77amd

@amdfaa amdfaa requested a review from a team as a code owner October 29, 2025 17:34
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 29, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/166544

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

⏳ No Failures, 13 Pending

As of commit 98717aa with merge base deb7763 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Oct 29, 2025
@jeffdaily jeffdaily changed the title Create periodic-mi200 [ROCm][CI] Create periodic-mi200 Oct 29, 2025
@pytorch-bot pytorch-bot bot added the module: rocm AMD GPU support for Pytorch label Oct 29, 2025
@jithunnair-amd jithunnair-amd changed the title [ROCm][CI] Create periodic-mi200 [ROCm][CI] Create periodic-rocm-mi200.yml Oct 29, 2025
@pytorch-bot pytorch-bot bot added the ciflow/rocm Trigger "default" config CI on ROCm label Oct 29, 2025
@pytorch-bot pytorch-bot bot removed the ciflow/rocm Trigger "default" config CI on ROCm label Oct 29, 2025
@jeffdaily
Copy link
Collaborator

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 29, 2025
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@jithunnair-amd jithunnair-amd added the ciflow/periodic-rocm-mi200 Trigger "distributed" config CI on ROCm MI200 label Oct 29, 2025
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 29, 2025

Warning: Unknown label ciflow/periodic-rocm-mi200.
Currently recognized labels are

  • ciflow/b200
  • ciflow/b200-symm-mem
  • ciflow/b200-distributed
  • ciflow/binaries
  • ciflow/binaries_libtorch
  • ciflow/binaries_wheel
  • ciflow/h100
  • ciflow/h100-cutlass-backend
  • ciflow/h100-distributed
  • ciflow/h100-symm-mem
  • ciflow/inductor
  • ciflow/inductor-cu126
  • ciflow/inductor-micro-benchmark
  • ciflow/inductor-micro-benchmark-cpu-x86
  • ciflow/inductor-perf-compare
  • ciflow/inductor-perf-test-nightly-rocm-mi300
  • ciflow/inductor-perf-test-nightly-rocm-mi355
  • ciflow/inductor-perf-test-nightly-x86-zen
  • ciflow/inductor-periodic
  • ciflow/inductor-rocm
  • ciflow/linux-aarch64
  • ciflow/mps
  • ciflow/nightly
  • ciflow/op-benchmark
  • ciflow/periodic
  • ciflow/periodic-rocm-mi300
  • ciflow/pull
  • ciflow/quantization-periodic
  • ciflow/riscv64
  • ciflow/rocm
  • ciflow/rocm-mi300
  • ciflow/rocm-mi355
  • ciflow/rocm-navi31
  • ciflow/s390
  • ciflow/slow
  • ciflow/torchbench
  • ciflow/triton_binaries
  • ciflow/trunk
  • ciflow/unstable
  • ciflow/vllm
  • ciflow/win-arm64
  • ciflow/xpu

Please add the new label to .github/pytorch-probot.yml

@pytorch-bot pytorch-bot bot removed ciflow/trunk Trigger trunk jobs on your pull request ciflow/periodic-rocm-mi200 Trigger "distributed" config CI on ROCm MI200 labels Oct 30, 2025
@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: New commits were pushed while merging. Please rerun the merge command.

Details for Dev Infra team Raised by workflow job

@jithunnair-amd
Copy link
Collaborator

@pytorchbot merge -f "Force merging to relieve MI2xx queueing and provide separate workflow to target ROCm MI2xx distributed jobs"

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Merged module: rocm AMD GPU support for Pytorch open source topic: not user facing topic category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants