[Release only] Pin rocm docker images

[atalman]

Fix rocm failures in release branch

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126452

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 36 New Failures, 1 Unrelated Failure

As of commit f85309d with merge base 86a2d67 ():

NEW FAILURES - The following jobs have failed:

• pull / linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 2, 3, linux.8xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, linux.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 2, 5, linux.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 4, 5, linux.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 5, 5, linux.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 2, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 3, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 4, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 5, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-py3_8-clang9-xla / test (xla, 1, 1, linux.12xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-py3.11-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-py3.11-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-py3.11-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
  Process completed with exit code 1.
• pull / linux-focal-py3.8-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-py3.8-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-py3.8-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-focal-py3.8-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.10-clang15-asan / test (default, 1, 6, linux.4xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.10-clang15-asan / test (default, 2, 6, linux.4xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.10-clang15-asan / test (default, 3, 6, linux.4xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.10-clang15-asan / test (default, 4, 6, linux.4xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.10-clang15-asan / test (default, 5, 6, linux.4xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.10-clang15-asan / test (default, 6, 6, linux.4xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.8-gcc11 / test (default, 1, 3, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.8-gcc11 / test (default, 2, 3, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• pull / linux-jammy-py3.8-gcc11 / test (default, 3, 3, linux.2xlarge) (gh)
  ##[error]The operation was canceled.
• rocm / linux-focal-rocm6.0-py3.8 / test (default, 1, 6, linux.rocm.gpu.2) (gh)
  ##[error]The operation was canceled.
• rocm / linux-focal-rocm6.0-py3.8 / test (default, 2, 6, linux.rocm.gpu.2) (gh)
  ##[error]The operation was canceled.
• rocm / linux-focal-rocm6.0-py3.8 / test (default, 3, 6, linux.rocm.gpu.2) (gh)
  ##[error]The operation was canceled.
• rocm / linux-focal-rocm6.0-py3.8 / test (default, 4, 6, linux.rocm.gpu.2) (gh)
  ##[error]The operation was canceled.
• rocm / linux-focal-rocm6.0-py3.8 / test (default, 5, 6, linux.rocm.gpu.2) (gh)
  ##[error]The operation was canceled.
• rocm / linux-focal-rocm6.0-py3.8 / test (default, 6, 6, linux.rocm.gpu.2) (gh)
  ##[error]The operation was canceled.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / linux-focal-py3.12-clang10 / test (default, 3, 3, linux.2xlarge) (gh) (disabled by #123655 but the issue was closed recently and a rebase is needed to make it pass)
  test_binary_ufuncs.py::TestBinaryUfuncsCPU::test_logaddexp_cpu_complex128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jithunnair-amd]

@atalman For the sake of posterity, can we elaborate a bit on why we need to pin the ROCm docker images for the release branch?

[atalman]

@jithunnair-amd This is general problem, we would need to pin all CI docker images - not only rocm builds. I will plan changes in main. This PR is to make all rocm jobs green by using same docker image used in CI for 2.3.0