[ROCm] hipGraph support for pytorch mainline

[dllehr-amd]

With the release of ROCm 5.3 hip now supports a hipGraph implementation.

All necessary backend work and hipification is done to support the same functionality as cudaGraph.

Unit tests are modified to support a new TEST_GRAPH feature which allows us to create a single check for graph support instead of attempted to gather the CUDA level in annotations for every graph test

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/88202

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Failures

As of commit 394a1b6:

NEW FAILURES - The following jobs have failed:

• linux-bionic-py3.8-clang9 / test (dynamo, 1, 2, linux.2xlarge)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: dllehr-amd / name: Douglas Lehr (55e2dff)

[pruthvistony]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot successfully started a rebase job. Check the current status here

[pytorchmergebot]

Rebase failed due to Command git -C /home/runner/work/pytorch/pytorch rebase refs/remotes/origin/viable/strict pull/88202/head returned non-zero exit code 1

Rebasing (1/1)
Auto-merging c10/cuda/CUDACachingAllocator.cpp
CONFLICT (content): Merge conflict in c10/cuda/CUDACachingAllocator.cpp
Auto-merging cmake/Dependencies.cmake
Auto-merging test/test_cuda.py
Auto-merging torch/utils/hipify/cuda_to_hip_mappings.py
error: could not apply 55e2dff055... hipGraph support for pytorch mainline
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".
Could not apply 55e2dff055... hipGraph support for pytorch mainline

Raised by https://github.com/pytorch/pytorch/actions/runs/3408082704

[jithunnair-amd]

These changes have been tested in our fork of PyTorch with ROCm5.3.

[jithunnair-amd]

@dllehr-amd Let's take this PR back out of draft when the CI is green and it's ready to be reviewed and merged by PyTorch reviewers.

[pruthvistony]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot successfully started a rebase job. Check the current status here

[pytorchmergebot]

Successfully rebased dllehr_graph onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout dllehr_graph && git pull --rebase)

[dllehr-amd]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot successfully started a rebase job. Check the current status here

[pytorchmergebot]

Successfully rebased dllehr_graph onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout dllehr_graph && git pull --rebase)

[jithunnair-amd]

Checked the latest CI status: 4 jobs failed:
linux-bionic-py3.8-clang9 / test (dynamo, 1, 2, linux.2xlarge) : Unrelated to this PR
linux-focal-rocm5.4.2-py3.8 / test (distributed, 1, 2, linux.rocm.gpu) : Failed due to docker image not available: restarted job PASSED (https://github.com/pytorch/pytorch/actions/runs/4147373535/jobs/7213219132)
linux-focal-rocm5.4.2-py3.8 / test (distributed, 2, 2, linux.rocm.gpu) : Failed due to docker image not available: restarted job PASSED (https://github.com/pytorch/pytorch/actions/runs/4147373535/jobs/7221626142)
linux-focal-rocm5.4.2-py3.8 / test (slow, 1, 1, linux.rocm.gpu) : Failed due to docker image not available: restarted job PASSED (https://github.com/pytorch/pytorch/actions/runs/4147373535/jobs/7213218956)

[jithunnair-amd]

@pytorchbot merge -f "Unrelated CI failures"

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here