[ROCm] use correct workspace for hipblaslt, silence warning

[ethanwee1]

Follow up to #145130. That PR caused a warning on ROCm the first time hipblaslt was called for any workload, always.

Fixes #ISSUE_NUMBER

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @naromero77amd

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/150227

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Unrelated Failures

As of commit 03e8aa0 with merge base cbc0964 ():

NEW FAILURES - The following jobs have failed:

• linux-binary-manywheel / manywheel-py3_9-cuda12_6-test / test (gh)
  Process completed with exit code 1.
• linux-binary-manywheel / manywheel-py3_9-cuda12_8-test / test (gh)
  Process completed with exit code 1.

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / linux-jammy-py3-clang12-executorch / build (gh) (trunk failure)
  Final attempt failed. Child_process exited with error code 1

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

• pull / cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh) (#149370)
  REGRESSION: benchmark ('basic_modules_ListOfLinears_inductor_gpu_force_shape_pad', 'compile_time_instruction_count') failed, actual result 15648280608 is 1.55% higher than expected 15410000000 ±+1.50% if this is an expected regression, please update the expected results.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[cyyever]

@pytorchmergebot merge -r

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased rocm_fix_hipblaslt_workspace onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout rocm_fix_hipblaslt_workspace && git pull --rebase)

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: 3 mandatory check(s) failed. The first few are:

• BC Lint
• pull
• Lint

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

[cyyever]

@pytorchmergebot merge -i

[pytorchmergebot]

Merge started

Your change will be merged while ignoring the following 1 checks: pull / cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[facebook-github-bot]

@pytorchbot revert -m="Diff reverted internally" -c="ghfirst"

This Pull Request has been reverted by a revert inside Meta. To re-land this change, please open another pull request, assign the same reviewers, fix the CI failures that caused the revert and make sure that the failing CI runs on the PR by applying the proper ciflow label (e.g., ciflow/trunk).)

[pytorchmergebot]

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

[pytorchmergebot]

@ethanwee1 your PR has been successfully reverted.

This PR was reopened (likely due to being reverted), so your approval was removed. Please request another review.

[atalman]

lgtm

[atalman]

@pytorchmergebot merge -i

[pytorchmergebot]

Merge started

Your change will be merged while ignoring the following 4 checks: pull / linux-jammy-py3-clang12-executorch / build, pull / cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu), linux-binary-manywheel / manywheel-py3_9-cuda12_6-test / test, linux-binary-manywheel / manywheel-py3_9-cuda12_8-test / test

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: Command git -C /home/runner/work/pytorch/pytorch merge --squash __pull-request-150227__init__ returned non-zero exit code 1

Auto-merging aten/src/ATen/cuda/CUDABlas.cpp
CONFLICT (content): Merge conflict in aten/src/ATen/cuda/CUDABlas.cpp
Squash commit -- not updating HEAD
Automatic merge failed; fix conflicts and then commit the result.

Details for Dev Infra team

Raised by workflow job

[jeffdaily]

@atalman I appreciate you trying to reland this. Thanks, truly. But this PR is only needed if #145130 is relanded.

[eqy]

That handles can be shared between cuBLAS and cuBLASLt is factually true, but I think it is not really relevant here. The cuBLAS handle here is really only used as a key to get a corresponding workspace, and as we do not expect to run a cuBLAS-API backed and cuBLASLt-API backed matmul (on the same stream) at the same time, it's safe to use the workspace that is already allocated for one for the other.

My guess is the real reason the warning shows up on ROCm but not on CUDA is that at present the default CUBLAS_WORKSPACE_CONFIG effective size is always >= the default CUBLASLT_WORKSPACE_SIZE setting. On the CUDA side the intent is to only allocate the cuBLAS workspace and reuse it for Lt, but if Lt requests a larger workspace it precludes this unification.

If you agree with this I think a clearer explanation would be along the lines of "CUDA attempts to share workspaces with the assumption that cuBLAS workspace size >= cuBLASLt workspace size, but as this assumption may not hold on ROCm, we also add a mapping for Lt handle -> workspace in addition to BLAS handle -> workspace."

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[jeffdaily]

This PR is no longer needed.