[cuDNN][cuDNN V8 API] Improve hot path heuristics performance in V8

[eqy]

Small optimization for the hot path when thrashing the cache with dynamic shapes; in most cases we don't need the fallback generator so we can omit it unless needed later.

CC @ptrblck @ngimel

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90811

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 Failures

As of commit 4f1247b:

NEW FAILURES - The following jobs have failed:

• linux-focal-rocm5.3-py3.8-distributed / test (distributed, 1, 2, linux.rocm.gpu)
• linux-focal-rocm5.3-py3.8-distributed / test (distributed, 2, 2, linux.rocm.gpu)

FLAKY - The following jobs failed but were likely due to flakiness present on master:

• linux-focal-rocm5.3-py3.8 / test (default, 1, 2, linux.rocm.gpu)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[eqy]

@pytorchmergebot rebase

[pytorchmergebot]

@pytorchbot successfully started a rebase job. Check the current status here

[pytorchmergebot]

Successfully rebased cudnn_v8_hot onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout cudnn_v8_hot && git pull --rebase)

[eqy]

@pytorchmergebot rebase

[pytorchmergebot]

@pytorchbot successfully started a rebase job. Check the current status here

[pytorchmergebot]

Successfully rebased cudnn_v8_hot onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout cudnn_v8_hot && git pull --rebase)

[eqy]

@pytorchmergebot merge -f "failures unrelated"

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here