add blackwell support filter for 9.7<=cudnn<9.18.1 - #2775
Conversation
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
Greptile SummaryThis PR makes a one-line C++ clarity fix in Key points:
Confidence Score: 4/5
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["nvte_get_fused_attn_backend (F16/BF16 path)"] --> B{Architecture condition}
B --> C["Cond 1: cuDNN < 8.9.0.3\nAND sm_arch == 80 or 90"]
B --> D["Cond 2: cuDNN >= 8.9.0.3\nAND 80 <= sm_arch < 100\n(Ampere / Ada / Hopper)"]
B --> E["Cond 3 (updated): cuDNN >= 9.7.0\nAND sm_arch >= 100\n(Blackwell only)"]
C --> F[Check further conditions...]
D --> F
E --> F
F --> G{All conditions pass?}
G -- Yes --> H[Return F16_arbitrary_seqlen]
G -- No --> I[Return No_Backend / F16_max512_seqlen]
style E fill:#d4edda,stroke:#28a745
style C fill:#fff3cd,stroke:#ffc107
style D fill:#fff3cd,stroke:#ffc107
Reviews (5): Last reviewed commit: "Merge branch 'main' of github.com:NVIDIA..." | Re-trigger Greptile |
| if ( | ||
| fused_attention_backend == FusedAttnBackend["F16_arbitrary_seqlen"] | ||
| and deterministic | ||
| and (cudnn_version >= (9, 7) and cudnn_version < (9, 18, 1)) | ||
| and device_compute_capability >= (10, 0) | ||
| ): | ||
| logger.debug( | ||
| "Determinism not supported on Blackwell for BF16 with 9.7 <= cuDNN < 9.18.1" | ||
| ) | ||
| use_fused_attention = False | ||
| fused_attention_backend = None |
There was a problem hiding this comment.
Missing
is_training guard — may incorrectly disable FusedAttention for inference
Every other determinism filter in this same if use_fused_attention and deterministic: block guards against is_training (see lines 1070–1080 for FP8 and 1081–1092 for F16_arbitrary_seqlen), conveying that those non-determinism issues are backward-pass–specific. The new Blackwell / cuDNN-version filter does not include and is_training, so it will also disable FusedAttention during deterministic inference on Blackwell GPUs with cuDNN 9.7–9.18.1.
If the cuDNN bug only manifests during training (backward pass), the filter is overly broad and will unnecessarily fall back to a slower backend during inference. If it truly affects the forward pass as well, a comment explaining that would help reviewers and future maintainers understand the deviation from the existing pattern.
Consider either:
if (
fused_attention_backend == FusedAttnBackend["F16_arbitrary_seqlen"]
and is_training
and (cudnn_version >= (9, 7) and cudnn_version < (9, 18, 1))
and device_compute_capability >= (10, 0)
):or, if inference is also affected, add a comment explaining why is_training is deliberately omitted.
There was a problem hiding this comment.
updated, check again
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
…p8_determinism_check
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
…p8_determinism_check
* add blackwell support filter for 9.7<=cudnn<9.18.1 Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * simplify conditionals Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * fix conditionals again Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * fix conditionals again Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * update the error log Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * remove the python filter and correct the cpp filter Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> --------- Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* add blackwell support filter for 9.7<=cudnn<9.18.1 Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * simplify conditionals Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * fix conditionals again Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * fix conditionals again Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * update the error log Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * remove the python filter and correct the cpp filter Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> --------- Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Description
Please include a brief summary of the changes, relevant motivation and context.
Fixes # (issue)
Type of change
Changes
Please list the changes introduced in this PR:
Checklist: