[CUDA] [CI] Disable flash attention for sm87 architecture when the head dim > 192 #117678

nWEIdia · 2024-01-17T19:18:05Z

Head dim > 192 requires A100/H100 (sm80 or sm90) per TORCH_CHECK here.

cc @eqy @ptrblck @malfet

pytorch-bot · 2024-01-17T19:18:08Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/117678

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit d8d0b80 with merge base 25f7219 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

linux-foundation-easycla · 2024-01-17T19:18:09Z

The committers listed above are authorized under a signed CLA.

✅ login: nWEIdia / name: Wei Wang (2179e64, 607cf55, 6d94b05, d8d0b80)

nWEIdia · 2024-01-19T02:03:46Z

@pytorchbot merge

pytorch-bot · 2024-01-19T02:03:52Z

Please seek CI approval before scheduling CIFlow labels

pytorchmergebot · 2024-01-19T02:05:59Z

Merge failed

Reason: Approvers from one of the following sets are needed:

superuser (pytorch/metamates)
Core Reviewers (mruberry, lezcano, Skylion007, ngimel, peterbell10)
Core Maintainers (soumith, gchanan, ezyang, dzhulgakov, malfet)

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

nWEIdia · 2024-01-19T05:11:20Z

@malfet Could you please help review and approve the workflow runs? Thank you!

malfet

Sure, though name perhaps be changed to like isSM8X

nWEIdia · 2024-01-19T05:19:10Z

Sure, though name perhaps be changed to like isSM8X

Thank you for your prompt response! I will do the variable name simplification.

nWEIdia · 2024-01-19T05:46:19Z

Variable renaming done. Looks like approval to run workflows is required again.

malfet · 2024-01-19T07:16:54Z

@pytorchbot merge

pytorchmergebot · 2024-01-19T07:19:01Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-01-19T07:25:42Z

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

pull / linux-focal-py3-clang9-android-ndk-r21e-gradle-custom-build-single / build-and-test (default, 1, 1, linux.2xlarge)

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

nWEIdia · 2024-01-23T21:20:18Z

@pytorchbot merge -f "Android Failure does not seem related"

pytorch-bot · 2024-01-23T21:20:22Z

You are not authorized to force merges to this repository. Please use the regular @pytorchmergebot merge command instead

nWEIdia · 2024-01-23T21:20:45Z

@pytorchmergebot merge

pytorchmergebot · 2024-01-23T21:22:49Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-01-23T21:23:01Z

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

pull / linux-focal-py3-clang9-android-ndk-r21e-gradle-custom-build-single / build-and-test (default, 1, 1, linux.2xlarge)

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

nWEIdia · 2024-01-26T18:19:00Z

@pytorchbot merge

pytorchmergebot · 2024-01-26T18:21:12Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-01-26T18:21:26Z

Merge failed

Reason: 2 mandatory check(s) failed. The first few are:

pull
Lint

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

nWEIdia · 2024-01-26T18:24:40Z

@malfet Could you please help approve the workflow again? Thank you!

eqy · 2024-01-26T20:20:52Z

@pytorchmergebot merge

pytorchmergebot · 2024-01-26T20:22:52Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-01-26T21:28:03Z

Merge failed

Reason: 1 jobs have failed, first few of them are: Check mergeability and dependencies for ghstack prs / ghstack-mergeability-check

Details for Dev Infra team

Raised by workflow job

eqy · 2024-01-26T22:11:17Z

@pytorchmergebot rebase

pytorchmergebot · 2024-01-26T22:13:02Z

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

requires A100/H100 (sm80 or sm90).

update comments

pytorchmergebot · 2024-01-26T22:13:05Z

Successfully rebased disable_flash_attention_for_sm87 onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout disable_flash_attention_for_sm87 && git pull --rebase)

nWEIdia · 2024-01-27T01:18:44Z

@pytorchmergebot merge

nWEIdia · 2024-01-27T01:20:20Z

@pytorchmergebot merge

pytorchmergebot · 2024-01-27T01:22:20Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

nWEIdia · 2024-01-27T01:28:28Z

It is in, thank you @eqy and @malfet !

…ad dim > 192 (pytorch#117678) Head dim > 192 requires A100/H100 (sm80 or sm90) per TORCH_CHECK [here](https://github.com/pytorch/pytorch/blob/0c26565d5d76c371529320fd6176268028516a10/aten/src/ATen/native/transformers/cuda/flash_attn/flash_api.cpp#L760). Pull Request resolved: pytorch#117678 Approved by: https://github.com/eqy, https://github.com/malfet

pytorch-bot bot added the topic: not user facing topic category label Jan 17, 2024

pytorchbot added the open source label Jan 17, 2024

eqy approved these changes Jan 17, 2024

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jan 19, 2024

pytorch-bot bot removed the ciflow/trunk Trigger trunk jobs on your pull request label Jan 19, 2024

pytorchmergebot added the merging label Jan 19, 2024

pytorchmergebot removed the merging label Jan 19, 2024

malfet approved these changes Jan 19, 2024

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jan 19, 2024

pytorchmergebot added the merging label Jan 19, 2024

pytorchmergebot removed the merging label Jan 19, 2024

pytorchmergebot added the merging label Jan 23, 2024

pytorchmergebot removed the merging label Jan 23, 2024

nWEIdia force-pushed the disable_flash_attention_for_sm87 branch from d6e214d to 98b983e Compare January 23, 2024 21:31

pytorchmergebot removed the merging label Jan 26, 2024

pytorchmergebot added the merging label Jan 26, 2024

pytorchmergebot removed the merging label Jan 26, 2024

pytorchmergebot added the merging label Jan 26, 2024

pytorchmergebot removed the merging label Jan 26, 2024

nWEIdia and others added 4 commits January 26, 2024 22:13

Disable flash attention for sm87 architecture since the head dim > 192

2179e64

requires A100/H100 (sm80 or sm90).

cosmetic fix

607cf55

Simplify SM8687or89 -> SM8X

6d94b05

Update test_transformers.py

d8d0b80

update comments

pytorchmergebot force-pushed the disable_flash_attention_for_sm87 branch from 21a1cdb to d8d0b80 Compare January 26, 2024 22:13

pytorchmergebot added the merging label Jan 27, 2024

pytorchmergebot closed this in 80cb6db Jan 27, 2024

pytorchmergebot added Merged and removed merging labels Jan 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CUDA] [CI] Disable flash attention for sm87 architecture when the head dim > 192 #117678

[CUDA] [CI] Disable flash attention for sm87 architecture when the head dim > 192 #117678

nWEIdia commented Jan 17, 2024 •

edited

Loading

pytorch-bot bot commented Jan 17, 2024 •

edited

Loading

linux-foundation-easycla bot commented Jan 17, 2024 •

edited

Loading

nWEIdia commented Jan 19, 2024

pytorch-bot bot commented Jan 19, 2024

pytorchmergebot commented Jan 19, 2024

nWEIdia commented Jan 19, 2024

malfet left a comment

nWEIdia commented Jan 19, 2024

nWEIdia commented Jan 19, 2024

malfet commented Jan 19, 2024

pytorchmergebot commented Jan 19, 2024

pytorchmergebot commented Jan 19, 2024

nWEIdia commented Jan 23, 2024

pytorch-bot bot commented Jan 23, 2024

nWEIdia commented Jan 23, 2024

pytorchmergebot commented Jan 23, 2024

pytorchmergebot commented Jan 23, 2024

nWEIdia commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

nWEIdia commented Jan 26, 2024

eqy commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

eqy commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

nWEIdia commented Jan 27, 2024 •

edited

Loading

nWEIdia commented Jan 27, 2024

pytorchmergebot commented Jan 27, 2024

nWEIdia commented Jan 27, 2024

[CUDA] [CI] Disable flash attention for sm87 architecture when the head dim > 192 #117678

[CUDA] [CI] Disable flash attention for sm87 architecture when the head dim > 192 #117678

Conversation

nWEIdia commented Jan 17, 2024 • edited Loading

pytorch-bot bot commented Jan 17, 2024 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/117678

✅ No Failures

linux-foundation-easycla bot commented Jan 17, 2024 • edited Loading

nWEIdia commented Jan 19, 2024

pytorch-bot bot commented Jan 19, 2024

pytorchmergebot commented Jan 19, 2024

Merge failed

nWEIdia commented Jan 19, 2024

malfet left a comment

Choose a reason for hiding this comment

nWEIdia commented Jan 19, 2024

nWEIdia commented Jan 19, 2024

malfet commented Jan 19, 2024

pytorchmergebot commented Jan 19, 2024

Merge started

pytorchmergebot commented Jan 19, 2024

Merge failed

nWEIdia commented Jan 23, 2024

pytorch-bot bot commented Jan 23, 2024

nWEIdia commented Jan 23, 2024

pytorchmergebot commented Jan 23, 2024

Merge started

pytorchmergebot commented Jan 23, 2024

Merge failed

nWEIdia commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

Merge started

pytorchmergebot commented Jan 26, 2024

Merge failed

nWEIdia commented Jan 26, 2024

eqy commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

Merge started

pytorchmergebot commented Jan 26, 2024

Merge failed

eqy commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

pytorchmergebot commented Jan 26, 2024

nWEIdia commented Jan 27, 2024 • edited Loading

nWEIdia commented Jan 27, 2024

pytorchmergebot commented Jan 27, 2024

Merge started

nWEIdia commented Jan 27, 2024

nWEIdia commented Jan 17, 2024 •

edited

Loading

pytorch-bot bot commented Jan 17, 2024 •

edited

Loading

linux-foundation-easycla bot commented Jan 17, 2024 •

edited

Loading

nWEIdia commented Jan 27, 2024 •

edited

Loading