Skip to content

Minor fixes for attention#1504

Merged
timmoon10 merged 2 commits into
NVIDIA:mainfrom
cyanguwa:FA_minor_fixes
Feb 25, 2025
Merged

Minor fixes for attention#1504
timmoon10 merged 2 commits into
NVIDIA:mainfrom
cyanguwa:FA_minor_fixes

Conversation

@cyanguwa
Copy link
Copy Markdown
Collaborator

@cyanguwa cyanguwa commented Feb 25, 2025

Description

This PR fixes two things:

  • allow flash-attn 2.7.4 to run on sm100+, and allow head_dim=256 on sm100+
  • include sm120 in certain sm100+ restrictions that were left out (related to head_dim and bottom right causal mask)

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

See above.

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@cyanguwa
minor fixes for attention
94f42bf 
Signed-off-by: Charlene Yang <charleney@nvidia.com>
@cyanguwa cyanguwa requested a review from timmoon10 February 25, 2025 00:03
@cyanguwa
Copy link
Copy Markdown
Collaborator Author

cyanguwa commented Feb 25, 2025

/te-ci pytorch L0

timmoon10
timmoon10 approved these changes Feb 25, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@timmoon10 timmoon10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@timmoon10 timmoon10 closed this Feb 25, 2025
@timmoon10 timmoon10 reopened this Feb 25, 2025
@timmoon10 timmoon10 merged commit 8744188 into NVIDIA:main Feb 25, 2025
timmoon10 pushed a commit that referenced this pull request Feb 26, 2025
@cyanguwa @pre-commit-ci @timmoon10
Minor fixes for attention (#1504)
6266011 
* minor fixes for attention

Signed-off-by: Charlene Yang <charleney@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Charlene Yang <charleney@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@hungryGeek16 hungryGeek16 mentioned this pull request May 31, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@timmoon10 timmoon10 timmoon10 approved these changes

Assignees

No one assigned

Labels

2.1.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@cyanguwa @timmoon10