Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Flash Attention forward for Llama 2: #3595

Merged
merged 2 commits into from Jul 23, 2023
Merged

Update Flash Attention forward for Llama 2: #3595

merged 2 commits into from Jul 23, 2023

Conversation

jordiclive
Copy link
Collaborator

@jordiclive jordiclive commented Jul 23, 2023 •
edited

GQA for 34B and 70B and tp have been added

The current flash attn patch forward is for the old transformers' attention forward.

andreaskoepf
andreaskoepf approved these changes Jul 23, 2023
View reviewed changes
Copy link
Collaborator

@andreaskoepf andreaskoepf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool thanks a lot!

@andreaskoepf andreaskoepf merged commit 3c8f93e into LAION-AI:main Jul 23, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andreaskoepf andreaskoepf andreaskoepf approved these changes

@theblackcat102 theblackcat102 Awaiting requested review from theblackcat102 theblackcat102 is a code owner

@sanagno sanagno Awaiting requested review from sanagno sanagno is a code owner

@dvruette dvruette Awaiting requested review from dvruette dvruette is a code owner

@yk yk Awaiting requested review from yk yk is a code owner

@shahules786 shahules786 Awaiting requested review from shahules786 shahules786 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@jordiclive @andreaskoepf