Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Log tflops and other fixes #33

Merged
merged 11 commits into from
Mar 22, 2023
Merged

Log tflops and other fixes #33

merged 11 commits into from
Mar 22, 2023

Conversation

RaymondLi0
Copy link
Collaborator

@RaymondLi0 RaymondLi0 commented Mar 10, 2023 •
edited
Loading

@RaymondLi0 RaymondLi0 changed the title Log tflops Log tflops and other fixes Mar 10, 2023
NouamaneTazi
NouamaneTazi reviewed Mar 17, 2023
View reviewed changes
megatron/utils.py Outdated Show resolved Hide resolved
NouamaneTazi
NouamaneTazi reviewed Mar 17, 2023
View reviewed changes
megatron/utils.py Outdated Show resolved Hide resolved
Base automatically changed from multi-validation to multi-query-attention March 21, 2023 17:25
@RaymondLi0
Copy link
Collaborator Author

RaymondLi0 commented Mar 21, 2023 •
edited
Loading

Thank you for the suggestion @NouamaneTazi !
In b18ecf6 I adjusted the formula that's in the comments, could you confirm that this is correct?

@NouamaneTazi
Copy link

LGTM! You might wanna check the # TODO: maybe tp_size factor missing here depending on how you implemented MQA

@RaymondLi0
Copy link
Collaborator Author

I left the todo in the comments. Let's merge this and address this later.

@RaymondLi0 RaymondLi0 merged commit 8b38744 into multi-query-attention Mar 22, 2023
@RaymondLi0 RaymondLi0 deleted the log-tflops branch March 22, 2023 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NouamaneTazi NouamaneTazi NouamaneTazi left review comments

@loubnabnl loubnabnl Awaiting requested review from loubnabnl

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Log GPU throughput
2 participants
@RaymondLi0 @NouamaneTazi