Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce binary size, add sm_89 and sm_90 targets #1383

Merged
merged 6 commits into from
Apr 4, 2024
Merged

Reduce binary size, add sm_89 and sm_90 targets #1383

merged 6 commits into from
Apr 4, 2024

Conversation

lzhangzz
Copy link
Collaborator

@lzhangzz lzhangzz commented Apr 2, 2024 •
edited

  • Stop building FP32 variant
  • Remove unused kernels
  • Skip building flash attention kernels when test is not enabled
  • Add sm_89 (Ada) and sm_90 (Hopper) target, we now have 70, 75, 80, 86, 89 and 90 by default
  • Drop all PTX as JIT compilation for all the kernels is impractical

@lzhangzz lzhangzz removed the WIP label Apr 2, 2024
@lvhan028 lvhan028 requested a review from irexyc April 2, 2024 12:04
@lzhangzz lzhangzz changed the title Reduce binary size Reduce binary size, add sm_89 and sm_90 targets Apr 3, 2024
@lvhan028 lvhan028 merged commit 620236d into InternLM:main Apr 4, 2024
9 checks passed
@zhyncs
Copy link
Contributor

zhyncs commented Apr 7, 2024

  • Skip building flash attention kernels when test is not enabled

ref #1348

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lvhan028 lvhan028 lvhan028 approved these changes

@irexyc irexyc Awaiting requested review from irexyc

Assignees
No one assigned
Labels
improvement turbomind
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@lzhangzz @zhyncs @lvhan028