Skip to content

[Op] Add attention and bias_gelu ops#41

Merged
comaniac merged 7 commits intoawslabs:mainfrom
comaniac:op
Feb 7, 2023
Merged

[Op] Add attention and bias_gelu ops#41
comaniac merged 7 commits intoawslabs:mainfrom
comaniac:op

Conversation

@comaniac
Copy link
Contributor

@comaniac comaniac commented Feb 6, 2023

Description

This is a prerequisite PR for adding HF GPT-2 schedule.

  • Implement attention ops that use flash-attention and xformrs.
  • Implement bias_gelu ops that use torchscript or torch compiler.
  • Change the GPT-Neo schedule to use these ops, so that now GPT-Neo schedule doesn't depend on epoi anymore. Later we will update the schedules of other example models accordingly.
  • [Test] Add -rxXs to let pytest print reasons of skipped tests.
  • [Docker] Update flash-attention commit hash which improves the kernel performance by ~12%. CI image is not updated because this change doesn't impact the functionality.

Checklist

  • PR's title starts with a category (e.g. [Bugfix], [Model], [Tutorial], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

@szhengac @chhzh123

@comaniac comaniac merged commit d2dbaeb into awslabs:main Feb 7, 2023
@comaniac
Copy link
Contributor Author

comaniac commented Feb 7, 2023

Thanks @chhzh123

@comaniac comaniac deleted the op branch February 7, 2023 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants