Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better transformer HF example #2002

Merged
merged 17 commits into from
Dec 6, 2022
Merged

Better transformer HF example #2002

merged 17 commits into from
Dec 6, 2022

Conversation

HamidShojanazeri
Copy link
Collaborator

@HamidShojanazeri HamidShojanazeri commented Nov 23, 2022

Description

This PR will add the Better Transformer optimization to the current HF example. This optimization does not change anything in the model just map the layer names from HF to match expected names in NN.Transformer module where we have faster kernels.

Also, need to mention that the optimized model can not be saved for now, it is numerically stable meaning there is no accuracy degradation after the changes.

Fixes #(issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ x] New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Feature/Issue validation/testing

This requires PT 1.13, will upload the regression test logs.

  • Test A
    Logs for Test A

  • Test B
    Logs for Test B

@codecov
Copy link

codecov bot commented Nov 23, 2022

Codecov Report

Merging #2002 (52998b2) into master (8d877b0) will not change coverage.
The diff coverage is n/a.

❗ Current head 52998b2 differs from pull request most recent head b794b7c. Consider uploading reports for the commit b794b7c to get more accurate results

@@           Coverage Diff           @@
##           master    #2002   +/-   ##
=======================================
  Coverage   53.80%   53.80%           
=======================================
  Files          70       70           
  Lines        3169     3169           
  Branches       56       56           
=======================================
  Hits         1705     1705           
  Misses       1464     1464           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@msaroufim msaroufim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, left some minor nits. Also please run the lint and spellchecking jobs before checking this in

README.md Outdated Show resolved Hide resolved
examples/Huggingface_Transformers/README.md Outdated Show resolved Hide resolved
* [Model parallel inference](examples/Huggingface_Transformers#model-parallelism)
* [Better Transformer for HuggingFace Transformers](examples/Huggingface_Transformers#Speed-up-inference-with-Better-Transformer)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can remove this

Copy link
Member

@msaroufim msaroufim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming all CI is green we're good to go

@HamidShojanazeri HamidShojanazeri changed the title [WIP] Better transformer HF example Better transformer HF example Dec 5, 2022
@lxning lxning merged commit c59c6ac into master Dec 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants