New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Performance optimization for language generation #15

Merged

guillaume-be merged 1 commit into master from generation_optimization

Mar 17, 2020

Owner

guillaume-be commented Mar 17, 2020

Optimization of the performance for language generation:

skip gradients calculation (~10% performance benefit)
allow for batched prompts generation. Sequences are left-padded with the pad token when present, unknown token otherwise.


          Performance optimization for language generation

344e27c

guillaume-be merged commit d43c58a into master

guillaume-be deleted the generation_optimization branch

May 1, 2020 16:16

sachaarbonel mentioned this pull request

Support for OpenAIGPTDoubleHeadsModel, GPT2DoubleHeadsModel, SequenceSummary,AdamW #52

Closed

ycat3 mentioned this pull request

GPT2 model for T5Tokenizer does not work. #221

Closed

SorooshMortazavi mentioned this pull request

cargo run doesn`t work as expected! #240

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment