Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set the dtype correctly for vision GPT model #694

Merged
merged 2 commits into from Jul 26, 2021
Merged

Set the dtype correctly for vision GPT model #694

merged 2 commits into from Jul 26, 2021

Conversation

SeanNaren
Copy link
Contributor

@SeanNaren SeanNaren commented Jul 26, 2021

What does this PR do?

Closes Lightning-AI/pytorch-lightning#8125

We need to ensure the tokens are in the same type or they will be casted to the default FloatTensor.

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests? [not needed for typos/docs]
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link

codecov bot commented Jul 26, 2021

Codecov Report

Merging #694 (5ae8247) into master (ab801aa) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #694   +/-   ##
=======================================
  Coverage   72.64%   72.64%           
=======================================
  Files         119      119           
  Lines        7356     7356           
=======================================
  Hits         5344     5344           
  Misses       2012     2012           
Flag Coverage Δ
cpu 72.64% <100.00%> (ø)
pytest 72.64% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pl_bolts/models/vision/image_gpt/gpt2.py 96.15% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab801aa...5ae8247. Read the comment docs.

@SeanNaren SeanNaren changed the title Set the dtype correctly Set the dtype correctly for vision GPT model Jul 26, 2021
Copy link

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@SeanNaren SeanNaren enabled auto-merge (squash) July 26, 2021 11:18
@Borda Borda disabled auto-merge July 26, 2021 18:38
@Borda Borda merged commit e22b8d0 into master Jul 26, 2021
@Borda Borda deleted the feat/fix_fp16 branch July 26, 2021 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RuntimeError: expected scalar type Float but found Half .. : when training image-gpt model with deepspeed
5 participants