Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Image GPT #108

Merged
merged 2 commits into from Jul 11, 2020
Merged

Fix Image GPT #108

merged 2 commits into from Jul 11, 2020

Conversation

teddykoker
Copy link
Contributor

@teddykoker teddykoker commented Jul 9, 2020

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Fixes Image GPT. Images were not being properly quantized before. See code:

    def forward(self, x, classify=False):
        x = _shape_input(x)

        # TODO(teddykoker): this is a hack to quantize images into `vocab_size` bins.
        # This only works with 1 channel images; something like KNN needs to be used
        # for RGB. Assumes data is in [0.0, 1.0].
        x = torch.round(x * (self.hparams.vocab_size - 1)).long()

        return self.gpt(x, classify)

Also fixes docs.

Review

@williamFalcon

Discussion

Implementing KNN for image quantization is fairly straight forward. I would be willing to implement at some point. But I am unsure where this code would go as is it is an independent preprocessing step that just needs to be run once (in practice I would just create my own dataset for the new quantized images).

@mergify mergify bot requested a review from Borda July 9, 2020 16:35
@codecov-commenter
Copy link

codecov-commenter commented Jul 9, 2020

Codecov Report

Merging #108 into master will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #108   +/-   ##
=======================================
  Coverage   91.91%   91.91%           
=======================================
  Files          77       77           
  Lines        3944     3944           
=======================================
  Hits         3625     3625           
  Misses        319      319           
Flag Coverage Δ
#unittests 91.91% <100.00%> (ø)
Impacted Files Coverage Δ
pl_bolts/models/vision/image_gpt/gpt2.py 100.00% <100.00%> (ø)
pl_bolts/models/vision/image_gpt/igpt_module.py 98.78% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 024b574...17048c6. Read the comment docs.

@williamFalcon williamFalcon merged commit 6cc662e into Lightning-Universe:master Jul 11, 2020
@Borda Borda added the fix fixing issues... label Sep 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix fixing issues...
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants