Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use kl_coeff in loss function calculation #330

Merged
merged 2 commits into from
Nov 2, 2020

Conversation

miccio-dk
Copy link
Contributor

What does this PR do?

The VAE model contained here takes an optional kl_coeff parameter that's supposed to be a scaling factor for the KL term of the variational loss. This might be useful to avoid the "posterior collapse phenomenon". This pull request makes sure that it's used in the loss function.

Fixes #328.

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
  • Did you make sure to update the documentation with your changes? not needed
  • Did you write any new necessary tests? not needed
  • Did you verify new and existing tests pass locally with your changes? no changes in tests
  • If you made a notable change (that affects users), did you update the CHANGELOG? it's a bug fix, no new feature

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Absolutely! =)

@codecov
Copy link

codecov bot commented Nov 2, 2020

Codecov Report

Merging #330 into master will increase coverage by 0.05%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #330      +/-   ##
==========================================
+ Coverage   82.18%   82.24%   +0.05%     
==========================================
  Files          98       98              
  Lines        5491     5492       +1     
==========================================
+ Hits         4513     4517       +4     
+ Misses        978      975       -3     
Flag Coverage Δ
cpu 24.24% <0.00%> (-0.01%) ⬇️
pytest 24.24% <0.00%> (-0.01%) ⬇️
unittests 81.62% <100.00%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ts/models/autoencoders/basic_ae/basic_ae_module.py 88.00% <ø> (ø)
.../models/autoencoders/basic_vae/basic_vae_module.py 91.66% <100.00%> (+0.08%) ⬆️
...l_bolts/models/rl/vanilla_policy_gradient_model.py 94.64% <0.00%> (+2.67%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 978fa1c...b686780. Read the comment docs.

Copy link
Contributor

@akihironitta akihironitta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you for the swift action!

@miccio-dk
Copy link
Contributor Author

My pleasure! Excuse the weird question, but would you be able to give me an estimate of when this will be included in a minor release? Thanks! :)

@ananyahjha93
Copy link
Contributor

we'll have a release as soon as we are done with simclr fixes, which should be a couple of days max.

Copy link
Contributor

@teddykoker teddykoker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

On the VAE model, kl_coeff is never applied in the loss function
5 participants