bugfix: batch_size parameter for DataModules remaining #344

hecoding · 2020-11-06T18:28:14Z

What does this PR do?

Fixes #334
This PR fixes data modules the same way as PR #331. On issue #334 a number of modules are listed apart from MNIST one, so all them are now fixed and their docstrings updated.

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

codecov · 2020-11-06T18:29:46Z

Codecov Report

Merging #344 (b4124bf) into master (2e903c3) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #344   +/-   ##
=======================================
  Coverage   81.09%   81.09%           
=======================================
  Files         100      100           
  Lines        5722     5722           
=======================================
  Hits         4640     4640           
  Misses       1082     1082

Flag	Coverage Δ
cpu	`25.24% <0.00%> (ø)`
pytest	`25.24% <0.00%> (ø)`
unittests	`80.46% <100.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pl_bolts/models/regression/linear_regression.py	`98.50% <100.00%> (ø)`
pl_bolts/models/regression/logistic_regression.py	`95.89% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e903c3...b4124bf. Read the comment docs.

Borda · 2020-11-09T17:35:07Z

In principle it looks fine, just fix falling tests:

FAILED pl_bolts/datamodules/sklearn_datamodule.py::pl_bolts.datamodules.sklearn_datamodule.SklearnDataModule
FAILED tests/models/test_executable_scripts.py::test_cli_run_lin_regression[--max_epochs 1 --max_steps 2]
FAILED tests/models/test_executable_scripts.py::test_cli_run_log_regression[--max_epochs 1 --max_steps 2]

hecoding · 2020-11-10T01:11:57Z

Oops my bad. I think everything's fine now.

akihironitta

Thank you for your contribution!

pl_bolts/datamodules/binary_mnist_datamodule.py

annikabrundyn · 2020-11-18T00:10:35Z

Aside from keeping the default_transforms as internal, everything else looks good

hecoding · 2020-11-19T17:53:31Z

Ok fixed that thanks, I think that's it.

hecoding · 2020-11-26T14:58:21Z

ping @annikabrundyn

annikabrundyn · 2020-11-26T17:57:55Z

looks great! thanks for the contribution @hecoding :)

hecoding · 2020-11-26T18:33:35Z

Awesome! Welcome guys.

cc @Borda for merging

@akihironitta

thanks @akihironitta Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

akihironitta · 2020-12-01T16:22:08Z

@hecoding Thank you :]

@akihironitta

…verse#344) * bugfix: batch_size for DataModules remaining * Update sklearn datamodule tests * Fix default_transforms. Keep internal for every data module * fix typo on binary_mnist_datamodule thanks @akihironitta Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

@akihironitta

…verse#344) * bugfix: batch_size for DataModules remaining * Update sklearn datamodule tests * Fix default_transforms. Keep internal for every data module * fix typo on binary_mnist_datamodule thanks @akihironitta Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

@akihironitta

…verse#344) * bugfix: batch_size for DataModules remaining * Update sklearn datamodule tests * Fix default_transforms. Keep internal for every data module * fix typo on binary_mnist_datamodule thanks @akihironitta Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

@akihironitta

* Add DCGAN module * Undo black on conf.py * Add tests for DCGAN * Fix flake8 and codefactor * Add types and small refactoring * Make image sampler callback work * Upgrade DQN to use .log (#404) * Upgrade DQN to use .log * remove unused * pep8 * fixed other dqn * fix loss test case for batch size variation (#402) * Decouple DataModules from Models - CPCV2 (#386) * Decouple dms from CPCV2 * Update tests * Add docstrings, fix import, and update changelog * Update transforms * bugfix: batch_size parameter for DataModules remaining (#344) * bugfix: batch_size for DataModules remaining * Update sklearn datamodule tests * Fix default_transforms. Keep internal for every data module * fix typo on binary_mnist_datamodule thanks @akihironitta Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> * Fix a typo/copy paste error (#415) * Just a Typo (#413) missing a ' at the end of dataset='stl10 * Remove unused arguments (#418) * tests: Use cached datasets in LitMNIST and the doctests (#414) * Use cached datasets * Use cached datasets in doctests * clear replay buffer after trajectory (#425) * stale: update label * bugfix: Add missing imports to pl_bolts/__init__.py (#430) * Add missing imports * Add missing imports * Apply isort * Fix CIFAR num_samples (#432) * Add static type checker mypy to the tests and pre-commit hooks (#433) * Add mypy check to GitHub Actions * Run mypy on pl_bolts only * Add mypy check to pre-commit * Add an empty line at the end of files * Update mypy config * Update mypy config * Update mypy config * show Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * missing logo * Add type annotations to pl_bolts/__init__.py (#435) * Run mypy on pl_bolts only * Update mypy config * Add type hints to pl_bolts/__init__.py * mypy Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz> * skip hanging (#437) * Option to normalize latent interpolation images (#438) * add option to normalize latent interpolation images * linspace * update Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai> * 0.2.6rc1 * Warnings fix (#449) * Revert "Merge pull request #1 from ganprad/warnings_fix" This reverts commit 7c5aaf0. * Fixes warning related np.integer in SklearnDataModule Fixes this warning: ```DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32```` * Refactor datamodules/datasets (#338) * Remove try: ... except: ... * Fix experience_source * Fix imagenet * Fix kitti * Fix sklearn * Fix vocdetection * Fix typo * Remove duplicate * Fix by flake8 * Add optional packages availability vars * binary_mnist * Use pl_bolts._SKLEARN_AVAILABLE * Apply isort * cifar10 * mnist * cityscapes * fashion mnist * ssl_imagenet * stl10 * cifar10 * dummy * fix city * fix stl10 * fix mnist * ssl_amdim * remove unused DataLoader and fix docs * use from ... import ... * fix pragma: no cover * Fix forward reference in annotations * binmnist * Same order as imports * Move vars from __init__ to utils/__init__ * Remove vars from __init__ * Update vars * Apply isort * update min requirements - PL 1.1.1 (#448) * update min requirements * rc0 * imports * isort * flake8 * 1.1.1 * flake8 * docs * Add missing optional packages to `requirements/*.txt` (#450) * Import matplotlib at the top * Add missing optional packages * Update wandb * Add mypy to requirements * update Isort (#457) * Adding flags to datamodules (#388) * Adding flags to datamodules * Finishing up changes * Fixing syntax error * More syntax errors * More * Adding drop_last flag to sklearn test * Adding drop_last flag to sklearn test * Updating doc for reflect drop_last=False * Adding flags to datamodules * Finishing up changes * Fixing syntax error * More syntax errors * More * Adding drop_last flag to sklearn test * Adding drop_last flag to sklearn test * Updating doc for reflect drop_last=False * Cleaning up parameters and docstring * Fixing syntax error * Fixing documentation * Hardcoding shuffle=False for val and test * Add DCGAN module * Small fixes * Remove DataModules * Update docs * Update docs * Update torchvision import * Import gym as optional package to build docs successfully (#458) * Import gym as optional package * Fix import * Apply isort * bugfix: batch_size parameter for DataModules remaining (#344) * bugfix: batch_size for DataModules remaining * Update sklearn datamodule tests * Fix default_transforms. Keep internal for every data module * fix typo on binary_mnist_datamodule thanks @akihironitta Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> * Option to normalize latent interpolation images (#438) * add option to normalize latent interpolation images * linspace * update Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai> * update min requirements - PL 1.1.1 (#448) * update min requirements * rc0 * imports * isort * flake8 * 1.1.1 * flake8 * docs * Apply suggestions from code review * Apply suggestions from code review * Add docs * Use LSUN instead of CIFAR10 * Update TensorboardGenerativeModelImageSampler * Update docs with lsun * Update test * Revert TensorboardGenerativeModelImageSampler changes * Remove ModelCheckpoint callback and nrow=5 arg * Apply suggestions from code review * Fix test_dcgan * Apply yapf * Apply suggestions from code review Co-authored-by: Teddy Koker <teddy.koker@gmail.com> Co-authored-by: Sidhant Sundrani <sidhant96@outlook.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> Co-authored-by: Héctor Laria <hector_laria@hotmail.com> Co-authored-by: Bartol Karuza <bartol.k@gmail.com> Co-authored-by: Happy Sugar Life <777Jonathansum@gmail.com> Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz> Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai> Co-authored-by: Pradeep Ganesan <ganprad@users.noreply.github.com> Co-authored-by: Brian Ko <briankosw@gmail.com> Co-authored-by: Christoph Clement <christoph.clement@artorg.unibe.ch>

Borda requested a review from annikabrundyn November 9, 2020 16:57

Borda added the fix fixing issues... label Nov 9, 2020

Borda approved these changes Nov 9, 2020

View reviewed changes

Borda added datamodule Anything related to datamodules model labels Nov 9, 2020

akihironitta reviewed Nov 11, 2020

View reviewed changes

pl_bolts/datamodules/binary_mnist_datamodule.py Outdated Show resolved Hide resolved

pl_bolts/datamodules/binary_mnist_datamodule.py Outdated Show resolved Hide resolved

annikabrundyn approved these changes Nov 26, 2020

View reviewed changes

hecoding and others added 4 commits November 26, 2020 20:22

bugfix: batch_size for DataModules remaining

3c2e4ec

Update sklearn datamodule tests

afca45d

Fix default_transforms. Keep internal for every data module

08ccd0c

fix typo on binary_mnist_datamodule

b4124bf

thanks @akihironitta Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

Borda force-pushed the fix/datamodules_bsize branch from c6aab93 to b4124bf Compare November 26, 2020 19:22

akihironitta approved these changes Nov 27, 2020

View reviewed changes

akihironitta mentioned this pull request Dec 1, 2020

All data modules don't support providing batch size in the constructor. #417

Closed

akihironitta merged commit 7c2e651 into Lightning-Universe:master Dec 1, 2020

Borda added this to the v0.3 milestone Jan 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bugfix: batch_size parameter for DataModules remaining #344

bugfix: batch_size parameter for DataModules remaining #344

hecoding commented Nov 6, 2020

codecov bot commented Nov 6, 2020 •

edited

Borda commented Nov 9, 2020

hecoding commented Nov 10, 2020

akihironitta left a comment

annikabrundyn commented Nov 18, 2020

hecoding commented Nov 19, 2020

hecoding commented Nov 26, 2020

annikabrundyn commented Nov 26, 2020

hecoding commented Nov 26, 2020

akihironitta commented Dec 1, 2020

bugfix: batch_size parameter for DataModules remaining #344

bugfix: batch_size parameter for DataModules remaining #344

Conversation

hecoding commented Nov 6, 2020

What does this PR do?

Before submitting

PR review

Did you have fun?

codecov bot commented Nov 6, 2020 • edited

Codecov Report

Borda commented Nov 9, 2020

hecoding commented Nov 10, 2020

akihironitta left a comment

Choose a reason for hiding this comment

annikabrundyn commented Nov 18, 2020

hecoding commented Nov 19, 2020

hecoding commented Nov 26, 2020

annikabrundyn commented Nov 26, 2020

hecoding commented Nov 26, 2020

akihironitta commented Dec 1, 2020

codecov bot commented Nov 6, 2020 •

edited