Add Pix2Pix model #533

aniketmaurya · 2021-01-21T19:45:49Z

What does this PR do?

Fixes #512

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests? [not needed for typos/docs]
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

codecov · 2021-01-21T19:47:49Z

Codecov Report

Merging #533 (1b2b47a) into master (86e3f52) will decrease coverage by 1.07%.
The diff coverage is 21.53%.

@@            Coverage Diff             @@
##           master     #533      +/-   ##
==========================================
- Coverage   77.58%   76.51%   -1.08%     
==========================================
  Files         115      117       +2     
  Lines        6701     6829     +128     
==========================================
+ Hits         5199     5225      +26     
- Misses       1502     1604     +102

Flag	Coverage Δ
cpu	`26.12% <21.53%> (-0.12%)`	⬇️
pytest	`26.12% <21.53%> (-0.12%)`	⬇️
unittests	`75.99% <21.53%> (-1.07%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pl_bolts/models/gans/pix2pix/components.py	`17.94% <17.94%> (ø)`
pl_bolts/models/gans/pix2pix/pix2pix_module.py	`22.44% <22.44%> (ø)`
pl_bolts/models/gans/__init__.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 86e3f52...66317f4. Read the comment docs.

pep8speaks · 2021-01-21T20:23:31Z

Hello @aniketmaurya! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-02-24 21:04:04 UTC

pl_bolts/models/gans/pix2pix/components.py

oke-aditya

I had glance of it, let me know what you think.
Very nice PR, Pix2Pix is really useful model.

oke-aditya · 2021-01-22T05:07:30Z

pl_bolts/models/gans/pix2pix/components.py

+
+    def __init__(self, input_channels, use_dropout=False, use_bn=True):
+        super(self).__init__()
+        self.upsample = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)


Let's just confirm once if Upsample is done using nn.Upsample or nn.ConvTranspose2d both work fine. I haven't read Pix2Pix paper so let me check once.

Hi, thank you for the review. In section 6 of the Pix2Pix paper authors have mentioned that they upsampled the tensors by a factor of 2 but they haven't exactly mentioned if Transposed Conv is used or Upsample followed by Conv layer.

I quickly checked the paper and found that the PyTorch implementation linked from the author's Lua implementation uses nn.ConvTranspose2d, so shall we follow that architecture unless someone has a strong opinion?

Yes I too confirmed that it is nn.ConvTranspose2d. I have referred TensorFlow docs, which give a really nice implementation.

pl_bolts/models/gans/pix2pix/components.py

Co-authored-by: Aditya Oke <47158509+oke-aditya@users.noreply.github.com>

akihironitta

Thank you for your work! Could you have a look at my nitpicking comments? 😅

pl_bolts/models/gans/__init__.py

aniketmaurya · 2021-01-30T19:33:31Z

Hi, I have added the training step code to the PR. Also I have tried to train the model using Facades dataset.

Please help in review the PR.

Cc: @Borda @akihironitta @oke-aditya

akihironitta

@aniketmaurya Would you mind merging master and applying yapf to pass the failing tests?

akihironitta · 2021-02-03T04:09:05Z

pl_bolts/models/gans/pix2pix/components.py

+
+    def __init__(self, input_channels, use_dropout=False, use_bn=True):
+        super(self).__init__()
+        self.upsample = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)


I quickly checked the paper and found that the PyTorch implementation linked from the author's Lua implementation uses nn.ConvTranspose2d, so shall we follow that architecture unless someone has a strong opinion?

pl_bolts/models/gans/pix2pix/components.py

oke-aditya

I'm bit unsure about some aspects of implementation.
This colab notebook gives a really simple and effective Pix2Pix.

pl_bolts/models/gans/pix2pix/components.py

oke-aditya · 2021-02-03T04:39:04Z

pl_bolts/models/gans/pix2pix/components.py

+
+    def __init__(self, input_channels, use_dropout=False, use_bn=True):
+        super(self).__init__()
+        self.upsample = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)


Yes I too confirmed that it is nn.ConvTranspose2d. I have referred TensorFlow docs, which give a really nice implementation.

pl_bolts/models/gans/pix2pix/components.py

oke-aditya · 2021-02-03T04:50:06Z

GANs are notorius 😅 Small differences such as LeakyRelu vs Relu, Dropout probability, ConvTranspose vs Upsampling really make difference. Really tough time in getting a GAN trained, really great job @aniketmaurya

@akihironitta I would suggest to use torchvision, it is well tested and a standard (over 15Million downloads).
Even with lightning flash we use it now 😄 It avoids lot of code writing.

akihironitta · 2021-02-03T07:38:22Z

@oke-aditya I agree, thanks for your comment :)

akihironitta

Hey @aniketmaurya, it'd be great to have this model in Bolts!!! Please mark this PR as ready for review when you're ready :)

requirements.txt

initial commit

118ca75

github-actions bot added the model label Jan 21, 2021

aniketmaurya changed the title ~~Draft PR:~~ Add Pix2Pix model: #533 Jan 21, 2021

aniketmaurya changed the title ~~Add Pix2Pix model: #533~~ Add Pix2Pix model Jan 21, 2021

added components for pix2pix

57ac369

pix2pix model created

21b8f90

oke-aditya reviewed Jan 22, 2021

View reviewed changes

pl_bolts/models/gans/pix2pix/components.py Outdated Show resolved Hide resolved

oke-aditya reviewed Jan 22, 2021

View reviewed changes

pl_bolts/models/gans/pix2pix/components.py Outdated Show resolved Hide resolved

oke-aditya reviewed Jan 22, 2021

View reviewed changes

Update pl_bolts/models/gans/pix2pix/components.py

8c69cb2

Co-authored-by: Aditya Oke <47158509+oke-aditya@users.noreply.github.com>

Borda force-pushed the master branch from feb8f77 to 3d49fd4 Compare January 22, 2021 21:30

akihironitta reviewed Jan 23, 2021

View reviewed changes

pl_bolts/models/gans/__init__.py Outdated Show resolved Hide resolved

pl_bolts/models/gans/__init__.py Outdated Show resolved Hide resolved

akihironitta self-assigned this Jan 23, 2021

Aniket Maurya added 3 commits January 23, 2021 19:29

yapf consistency

635900c

removed tilde

bac0eeb

added training step and some reformatting

ecbe987

aniketmaurya requested a review from akihironitta January 30, 2021 20:04

akihironitta reviewed Feb 3, 2021

View reviewed changes

oke-aditya reviewed Feb 3, 2021

View reviewed changes

Aniket Maurya and others added 6 commits February 10, 2021 01:31

center_crop torchvision and dropout value 0.5

16368c1

renamed variables

2aac8d4

update

6ffad85

updated requirements file

c4f6a0d

Merge branch 'master' into master

5d8f2d5

refactored and need to fix generator UPSAMPLE

45ccdf8

merge

5eaab53

akihironitta reviewed Feb 13, 2021

View reviewed changes

requirements.txt Outdated Show resolved Hide resolved

Aniket Maurya and others added 4 commits February 13, 2021 11:29

removed torchvision

bd9dcc8

refactored

216a4b9

updated

7e90f80

yapf

66317f4

aniketmaurya marked this pull request as ready for review February 24, 2021 21:15

aniketmaurya requested review from ananyahjha93 and Borda as code owners February 24, 2021 21:15

Borda requested a review from akihironitta March 4, 2021 21:46

Borda approved these changes Mar 4, 2021

View reviewed changes

Borda merged commit 53f5370 into Lightning-Universe:master Mar 4, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Pix2Pix model #533

Add Pix2Pix model #533

aniketmaurya commented Jan 21, 2021 •

edited

codecov bot commented Jan 21, 2021 •

edited

pep8speaks commented Jan 21, 2021 •

edited

oke-aditya left a comment

oke-aditya Jan 22, 2021

aniketmaurya Jan 22, 2021

akihironitta Feb 3, 2021

oke-aditya Feb 3, 2021

akihironitta left a comment

aniketmaurya commented Jan 30, 2021 •

edited

akihironitta left a comment

akihironitta Feb 3, 2021

oke-aditya left a comment

oke-aditya Feb 3, 2021

oke-aditya commented Feb 3, 2021

akihironitta commented Feb 3, 2021

akihironitta left a comment

Add Pix2Pix model #533

Add Pix2Pix model #533

Conversation

aniketmaurya commented Jan 21, 2021 • edited

What does this PR do?

Before submitting

PR review

Did you have fun?

codecov bot commented Jan 21, 2021 • edited

Codecov Report

pep8speaks commented Jan 21, 2021 • edited

Comment last updated at 2021-02-24 21:04:04 UTC

oke-aditya left a comment

Choose a reason for hiding this comment

oke-aditya Jan 22, 2021

Choose a reason for hiding this comment

aniketmaurya Jan 22, 2021

Choose a reason for hiding this comment

akihironitta Feb 3, 2021

Choose a reason for hiding this comment

oke-aditya Feb 3, 2021

Choose a reason for hiding this comment

akihironitta left a comment

Choose a reason for hiding this comment

aniketmaurya commented Jan 30, 2021 • edited

akihironitta left a comment

Choose a reason for hiding this comment

akihironitta Feb 3, 2021

Choose a reason for hiding this comment

oke-aditya left a comment

Choose a reason for hiding this comment

oke-aditya Feb 3, 2021

Choose a reason for hiding this comment

oke-aditya commented Feb 3, 2021

akihironitta commented Feb 3, 2021

akihironitta left a comment

Choose a reason for hiding this comment

aniketmaurya commented Jan 21, 2021 •

edited

codecov bot commented Jan 21, 2021 •

edited

pep8speaks commented Jan 21, 2021 •

edited

aniketmaurya commented Jan 30, 2021 •

edited