Add RetinaNet Object detection with Backbones #529

oke-aditya · 2021-01-19T16:38:51Z

What does this PR do?

Fixes #391

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests? [not needed for typos/docs]
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Is this pull request ready for review?

Did you have fun?

I think yes 😛

codecov · 2021-01-20T16:02:32Z

Codecov Report

Merging #529 (857a19f) into master (7369ad9) will increase coverage by 0%.
The diff coverage is 78%.

❗ Current head 857a19f differs from pull request most recent head 9817ced. Consider uploading reports for the commit 9817ced to get more accurate results

@@          Coverage Diff           @@
##           master   #529    +/-   ##
======================================
  Coverage      73%    73%            
======================================
  Files         125    131     +6     
  Lines        7884   8552   +668     
======================================
+ Hits         5737   6259   +522     
- Misses       2147   2293   +146

oke-aditya · 2021-01-20T16:13:54Z

This PR also bumps minimum torchvision to 0.8.1 and PyTorch to 1.7.
This is must to include retinanet as it comes from torchvision 0.8.1 only
I will ensure that all other tests pass with this upgrade.
We can possibly enable testing for Python 3.9 too with this (do in later PR to avoid unrelated failure)

Also this version bump enables us to use CUDA 11 compatible GPUs Ampere Architecture (A100, RTX30XX series).
It might fix distributed problems over Windwos as Compatibility is improved (with Gloo backend).

cc @Borda @akihironitta @ananyahjha93

…bolts into add_retina

…g-bolts into add_retina

oke-aditya

I left some comments to ease further discussions :)

pl_bolts/metrics/object_detection.py

pl_bolts/models/detection/faster_rcnn/faster_rcnn_module.py

pl_bolts/models/detection/retinanet/backbones.py

tests/models/test_detection.py

oke-aditya · 2021-01-20T19:10:51Z

The RetinaNet test is very slow, the reason being not using Batched NMS implementation. This is fixed in torchvision master and we can expect faster RetinaNet model + tests in next torchvision release

pl_bolts/models/detection/__init__.py

pl_bolts/models/detection/faster_rcnn/faster_rcnn_module.py

tests/models/test_detection.py

Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

oke-aditya · 2021-01-25T19:36:04Z

@Borda @akihironitta can you have look 😅

akihironitta

It'd be nice to hear your thoughts!

pl_bolts/models/detection/retinanet/retainanet_module.py

tests/models/test_detection.py

pl_bolts/models/detection/retinanet/backbones.py

akihironitta

I believe this PR is ready for getting merged.

Just for the note, here's the summary of my last tens of commits:

Updated CHANGELOG
Added its doc
Reverted the change in Faster R-CNN as it's irrelevant to this PR
Made RetinaNet use torchvision.ops.box_iou instead of our in-house pl_bolts.metrics.iou
Resolved all the failing tests related to this PR

cc @Borda

Borda · 2021-12-18T23:04:09Z

Just for the note, the currently failing tests are irrelevant to this PR. The failures are due to PyTorchLightning/pytorch-lightning#9535 where the error occurs only with Python 3.6 and pytorch_lightning<1.4.8. The fix (PyTorchLightning/pytorch-lightning#9554) is included in >=1.4.8.

so if we set py3.7 as the min version then we do not need to upgrade PL?

UPDATE: also if we replace the argparse with LightningCLI we are fine with the PL 1.4.0?

This reverts commit 5fccad6, reversing changes made to c30e68e.

akihironitta · 2021-12-19T00:30:49Z

TODO:

Set the min PL version to 1.4.0
Remove argparse and use LightningCLI

for more information, see https://pre-commit.ci

…bolts into add_retina

for more information, see https://pre-commit.ci

oke-aditya · 2021-12-20T07:30:35Z

Looks great. So kind of you @akihironitta Aki : 💖

oke-aditya · 2021-12-20T10:57:35Z

Thanks a lot to @akihironitta and @Borda for your patience. I'm Extremely sorry for this being delayed.

akihironitta · 2021-12-20T13:32:38Z

@oke-aditya Hey Aditya, not at all! Thank you for your work and patience for almost a year 🎉

oke-aditya added 2 commits January 19, 2021 22:06

refactor frcnn

a665049

start adding retina

7827b44

oke-aditya requested review from akihironitta, ananyahjha93 and Borda as code owners January 19, 2021 16:38

github-actions bot added the model label Jan 19, 2021

oke-aditya added 3 commits January 19, 2021 22:15

add test

7a4955a

complete Retinanet

2feca3b

Isort test

c4e25bd

oke-aditya added 3 commits January 20, 2021 21:33

reformat

f912d9e

update requirments

00322cd

bump torch

6513b74

oke-aditya added 3 commits January 20, 2021 21:50

update

61aafbf

Merge branch 'add_retina' of github.com:oke-aditya/pytorch-lightning-…

5b8c65b

…bolts into add_retina

Merge branch 'master' of github.com:PyTorchLightning/pytorch-lightnin…

c476492

…g-bolts into add_retina

oke-aditya changed the title ~~[WIP] Add RetinaNet Object detection with Backbones~~ Add RetinaNet Object detection with Backbones Jan 20, 2021

slight documentation edit

a8fda1a

oke-aditya commented Jan 20, 2021

View reviewed changes

sppedup test

e4803a2

oke-aditya mentioned this pull request Jan 21, 2021

Add Documentation for Detection Models #532

Closed

Borda force-pushed the master branch from feb8f77 to 3d49fd4 Compare January 22, 2021 21:30

akihironitta reviewed Jan 23, 2021

View reviewed changes

pl_bolts/models/detection/__init__.py Outdated Show resolved Hide resolved

pl_bolts/models/detection/faster_rcnn/faster_rcnn_module.py Show resolved Hide resolved

tests/models/test_detection.py Outdated Show resolved Hide resolved

oke-aditya and others added 3 commits January 23, 2021 20:03

Apply suggestions from code review

2a53b90

Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

remove max epochs

38e8a2e

fmt

5bee954

akihironitta reviewed Jan 26, 2021

View reviewed changes

pl_bolts/models/detection/retinanet/retainanet_module.py Outdated Show resolved Hide resolved

tests/models/test_detection.py Outdated Show resolved Hide resolved

pl_bolts/models/detection/retinanet/backbones.py Show resolved Hide resolved

Set pytorch-lightning>=1.4.8

1e0b268

akihironitta mentioned this pull request Dec 18, 2021

Drop Python 3.6 support #785

Merged

5 tasks

Merge branch 'ci/pl-1.4.8' into add_retina

5fccad6

akihironitta approved these changes Dec 18, 2021

View reviewed changes

Revert "Merge branch 'ci/pl-1.4.8' into add_retina"

0af474e

This reverts commit 5fccad6, reversing changes made to c30e68e.

mergify bot and others added 6 commits December 19, 2021 16:52

Merge branch 'master' into add_retina

2579bf4

Set pytorch-lightning>=1.4.0

3be86f2

Use LightningCLI

b235c35

Add LightningCLI requirement

2e351df

Use LightningCLI in tests

0a7a877

[pre-commit.ci] auto fixes from pre-commit.com hooks

7176d5a

for more information, see https://pre-commit.ci

mergify bot removed the ready label Dec 19, 2021

akihironitta added 5 commits December 20, 2021 04:13

Follow flake8

6ba5b86

Use gpus

255c192

Merge branch 'add_retina' of github.com:oke-aditya/pytorch-lightning-…

afd0a85

…bolts into add_retina

Run only on gpu env

8182964

Use LightningCLI v2

8e15fd0

mergify bot added the ready label Dec 19, 2021

akihironitta and others added 2 commits December 20, 2021 11:27

Add LightningCLI requirement

6baa9c7

[pre-commit.ci] auto fixes from pre-commit.com hooks

48326ee

for more information, see https://pre-commit.ci

mergify bot added the has conflicts label Dec 20, 2021

Merge branch 'master' into add_retina

00aa03d

mergify bot removed the has conflicts label Dec 20, 2021

Borda merged commit 51edf4a into Lightning-Universe:master Dec 20, 2021

oke-aditya deleted the add_retina branch December 20, 2021 13:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add RetinaNet Object detection with Backbones #529

Add RetinaNet Object detection with Backbones #529

oke-aditya commented Jan 19, 2021 •

edited by akihironitta

codecov bot commented Jan 20, 2021 •

edited

oke-aditya commented Jan 20, 2021 •

edited

oke-aditya left a comment

oke-aditya commented Jan 20, 2021

oke-aditya commented Jan 25, 2021 •

edited

akihironitta left a comment

akihironitta left a comment

Borda commented Dec 18, 2021 •

edited

akihironitta commented Dec 19, 2021 •

edited

oke-aditya commented Dec 20, 2021

oke-aditya commented Dec 20, 2021

akihironitta commented Dec 20, 2021

Add RetinaNet Object detection with Backbones #529

Add RetinaNet Object detection with Backbones #529

Conversation

oke-aditya commented Jan 19, 2021 • edited by akihironitta

What does this PR do?

Before submitting

PR review

Did you have fun?

codecov bot commented Jan 20, 2021 • edited

Codecov Report

oke-aditya commented Jan 20, 2021 • edited

oke-aditya left a comment

Choose a reason for hiding this comment

oke-aditya commented Jan 20, 2021

oke-aditya commented Jan 25, 2021 • edited

akihironitta left a comment

Choose a reason for hiding this comment

akihironitta left a comment

Choose a reason for hiding this comment

Borda commented Dec 18, 2021 • edited

akihironitta commented Dec 19, 2021 • edited

oke-aditya commented Dec 20, 2021

oke-aditya commented Dec 20, 2021

akihironitta commented Dec 20, 2021

oke-aditya commented Jan 19, 2021 •

edited by akihironitta

codecov bot commented Jan 20, 2021 •

edited

oke-aditya commented Jan 20, 2021 •

edited

oke-aditya commented Jan 25, 2021 •

edited

Borda commented Dec 18, 2021 •

edited

akihironitta commented Dec 19, 2021 •

edited