feat: Added random noise augmentation to object detection #654

SiddhantBahuguna · 2021-11-29T13:11:20Z

With regards to an issue cf. #730 Extends the list of supported data augmentations,
This PR introduces the following modifications:

It adds Gaussian noise as an augmentation


from doctr.transforms import GaussianNoise
import torch
transfo = GaussianNoise(0., .25) 
output = transfo(torch.rand((3, 224, 224))) # input image as torch.Tensor

It adds a unittest for the same class

Your feedback is highly welcomed :)

1) random horizontal flip 2) random vertical flip

…cal_flips

codecov · 2021-11-29T13:38:23Z

Codecov Report

Merging #654 (94535b1) into main (c72da96) will decrease coverage by 0.01%.
The diff coverage is 91.66%.

@@            Coverage Diff             @@
##             main     #654      +/-   ##
==========================================
- Coverage   96.00%   95.99%   -0.02%     
==========================================
  Files         129      129              
  Lines        4807     4818      +11     
==========================================
+ Hits         4615     4625      +10     
- Misses        192      193       +1

Flag	Coverage Δ
unittests	`95.99% <91.66%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
doctr/transforms/modules/pytorch.py	`97.67% <91.66%> (-2.33%)`	⬇️
doctr/transforms/modules/base.py	`94.44% <0.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c72da96...94535b1. Read the comment docs.

fg-mindee · 2021-11-29T23:22:32Z

The modifications seem to be highly similar to #648, is there a reason to have them both?
I suggest to first check my feedback on the other PR, fix flake8 here and we'll go from there 👌

SiddhantBahuguna · 2021-12-02T10:00:15Z

The modifications seem to be highly similar to #648, is there a reason to have them both? I suggest to first check my feedback on the other PR, fix flake8 here and we'll go from there ok_hand

Yes, this PR introduces random horizontal and vertical flip :)

fg-mindee · 2021-12-02T11:41:08Z

Yes, this PR introduces random horizontal and vertical flip :)

Alright, we if you open several PRs in parallel, in order to avoid conflict and ease the review, it's better if they are independent. Here you have the same modifications appearing in both PRs 😅

SiddhantBahuguna · 2021-12-02T12:03:53Z

Yes, this PR introduces random horizontal and vertical flip :)

Alright, we if you open several PRs in parallel, in order to avoid conflict and ease the review, it's better if they are independent. Here you have the same modifications appearing in both PRs sweat_smile

Yup, I am making changes in these ones as well :) Thanks

SiddhantBahuguna · 2021-12-02T15:20:24Z

I am getting a ValueError when I change the batch_size from 8 to any other size.
ValueError: could not broadcast input array from shape (9,5) into shape (9,4).
9 being the number of artefacts and 4 being the box coordinates for each batch
Traceback below:

doctr/references/obj_detection/train_pytorch.py

Line 63 in 1c71c3c

images[id], bbox[id] = rotate(images[id], bbox[id], angle[id] if bool(

Could you please take a look:) Thanks a lot!

fg-mindee · 2021-12-02T16:59:06Z

I am getting a ValueError when I change the batch_size from 8 to any other size. ValueError: could not broadcast input array from shape (9,5) into shape (9,4). 9 being the number of artefacts and 4 being the box coordinates for each batch

That means the dimensions are most likely interpreted in the wrong order 🙃

Traceback below:

doctr/references/obj_detection/train_pytorch.py

Line 63 in 1c71c3c

images[id], bbox[id] = rotate(images[id], bbox[id], angle[id] if bool(

? that's the one causing the error but not the traceback :) (traceback = the complete error message + tracing you get when you run your code)

Also flake8 isn't happy 😅

fg-mindee · 2021-12-10T10:56:38Z

Mind merging main into this please? #691 has changed a few things 👍

fg-mindee · 2021-12-16T18:45:48Z

Would you mind resolving the conflicts so that we can merge the PR? 🙏

SiddhantBahuguna · 2021-12-17T11:12:47Z

Would you mind resolving the conflicts so that we can merge the PR? pray

Yup, I have resolved the conflicts and have fixed the problem that was causing us to choose arbitrary batch_sizes. Doing some minor code style fixes ta the moment before I push

fg-mindee

Thanks for the PR! As discussed, the PR is too massive to correctly review it. I suggest we do several PRs to progressively integrate:

uniform photometric transformations (noise, blur, etc.)
non-uniform photometric transformations (shadows, etc.)
geometric transformations (flips, rotations, crops, etc.)

Ideally, if we could have those as doctr.transforms (working on PIL or Torch/TF tensors) that would be perfect! I'll open an issue to track this 👌

.idea/.gitignore

…ions

fg-mindee

The PR is massive, let's narrow it down to photometric augmentations as suggested previously. For instance, this PR could be only about adding a Noise augmentation. Everything else needs to be taken out, it's way too big for me to perform a careful review

On that note, if we use the dataloader, we do the transformation on pytorch tensor. Considering the operation you are doing in numpy, you can perfectly implement them in PyTorch or TensorFlow. In your case, you want to use it for a PyTorch script, so :

implement the RandomGaussianNoise transformation over here https://github.com/mindee/doctr/blob/main/doctr/transforms/modules/pytorch.py
adds some unittest
adds the entry in the documentation
adds an illustration of the augmentation in the PR description
and finally add the transform in the Compose passed to the dataset 👍

references/obj_detection/constants.py

fg-mindee

Thanks, added some comments!

doctr/transforms/modules/pytorch.py

references/obj_detection/train_pytorch.py

references/obj_detection/utils.py

fg-mindee · 2021-12-24T16:14:05Z

Also, you'll need to add unittest for this transformation

fg-mindee

Looking good, thanks!

fg-mindee · 2021-12-27T14:31:54Z

references/obj_detection/train_pytorch.py

+
+    Args:
+       freeze_until (str, optional): last layer to freeze
+       start_lr (float, optional): initial learning rate
+       end_lr (float, optional): final learning rate
+       num_it (int, optional): number of iterations to perform


This was removed on purpose, it's not accurate

SiddhantBahuguna added 2 commits November 24, 2021 17:09

chore: Added rotations as augmentations

327c244

chore: Added miinor augmentations

77402de

1) random horizontal flip 2) random vertical flip

SiddhantBahuguna added the ext: references Related to references folder label Nov 29, 2021

SiddhantBahuguna requested a review from fg-mindee November 29, 2021 13:11

SiddhantBahuguna self-assigned this Nov 29, 2021

SiddhantBahuguna added 2 commits November 29, 2021 14:17

fix: Added the height, width of the image for random-horizontal/verti…

d82786f

…cal_flips

Merge branch 'main' into minor_aug

1ee4810

charlesmindee changed the title ~~Minor aug~~ feat: Minor aug Nov 29, 2021

charlesmindee changed the title ~~feat: Minor aug~~ feat: Minor augmentations obj. detection Nov 29, 2021

fg-mindee added the topic: object detection Related to the task of object detection label Nov 29, 2021

SiddhantBahuguna added 2 commits December 2, 2021 13:34

chore: added rotations

41b9dd8

Merge branch 'main' into minor_aug

1c71c3c

fg-mindee mentioned this pull request Dec 2, 2021

chore: Added rotations as augmentations #648

Closed

SiddhantBahuguna added 3 commits December 7, 2021 12:15

feat: Added shadows as augmentations

e8e3791

feat: Added shadows as augmentations

e513c04

chore: Used numpy for random bool response

349ea72

SiddhantBahuguna added 3 commits December 17, 2021 13:54

fix: FIxed the conflicts with the main

d3e537b

fix: Merged main with this PR fixing timeouts in CI

fad5b7b

feat: Removed geometric augmentations.

2617e27

fg-mindee suggested changes Dec 20, 2021

View reviewed changes

.idea/.gitignore Outdated Show resolved Hide resolved

SiddhantBahuguna added 2 commits December 23, 2021 15:58

chore: Removed unnecessary functions related to photometric augmentat…

287ee76

…ions

chore: Reverted last commit

7507cd5

fg-mindee suggested changes Dec 23, 2021

View reviewed changes

references/obj_detection/constants.py Outdated Show resolved Hide resolved

feat: Removed shadows and added randomgaussian noise

d4985ab

fg-mindee suggested changes Dec 24, 2021

View reviewed changes

fg-mindee changed the title ~~feat: Minor augmentations obj. detection~~ feat: Added random noise augmentation to object detection Dec 24, 2021

fg-mindee added topic: documentation Improvements or additions to documentation module: transforms Related to doctr.transforms framework: pytorch Related to PyTorch backend labels Dec 24, 2021

SiddhantBahuguna added 6 commits December 27, 2021 11:29

chore: Merged main with the PR

2492ab5

fix: Fixed style

db8fe38

fix: Fixed the distribution range

ecff1c8

test: Added unittest for GaussianNoise

ad9cf4a

fix: Reverted to original script

6161e25

fix: Reverted to old train script

bacbfb5

SiddhantBahuguna added the ext: tests Related to tests folder label Dec 28, 2021

SiddhantBahuguna added this to the 0.5.0 milestone Dec 28, 2021

SiddhantBahuguna requested a review from fg-mindee December 28, 2021 17:13

SiddhantBahuguna added 6 commits December 29, 2021 14:46

chore: Merged main with the PR

89d5be2

Merge branch 'main' into minor_aug

5f21008

fix: Updated the new checkpoint path

55d984a

chore: Merged with main

76adc41

fix: Fixed typo

2f260ae

style: Fixed brackets indentation

94535b1

fg-mindee approved these changes Dec 29, 2021

View reviewed changes

SiddhantBahuguna merged commit 8e7b0ee into main Dec 29, 2021

SiddhantBahuguna deleted the minor_aug branch December 29, 2021 14:47

fg-mindee mentioned this pull request Dec 29, 2021

feat: Added ChannelShuffle transformation and fixes RandomCrop #768

Merged

fg-mindee added the type: new feature New feature label Dec 31, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Added random noise augmentation to object detection #654

feat: Added random noise augmentation to object detection #654

SiddhantBahuguna commented Nov 29, 2021 •

edited

codecov bot commented Nov 29, 2021 •

edited

fg-mindee commented Nov 29, 2021 •

edited by SiddhantBahuguna

SiddhantBahuguna commented Dec 2, 2021

fg-mindee commented Dec 2, 2021

SiddhantBahuguna commented Dec 2, 2021

SiddhantBahuguna commented Dec 2, 2021 •

edited

fg-mindee commented Dec 2, 2021

fg-mindee commented Dec 10, 2021

fg-mindee commented Dec 16, 2021

SiddhantBahuguna commented Dec 17, 2021 •

edited

fg-mindee left a comment

fg-mindee left a comment

fg-mindee left a comment

fg-mindee commented Dec 24, 2021 •

edited

fg-mindee left a comment

fg-mindee Dec 27, 2021

feat: Added random noise augmentation to object detection #654

feat: Added random noise augmentation to object detection #654

Conversation

SiddhantBahuguna commented Nov 29, 2021 • edited

codecov bot commented Nov 29, 2021 • edited

Codecov Report

fg-mindee commented Nov 29, 2021 • edited by SiddhantBahuguna

SiddhantBahuguna commented Dec 2, 2021

fg-mindee commented Dec 2, 2021

SiddhantBahuguna commented Dec 2, 2021

SiddhantBahuguna commented Dec 2, 2021 • edited

fg-mindee commented Dec 2, 2021

fg-mindee commented Dec 10, 2021

fg-mindee commented Dec 16, 2021

SiddhantBahuguna commented Dec 17, 2021 • edited

fg-mindee left a comment

Choose a reason for hiding this comment

fg-mindee left a comment

Choose a reason for hiding this comment

fg-mindee left a comment

Choose a reason for hiding this comment

fg-mindee commented Dec 24, 2021 • edited

fg-mindee left a comment

Choose a reason for hiding this comment

fg-mindee Dec 27, 2021

Choose a reason for hiding this comment

SiddhantBahuguna commented Nov 29, 2021 •

edited

codecov bot commented Nov 29, 2021 •

edited

fg-mindee commented Nov 29, 2021 •

edited by SiddhantBahuguna

SiddhantBahuguna commented Dec 2, 2021 •

edited

SiddhantBahuguna commented Dec 17, 2021 •

edited

fg-mindee commented Dec 24, 2021 •

edited