Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for multi-channel images in PyTorchObjectDetector #1633

Merged
merged 2 commits into from
Apr 20, 2022

Conversation

beat-buesser
Copy link
Collaborator

Signed-off-by: Beat Buesser beat.buesser@ie.ibm.com

Description

This pull request adds support for multi-channel images in PyTorchObjectDetector.

Fixes #1630

Type of change

Please check all relevant options.

  • Improvement (non-breaking)
  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Signed-off-by: Beat Buesser <beat.buesser@ie.ibm.com>
@beat-buesser beat-buesser self-assigned this Apr 18, 2022
@beat-buesser beat-buesser added the improvement Improve implementation label Apr 18, 2022
@beat-buesser beat-buesser added this to the ART 1.10.1 milestone Apr 18, 2022
@beat-buesser beat-buesser linked an issue Apr 18, 2022 that may be closed by this pull request
@codecov-commenter
Copy link

codecov-commenter commented Apr 18, 2022

Codecov Report

Merging #1633 (e12a3ca) into dev_1.10.1 (77643d4) will increase coverage by 0.00%.
The diff coverage is 0.00%.

Impacted file tree graph

@@             Coverage Diff             @@
##           dev_1.10.1    #1633   +/-   ##
===========================================
  Coverage       88.09%   88.10%           
===========================================
  Files             259      259           
  Lines           21328    21328           
  Branches         3779     3779           
===========================================
+ Hits            18788    18790    +2     
+ Misses           1597     1596    -1     
+ Partials          943      942    -1     
Impacted Files Coverage Δ
...imators/object_detection/python_object_detector.py 79.21% <0.00%> (ø)
...estimators/object_detection/pytorch_faster_rcnn.py 100.00% <ø> (ø)
art/attacks/evasion/boundary.py 92.77% <0.00%> (-1.21%) ⬇️
art/defences/detector/poison/activation_defence.py 84.61% <0.00%> (+0.33%) ⬆️
art/estimators/poison_mitigation/strip/strip.py 100.00% <0.00%> (+5.55%) ⬆️
art/estimators/certification/abstain.py 100.00% <0.00%> (+9.09%) ⬆️

Copy link
Collaborator

@GiulioZizzo GiulioZizzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Beat, I just had a minor comment, but otherwise looks good!

@@ -214,7 +214,7 @@ def _get_losses(
x_grad.requires_grad = True
else:
x_grad = x[i].to(self.device)
if x_grad.shape[-1] in [1, 3]:
if x_grad.shape[2] < x_grad.shape[0] and x_grad.shape[2] < x_grad.shape[1]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep the channels_first variable to allow manual configuration of the channels, and if it is not supplied, infer the channel number via this logic? There could be unusual cases in which the channels are more than one of the height/width dimensions of the input.

Copy link
Collaborator Author

@beat-buesser beat-buesser Apr 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. For this patch release we decided to keep the channels_first unused, but update the documentation accordingly, with the goal not to break the estimator. I agree, there could be images with more channels than pixels in height and width. All the models supported by this estimator are expecting channels first, therefore we should update the channels_First argument` for ART 1.11, I have opened issue #1637.

@beat-buesser beat-buesser merged commit c25abb5 into dev_1.10.1 Apr 20, 2022
@beat-buesser beat-buesser deleted the development_issue_1630 branch April 20, 2022 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improve implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update object detector _get_losses() to support mulitmodal inputs
3 participants