[ONNX] Fix export of images with no detection #2215
Conversation
…of/fixNoDetection # Conflicts: # torchvision/models/detection/transform.py
… into neraoof/fixDynamicResize
…on into neraoof/fixDynamicResize
…of/fixDynamicResize
…on into neraoof/fixDynamicResize
…of/fixDynamicResize # Conflicts: # torchvision/models/detection/roi_heads.py
…of/fixDynamicResize
…on into neraoof/fixDynamicResize
|
cc @fmassa for review. Thanks! |
| else: | ||
| w_half = (boxes[:, 2] - boxes[:, 0]).to(dtype=torch.float64) * .5 | ||
| h_half = (boxes[:, 3] - boxes[:, 1]).to(dtype=torch.float64) * .5 | ||
| x_c = (boxes[:, 2] + boxes[:, 0]).to(dtype=torch.float64) * .5 | ||
| y_c = (boxes[:, 3] + boxes[:, 1]).to(dtype=torch.float64) * .5 | ||
|
|
||
| w_half = w_half.to(dtype=torch.float64) * scale | ||
| h_half = h_half.to(dtype=torch.float64) * scale | ||
|
|
||
| boxes_exp0 = x_c - w_half | ||
| boxes_exp1 = y_c - h_half | ||
| boxes_exp2 = x_c + w_half | ||
| boxes_exp3 = y_c + h_half | ||
| boxes_exp = torch.stack((boxes_exp0, boxes_exp1, boxes_exp2, boxes_exp3), 1) | ||
| return boxes_exp.to(dtype=boxes.dtype) |
There was a problem hiding this comment.
Out of curiosity, which operation doesn't support empty tensors in ONNX?
There was a problem hiding this comment.
So I was seeing an issue when broadcasting And op:
"Can broadcast 0 by 0 or 1. 3 is invalid"
But after looks like this is not needed anymore. I've reverted this change.
| boxes_exp = torch.stack((boxes_exp0, boxes_exp1, boxes_exp2, boxes_exp3), 1) | ||
| return boxes_exp | ||
| if boxes.numel() == 0: | ||
| return torch.empty(0, dtype=boxes.dtype) |
There was a problem hiding this comment.
This will return a box with a different number of dimensions in this case, as we normally return boxes with Nx4 dimensions. It might bring issues in the future
There was a problem hiding this comment.
Actually looks like this branch is not needed. I have cleaned this up.
| @@ -237,7 +237,7 @@ def __init__(self, in_channels, num_keypoints): | |||
| super(KeypointRCNNPredictor, self).__init__() | |||
| input_features = in_channels | |||
| deconv_kernel = 4 | |||
| self.kps_score_lowres = misc_nn_ops.ConvTranspose2d( | |||
| self.kps_score_lowres = nn.ConvTranspose2d( | |||
There was a problem hiding this comment.
Great that we already support empty tensors in PyTorch! Thanks for the cleanup!
|
Apart from this question, I think the PR looks good to merge (minus the tensor having a different number of dimensions) |
|
@fmassa Thanks a lot for pointing out the changes in roi_heads. |
…of/fixDynamicResize # Conflicts: # torchvision/models/detection/keypoint_rcnn.py
| class ConvTranspose2d(torch.nn.ConvTranspose2d): | ||
| """ | ||
| Equivalent to nn.ConvTranspose2d, but with support for empty batch sizes. | ||
| This will eventually be supported natively by PyTorch, and this | ||
| class can go away. | ||
| """ | ||
| def forward(self, x): | ||
| if x.numel() > 0: | ||
| return self.super_forward(x) | ||
| # get output shape | ||
|
|
||
| output_shape = [ | ||
| (i - 1) * d - 2 * p + (di * (k - 1) + 1) + op | ||
| for i, p, di, k, d, op in zip( | ||
| x.shape[-2:], | ||
| list(self.padding), | ||
| list(self.dilation), | ||
| list(self.kernel_size), | ||
| list(self.stride), | ||
| list(self.output_padding), | ||
| ) | ||
| ] | ||
| output_shape = [x.shape[0], self.out_channels] + output_shape | ||
| return _new_empty_tensor(x, output_shape) | ||
|
|
||
| def super_forward(self, input, output_size=None): | ||
| # type: (Tensor, Optional[List[int]]) -> Tensor | ||
| if self.padding_mode != 'zeros': | ||
| raise ValueError('Only `zeros` padding mode is supported for ConvTranspose2d') | ||
|
|
||
| output_padding = self._output_padding(input, output_size, self.stride, self.padding, self.kernel_size) | ||
|
|
||
| return F.conv_transpose2d( | ||
| input, self.weight, self.bias, self.stride, self.padding, | ||
| output_padding, self.groups, self.dilation) | ||
|
|
||
|
|
||
| class BatchNorm2d(torch.nn.BatchNorm2d): |
There was a problem hiding this comment.
This is technically a BC-breaking change, as if users were relying on ConvTranspose2d or BatchNorm2d from misc.py this will error now. I'll merge this as is to move forward, but I'll send a follow-up PR adding back a backward-compatibility fix.
There was a problem hiding this comment.
Got it. Thanks a lot @fmassa
Fix export of images with no detection for Mask RCNN