Skip to content
Deformable ConvNets V2 (DCNv2) in PyTorch
Cuda C Python C++ Shell
Branch: master
Clone or download

Latest commit

chengdazhi Merge pull request #25 from Mukosame/patch-1
Update deform_conv_cuda_kernel.cu
Latest commit 2f57c5d May 27, 2019

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
functions rename non-modulated deform operators. Jan 2, 2019
modules rename non-modulated deform operators. Jan 2, 2019
src Update deform_conv_cuda_kernel.cu May 27, 2019
.gitignore use .so instead of .o Jan 2, 2019
README.md Update README.md Feb 3, 2019
build.py use .so instead of .o Jan 2, 2019
build_modulated.py use .so instead of .o Jan 2, 2019
make.sh use .so instead of .o Jan 2, 2019
test.py rename non-modulated deform operators. Jan 2, 2019
test_modulated.py rename non-modulated deform operators. Jan 2, 2019

README.md

Deformable-ConvNets-V2 in PyTorch

This repo is an implementation of Deformable Convolution V2. Ported from the original MXNet implementation.

Refer to mmdetection branch in this repo for a complete framework. Results of DCNv2 based on mmdetection code base can be found at model zoo. Many thanks to mmdetection for their strong and clean framework.

Operators in master branch are compatible with pytorch_v0.4.1. For operators on pytorch v1.0.0 (implemented by Jiarui Xu), please refer to pytorch_1.0.0 branch.

Thanks to Kai Chen and other contributors from mmlab, DCNv2 is now included in the official mmdetection repo based on the master branch of this one. It is now written with the new cpp extension apis and it supports both PyTorch 0.4.1 and 1.0, with some minor speed and memory optimization. Results and models can be found at https://github.com/open-mmlab/mmdetection/blob/master/MODEL_ZOO.md#deformable-convolution-v2.

Build

sh make.sh

See test.py and test_modulated.py for example usage.

Notice

This repo provides the deformable conv layer which can reproduce the results in the Deformable ConvNets v2 paper. The major changes are as follows:

  • To better handle occasions where sampling locations are outside of the image boundary.

    In the previous operator, if the sampling location is outside of the feature map boundary, its sampled value would be zero. Thus, the gradient with respect to learnable offset would be zero. We found such a scheme may deteriate the performance in ImageNet classification (perhaps because the feature maps are of low resolution). For object detection on COCO, both the previous and the updated operators deliver the same results.

    In the new operator, if the sampling location is within one pixel outside of the feature map boundary, bilinear sampling would also be applied. And gradient with respect to learnable offset can be non zero for such locations. This is implemented by padding zeros (by one row/column) outside of the boundaries of feature maps, and performing bilinear sampling on the padded feature maps.

  • The efficiency of processing multiple images in a mini-batch is considerably improved.

    Both the previous and the updated operators follow the following computation pipeline (illustrated by a 3x3 deformable convolution with input data of NxCxHxW and output data of NxC'xHxW):

    for i in range(N/S):
        step 1 (slicing): slicing the input data at the batch dimension from i*S to (i+1)*S, input (NxCxHxW) -> sliced input (SxCxHxW)
        step 2 (deformable im2col): sliced input (SxCxHxW)+sliced offset (Sx18xHxW) -> column (Cx9xSxHxW)
        step 3 (MatMul&reshape): weight matrix (C'x 9C) * column (9CxSHW) -> temp sliced output (C'xSxHxW) -> sliced output (SxC'xHxW)
        step 4 (Merge): merge sliced output to form the whole output data (NxC'xHxW) 
    end
    

    In the previous operator, S is fixed as 1. In the updated operator, S can be set by the im2col_step parameter, whose default value is min(N, 64). The updated operator is significantly faster than the existing one when the image batch size is large.

You can’t perform that action at this time.