[Feature] Add gather points op from mmdet3d

[DCNSW]

Motivation

Add gather points cuda operation from mmdet3d (branch: v1.0.0.dev0).

Modification

Several files in mmcv/ops folder.

BC-breaking (Optional)

No.

Use cases (Optional)

from mmcv.ops import gather_points

[codecov]

Codecov Report

Merging #1338 (d18d416) into master (8cac7c2) will decrease coverage by 0.08%.
The diff coverage is 40.00%.

❗ Current head d18d416 differs from pull request most recent head d4757ba. Consider uploading reports for the commit d4757ba to get more accurate results

@@            Coverage Diff             @@
##           master    #1338      +/-   ##
==========================================
- Coverage   68.59%   68.51%   -0.09%     
==========================================
  Files         164      165       +1     
  Lines       10891    10922      +31     
  Branches     1991     1993       +2     
==========================================
+ Hits         7471     7483      +12     
- Misses       3030     3047      +17     
- Partials      390      392       +2     

Flag	Coverage Δ
unittests	68.51% <40.00%> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmcv/ops/gather_points.py	37.50% <37.50%> (ø)
mmcv/ops/__init__.py	100.00% <100.00%> (ø)
mmcv/runner/dist_utils.py	50.00% <0.00%> (-1.07%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42c9e71...d4757ba. Read the comment docs.

[grimoire]

Hi, I think this ops can be implemented with PyTorch only:

# method0, permute might take extra computation
def gather_point_pytorch0(features, idx):
    batch_size = features.shape[0]
    batch_idx = torch.arange(batch_size, device=features.device).unsqueeze(-1)
    permute_output = features[batch_idx, :, idx]
    return permute_output.permute(0, 2, 1)

# method1, repeat might need more memory
def gather_point_pytorch1(features, idx):
    new_idx = idx.unsqueeze(1).repeat(1, 3, 1)
    return features.gather(2, new_idx)

[DCNSW]

Hi, I think this ops can be implemented with PyTorch only:

# method0, permute might take extra computation
def gather_point_pytorch0(features, idx):
    batch_size = features.shape[0]
    batch_idx = torch.arange(batch_size, device=features.device).unsqueeze(-1)
    permute_output = features[batch_idx, :, idx]
    return permute_output.permute(0, 2, 1)

# method1, repeat might need more memory
def gather_point_pytorch1(features, idx):
    new_idx = idx.unsqueeze(1).repeat(1, 3, 1)
    return features.gather(2, new_idx)

We tested that the running time of the PyTorch version code is 17.3 times that of the Cuda version code, so it is necessary to implement with cuda.

[grimoire]

We tested that the running time of the PyTorch version code is 17.3 times that of the Cuda version code, so it is necessary to implement with cuda.

Here is my timer:

def timer(func, num_test, num_warmup, *args, **kwargs):
    start = torch.cuda.Event(enable_timing=True)
    end = torch.cuda.Event(enable_timing=True)

    # warmup
    for _ in range(num_warmup):
        func(*args, **kwargs)

    torch.cuda.synchronize()

    start.record()
    for _ in range(num_test):
        output = func(*args, **kwargs)
    end.record()

    torch.cuda.synchronize()
    print(f'{func} take time {start.elapsed_time(end)/num_test}.')

    return output

With warmup=5 and num_test=10, method0 is 3 times slower and method1 is 2 times slower.
I believe custom cuda implement must be faster, but 17.3 times seems too much.

[ZwwWayne]

Need to resolve conflict and clean DIVUP before merge.