Speedup the Video Inference by Accelerating data-loading Stage

[chenxinfeng4]

Motivation

The video inference was not efficient, because there are many "resize", "padding", "to rgb" and "normalize" operations in CPU workload. Besides, the 'img_metas' is calculated for every frame, which is nonefficient. The CPU workload is really high, but the speed is slow.

Modification

I transformed the video "crop", "resize", "padding", "to rgb" in ffmpeg-based video reader, which is lite and CPU friendly. The video reader can support NVIDIA-VIDEO-DECODING if possible, to save more CPU workload. And the "normalize" is done in GPU. In all, I reduced the data-loading time and CPU workload.

Result

After this modification, the CPU workload can reduced from >1000% to 180%, and slightly improve the inference frame rate. It's significant in the case that dataloading is the heaviest cost than the network evalutation, such as the yolact.

[CLAassistant]

All committers have signed the CLA.

[chenxinfeng4]

You can compare the original video_demo.py with my video_gpuaccel_demo.py. They have some argment inputs.
To see how much improved, please delete the post-operations in both codes. Because the post-operations are also CPU heavy.

# post-operations in `video_demo.py` and my `video_gpuaccel_demo.py`.
model.show_result(xxx)

```

[hhaAndroid]

@chenxinfeng4 Thank you very much for your contribution.

[chenxinfeng4]

I tested the speed of data loading. It's about 230FPS , 800%CPU.

    with torch.cuda.device(args.device), torch.no_grad():
        for frame_resize, frame_origin in zip(tqdm.tqdm(video_resize), video_origin):
            data = process_img(frame_resize, img_metas)
            continue

[ZwwWayne]

lint failed.

[RangiLyu]

You can add some documents about this script in https://github.com/open-mmlab/mmdetection/blob/master/docs/en/1_exist_data_model.md

[RangiLyu]

Please use pre-commit hook to fix the lint.

[hhaAndroid]

@chenxinfeng4 Thanks for your very fast response.

[jbwang1997]

LGTM

[chenxinfeng4]

I'm not an expert of git. Why the PR is block? What should I do next?

[jbwang1997]

I'm not an expert of git. Why the PR is block? What should I do next?

Hello @chenxinfeng4.

Merging is controlled by maintainers. We will merge this pr as soon as possible. Seems the lint fails again. Please remember to fix it.

Thanks for your quick response.