New batch inference

[yhcao6]

No description provided.

[Johnson-Wang]

Please add the corresponding description at the tutorial part since it seems that the default config must be modified.

[Johnson-Wang]

In the tutorial, please also add a note that batch inference is not recommended for unfixed image sizes since it would be even slower and hurt the performance.

[Johnson-Wang]

Since this PR would change the interfaces when inference with batch size=1, it's better to make sure all models are compatible. is it convenient to run a benchmark test for all the pretrained models?

[codecov]

Codecov Report

Merging #3564 into master will increase coverage by 0.05%.
The diff coverage is 59.01%.

@@            Coverage Diff             @@
##           master    #3564      +/-   ##
==========================================
+ Coverage   61.01%   61.06%   +0.05%     
==========================================
  Files         205      216      +11     
  Lines       13827    15315    +1488     
  Branches     2344     2607     +263     
==========================================
+ Hits         8436     9352     +916     
- Misses       4962     5498     +536     
- Partials      429      465      +36     

Flag	Coverage Δ
#unittests	61.06% <59.01%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmdet/apis/inference.py	20.73% <0.00%> (ø)
mmdet/apis/test.py	13.08% <0.00%> (+0.47%)	⬆️
mmdet/core/bbox/__init__.py	100.00% <ø> (ø)
mmdet/core/bbox/assigners/max_iou_assigner.py	72.36% <ø> (ø)
mmdet/models/detectors/fast_rcnn.py	43.75% <ø> (+4.86%)	⬆️
mmdet/models/losses/cross_entropy_loss.py	100.00% <ø> (ø)
mmdet/models/roi_heads/point_rend_roi_head.py	21.81% <0.00%> (-2.93%)	⬇️
mmdet/models/roi_heads/htc_roi_head.py	31.83% <2.32%> (-2.63%)	⬇️
mmdet/core/bbox/assigners/grid_assigner.py	28.00% <28.00%> (ø)
mmdet/models/dense_heads/sabl_retina_head.py	30.41% <30.41%> (ø)
... and 48 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f93c00f...5f33731. Read the comment docs.

[Johnson-Wang]

I wonder if it is possible to summarize the tested scripts and configs to ensure compatibility. I think there are still several scripts left for compatibility check. To name a few, pytorch2onnx, webcam_demo etc.