Speed up RetinaNet's postprocess_detections()

[datumbox]

In relation to #2799, one way we could potentially speed up the RetinaNet's postprocess_detections() method is by applying a pre-filter method to:

• Eliminate any box whose scores do not surpass the threshold across any class.
• Remove from the candidate classes those that don't have an associated candidate box.

Benchmark (100 iterations) across different images:

• Before: 12.9425 sec
• After: 7.4375 sec

To measure the speed replace:

vision/torchvision/models/detection/retinanet.py

Line 567 in dd76ba9

detections = self.postprocess_detections(head_outputs, anchors, images.image_sizes)

with:

import time
start_time = time.time()
for i in range(100):
    detections = self.postprocess_detections(dict(head_outputs), anchors, images.image_sizes)
print('100 Iter in %.2f' % (time.time() - start_time))

I also used a Python profiler to confirm the results.

[codecov]

Codecov Report

Merging #2819 into master will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #2819      +/-   ##
==========================================
+ Coverage   73.26%   73.28%   +0.01%     
==========================================
  Files          99       99              
  Lines        8778     8784       +6     
  Branches     1387     1388       +1     
==========================================
+ Hits         6431     6437       +6     
  Misses       1920     1920              
  Partials      427      427              

Impacted Files	Coverage Δ
torchvision/models/detection/retinanet.py	73.62% <100.00%> (+0.63%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e33cc8...9d88383. Read the comment docs.

[datumbox]

I discussed with @fmassa and recommended a different optimisation approach I'm closing this and I'll follow up with a different PR.