PyTorch_YOLOv4 - multiple best weights for testing. Which one to use?

[valentinitnelav]

Hi @stark-t ,

When setting a job run using detect.py on the test dataset I remembered again that for YOLOv4 we have multiple best weights options. There are "best" options for several epochs and then these options:

best_ap50.pt    # ?
best_ap.pt      # probably best in terms of average precision
best_f.pt       # probably best in terms of F score
best_overall.pt # overall what?
best_p.pt       # probably best in terms of Precision
best.pt         # best in what?
best_r.pt       # probably best in terms of Recall

For now, I decided to use best_overall.pt in detect.py, but without checking what this imply actually. Yolov7 & v5 have a best.pt , but not _ap, _ap50, _f, _p, _r, _overall options.

Here are all the files containing the "best*" keyword in a runs/train folded for yolov4r:

ls ~/PAI/detectors/PyTorch_YOLOv4/runs/train/yolov4_pacsp_s_b8_e300_img640_hyp_custom/weights/ | grep best*

best_200.pt
best_201.pt
best_202.pt
best_203.pt
best_204.pt
best_205.pt
best_206.pt
best_207.pt
best_208.pt
best_209.pt
best_210.pt
best_211.pt
best_212.pt
best_213.pt
best_214.pt
best_215.pt
best_216.pt
best_217.pt
best_218.pt
best_219.pt
best_220.pt
best_221.pt
best_222.pt
best_223.pt
best_224.pt
best_225.pt
best_226.pt
best_227.pt
best_228.pt
best_229.pt
best_230.pt
best_231.pt
best_232.pt
best_233.pt
best_234.pt
best_235.pt
best_236.pt
best_237.pt
best_238.pt
best_239.pt
best_240.pt
best_241.pt
best_242.pt
best_243.pt
best_244.pt
best_245.pt
best_246.pt
best_247.pt
best_248.pt
best_249.pt
best_250.pt
best_251.pt
best_252.pt
best_253.pt
best_254.pt
best_255.pt
best_256.pt
best_257.pt
best_258.pt
best_259.pt
best_260.pt
best_261.pt
best_262.pt
best_263.pt
best_264.pt
best_265.pt
best_266.pt
best_267.pt
best_268.pt
best_269.pt
best_270.pt
best_271.pt
best_272.pt
best_273.pt
best_274.pt
best_275.pt
best_276.pt
best_277.pt
best_278.pt
best_279.pt
best_280.pt
best_281.pt
best_282.pt
best_283.pt
best_284.pt
best_285.pt
best_286.pt
best_287.pt
best_288.pt
best_289.pt
best_290.pt
best_291.pt
best_292.pt
best_293.pt
best_294.pt
best_295.pt
best_296.pt
best_297.pt
best_298.pt
best_ap50.pt
best_ap.pt
best_f.pt
best_overall.pt
best_p.pt
best.pt
best_r.pt

[stark-t]

pt file	metric
best_ap50.pt	ap50%-90%
best_ap	ap90%
best_f	f1 score
best_overall	overall accuracy
best_p	precission
best	fitness (mean ap&ap50)
best_r	recall

[stark-t]

i think the simple best.pt file should be used since the other yolo version should use the same. But I will check if all yolo version use the same function to calculate the "best fitness".

[stark-t]

• YOLOv7: w = [0.0, 0.0, 0.1, 0.9] # weights for [P, R, mAP@0.5, mAP@0.5:0.95] return (x[:, :4] * w).sum(1)
• YOLOv5: w = [0.0, 0.0, 0.1, 0.9] # weights for [P, R, mAP@0.5, mAP@0.5:0.95] return (x[:, :4] * w).sum(1)
• YOLOv4: w = [0.0, 0.0, 0.1, 0.9] # weights for [P, R, mAP@0.5, mAP@0.5:0.95] return (x[:, :4] * w).sum(1)

[valentinitnelav]

We don't implement YOLOv4 any longer. But this remains generally valid.

@stark-t , feel free to close this issue if you think it should be closed.