Mobile Working Group decision is to use CLIP Score on MS Coco 2014 mage Captions selected captions as the accuracy metric
Below are the CLIP accuracy results from v5 submissions
Qualcomm: 25.61%
Samsung: 25.47%
FP32 (PyTorch): 25.31%
Suggestion is to adopt 25.00% as the accuracy threshold for the CLIP Score,
@mohitmundhragithub @freedomtan @swasson488 Need finalize the threshold before v6 deadline
Mobile Working Group decision is to use CLIP Score on MS Coco 2014 mage Captions selected captions as the accuracy metric
Below are the CLIP accuracy results from v5 submissions
Qualcomm: 25.61%
Samsung: 25.47%
FP32 (PyTorch): 25.31%
Suggestion is to adopt 25.00% as the accuracy threshold for the CLIP Score,
@mohitmundhragithub @freedomtan @swasson488 Need finalize the threshold before v6 deadline