Evaluation code bug

[leoshine]

Traceback (most recent call last):
  File "train.py", line 183, in <module>
    main()
  File "train.py", line 85, in main
    train_multi_label_coco(model, train_loader, val_loader, args.lr)
  File "train.py", line 139, in train_multi_label_coco
    mAP_score = validate_multi(val_loader, model, ema)
  File "train.py", line 173, in validate_multi
    mAP_score_regular = mAP(targs, preds)
  File "/home/sliao1/working/_20multilabel/code.others/ASL/src/helper_functions/helper_functions.py", line 64, in mAP
    ap[k] = average_precision(scores, targets)
  File "/home/sliao1/working/_20multilabel/code.others/ASL/src/helper_functions/helper_functions.py", line 41, in average_precision
    pos_count_[np.logical_not(ind)] = 0
IndexError: too many indices for array

Can you confirm the following is a bug and the fix is valid?

ASL/train.py

Line 168 in eb52197

mAP_score_regular = mAP(torch.cat(targets).numpy(), torch.cat(preds_regular).numpy())

Should be:

mAP_score_regular = mAP(torch.cat(targets).numpy()[:,-1,:], torch.cat(preds_regular).numpy())

since:

targets is of shape (num_sample, 3, 80) and preds_regular is of shape (num_sample, 80).

The same issue is also on:

ASL/train.py

Line 169 in eb52197

mAP_score_ema = mAP(torch.cat(targets).numpy(), torch.cat(preds_ema).numpy())

Not so sure which targets to pass in:

mAP_score_ema = mAP(torch.cat(targets).numpy()[:,i,:], torch.cat(preds_ema).numpy()) # i=0,1,2 ?

B.t.w, what does "Ema" stand for?

[mrT23]

this code was an external contribution.

see again the response to the previous "bug":
there are several variants of coco 2014 annotations
open a merge request that takes into account both variants, or just fix it locally

regarding EMA, i strongly recommend to get acquainted with modern techniques in deep learning, EMA (exponential-moving-average) is one of them.

[leoshine]

Thank @mrT23 for the suggestion!
@SlongLiu, the fix works for me.