V-COCO Evaluation Error

[kilickaya]

Hello,

Many thanks for your great work.

I am trying to evaluate your pre-trained models on V-COCO.

1. So, I first generate the official detection pickle via:

python generate_vcoco_official.py \
        --param_path ./params/qpic_resnet50_vcoco.pth \
        --save_path ./logs_vcoco/vcoco.pickle \
        --hoi_path ./data/v-coco

2. Later, I use the official evaluation code via the following:

from vsrl_eval import VCOCOeval

vsrl_annot_file_s='./data/v-coco/data/vcoco/vcoco_val.json'
split_file_s='./data/v-coco/data/splits/vcoco_val.ids'

coco_file_s='./data/v-coco/data/instances_vcoco_all_2014.json'
vcocoeval = VCOCOeval(vsrl_annot_file_s, coco_file_s, split_file_s)

file_name= './logs_vcoco/vcoco.pickle'
vcocoeval._do_eval(file_name, ovr_thresh=0.5)

Please note that I adapted the latter script from VSG-Net repo. I face the following error during evaluation:

zero-size array to reduction operation minimum which has no identity

on assert(np.amax(rec) <= 1) within _do_agent_eval() function execution.

I wonder if this is a common error, and how it can be mitigated?

Many thanks.

[kilickaya]

Resolved by changing val to test set. Some images in the validation set do not return any results failing the evaluation.