Really slow performance of evaluate_single_core

[nmddc0211]

Hi, thank you for your work and the wonderful datasets,
I have some problems evaluating (and also training) my model because the evaluation script is really slow. For reference, I also ran the evaluation script of panopticapi and it took about 3 seconds while it took my computer 3 seconds per image to calculate PartPQ. I also notice that panopticapi uses all the cores and PartPQ only uses half of the cores. Is it because of performance issue? I'd be glad if you could tell me whether if this kind of slow performance is expected and maybe I should not evaluate after every epoch.

annotation_parsing in 0.991 seconds
pred_reader_fn in 0.116 seconds
prediction_parsing in 0.896 seconds
generate_ignore_info_tiff in 0.062 seconds
add removed segments to crowd in 0.002 seconds
ignore_img_parsing in 0.021 seconds
pq_part in 1.217 seconds
Processed one image in 3.312 seconds

Regards,
Dang Nguyen

[pmeletis]

Dear Dang,
We are working on accelerating evaluation scripts and adding extra functions to the repository. Soon they will be online. For now, I suggest you run the evaluation after every epoch on a subset of images/labels, and on the whole dataset a few times throughout the training.

[nmddc0211]

Thank you for your suggestion! I'm looking forward for the improvements.