compute_meandice input order

[ghost]

Hi,
I have two sets of expert ground-truth labels on some single class segmentations, and wish to quantify similarity of the segmentations to give a marker for my model's performance.

When I run my script, using compute_meandice, I get a different result depending on which way round I enter the data. Concretely:

Inputting [expert1-data] as the y_pred and [expert2-data] as the y results in a mean dice score of 0.88205

If instead I input [expert2-data] as the y_pred and [expert1-data] as the y results in a mean dice score of 0.880517

The difference is small, but I'd still like to get to the bottom of it! I've ensured I'm applying the same pre-processing transformations to both.
I can't figure out why there should be a difference, can anyone offer advice please? Thanks

[Nic-Ma]

Hi @ericspod and @wyli ,

Could you please help take a look at this question? You guys are expert researchers on DL training.

Thanks.

[wyli]

currently the class-wise Dice metric is set to 0 when there's no foreground in the ground truth label: https://github.com/Project-MONAI/MONAI/blob/master/monai/metrics/meandice.py#L127 this logic might make y and y_pred not interchangeable

[ghost]

Yep, that could be it. Thanks Wenqi!