You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When ignore_index is used in MulticlassPrecision, it zeros out the metric on the class to ignore but it still includes the value in the average computation, leading to wrong results.
To Reproduce
importtorchfromtorchmetrics.classificationimportMulticlassPrecision# simulate the output of a perfect predictor (i.e. preds == target)target=torch.tensor([0, 1, 2, 0, 1, 2])
preds=target.clone()
metric=MulticlassPrecision(num_classes=3, average='none', ignore_index=0)
res=metric(preds, target)
print(res)
# it prints [0., 1., 1.]metric=MulticlassPrecision(num_classes=3, average='macro', ignore_index=0)
res=metric(preds, target)
print(res)
# it prints 0.6667 instead of 1
Expected behavior
It should not consider the ignored class in the average computation.
Environment
TorchMetrics version: 0.11.4 installed from pip
Python & PyTorch Version: 3.7.9 - 1.11.0+cu113
Any other relevant information such as OS (e.g., Linux): Linux Mint 18.3 Sylvia
The text was updated successfully, but these errors were encountered:
馃悰 Bug
When ignore_index is used in MulticlassPrecision, it zeros out the metric on the class to ignore but it still includes the value in the average computation, leading to wrong results.
To Reproduce
Expected behavior
It should not consider the ignored class in the average computation.
Environment
The text was updated successfully, but these errors were encountered: