torchmetrics Accuracy

[Antoine101]

📚 Documentation

Hi Lightning team,

I was looking at metrics and found something a tad confusing.

In the following screenshot, the accuracy metric is imported like so:
torchmetrics.classification.Accuracy

But looking at this other page of the Torchmetrics doc, it shows: torchmetrics.Accuracy (without classification in between)

In my script, the following work:
import torchmetrics.Accuracy
import torchmetrics.classification.accuracy

But not the following:
import torchmetrics.classification.Accuracy

Which one is the right one?

If it is torchmetrics.Accuracy, why do the other classification metrics seem to require the classification in between?

Cheers

Antoine

cc @Borda

[Borda]

@Antoine101 thank you for sharing your convenes, just to clarify, this is related only to docs? and what version are you referring to?

[Antoine101]

Hi @Borda
Thanks for getting back so quickly

I let you judge of it, if you tell me it is not correct or explained well enough in the doc then it is a doc issue.

But yeah doc related.

I am talking torchmetrics 1.4.0.

[Borda]

But yeah doc related.

so no issue with importing metrics both ways:

• torchmetrics.Accuracy
• torchmetrics.classification.Accuracy

[Antoine101]

torchmetrics.Accuracy ✅
torchmetrics.classification.accuracy ✅
torchmetrics.classification.Accuracy ❌

Both the first and third ones pop up in the latest torchmetrics doc.

And so what is the difference between the two first ones?

[Borda]

torchmetrics.Accuracy ✅
torchmetrics.classification.accuracy ✅
torchmetrics.classification.Accuracy ❌

I can reproduce it, but on the other hand, this works:
from torchmetrics.classification import Accuracy

[Antoine101]

Sorry, all three work actually on my side as well!

But how do they all differ?

[Borda]

But how do they all differ?

Tbh, not sure...

[SkafteNicki]

Let me try to clarify

import torchmetrics.Accuracy
should not work. Reason being that there is no module/file in src/torchmetrics/* called Accuracy. Instead If you do
from torchmetrics import Accuracy
it will not look for a file called Accuracy.py but instead it looks in the src/torchmetrics/__init__.py file for a reference to a metric called Accuracy, which it gets from these lines of code: https://github.com/Lightning-AI/torchmetrics/blob/master/src/torchmetrics/__init__.py.

import torchmetrics.classification.accuracy
this should work, but you are not getting the accuracy metric you are instead importing the src/torchmetrics/classification/accuracy.py file which contains the definition to Accuracy metric. There is a difference between importing the file and importing the function/module you want.

import torchmetrics.classification.Accuracy
again should not work because there is no file called src/torchmetrics/classification/Accuracy.py. Instead if we do
from torchmetrics.classification import Accuracy
you will get the same Accuracy metric as in the first case.

The very short answer is: the recommendation is to ALWAYS import metrics from there respective domain e.g.

from torchmetrics.classification import Accuracy, Precision, Recall

or use full path references

import torchmetrics
metric = torchmetrics.classification.Accuracy(...)

however, some metrics can still be directly access from the root of torchmetrics e.g. torchmetrics.Accuracy and that is mostly due to historical reasons and to keep things backwards compatible. But that does not mean that there are multiple Accuracy metrics, just that there are different ways to import the same metric.

Closing issue.