About torchmetrics Accuracy

[Aelees0807]

from torchmetrics import Accuracy

#setup metric
torchmetric_accuracy = Accuracy(task="multiclass" , num_classes = 4).to(device)

#Calculate accuracy
torchmetric_accuracy(y_preds , y_blob_test)

in the newer version we want to add "task" parameters , in our case the accuracy is 95% , but the torchmetric_accuracy is tensor(0.3550, device='cuda:0')

so what will be the reason behind it?

[yetval]

Hey, I looked into this. torchmetrics isn't the problem, when I give it the trained model's predictions it returns 0.9950, same as accuracy_fn.

A score of 0.3550 means the y_preds you passed in came from the untrained model. The notebook makes predictions once before training (in the logits to probabilities section) and once after, so if cells run out of order you end up scoring the untrained ones, which is basically random guessing for 4 classes.

Fix is to recompute predictions right before the metric:

model_4.eval()
with torch.inference_mode():
    y_preds = torch.softmax(model_4(X_blob_test), dim=1).argmax(dim=1)

torchmetric_accuracy(y_preds, y_blob_test)

That should match your 95%.