Clarification on how test time accuracy is calculated

[devavratTomar]

Dear Authors,
Many thanks for the great work. While going through the paper I could not understand how you compute the final accuracy on the test samples. Is the accuracy computed in an online manner (evaluate the model on the current batch as it trains on) or offline manner (evaluate the whole test set after training is finished)?

Thanks.

[YuejiangLIU]

Thanks for your interest! We evaluated TTA methods offline, i.e., the accuracy of the whole test set after adaptation. Please refer to the following lines of our code:

ttt-plus-plus/cifar/ttt++.py

Line 327 in e429a19

err_cls = test(teloader, net)[0]

ttt-plus-plus/cifar/shot.py

Line 177 in e429a19

err_cls = test(teloader, net)[0]

[devavratTomar]

Thank you!