You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi there! Thanks for your inspiring work and releasing the code. I have a small question regarding the baseline results. I did not modify the code and run it with command python osr.py --dataset cifar10 --loss Softmax. If I understand correctly, this would be the baseline method, and according to the Table 1 in your paper, the result AUROC should be 67.7 for the CIFAR10 dataset. However, the log I obtained is as follows:
And the average AUROC is about 86.09, which is significantly higher than the results reported. I'd like to know if there is anything that I haven't done properly. Thanks in advance!
The text was updated successfully, but these errors were encountered:
The AUROC results of the baseline in our paper refer to the paper [1] that first proposed this experiment. Our implementation for baseline is indeed better than the AUROC results in [1]. For this problem, please refer to the following issue. lwneal/counterfactual-open-set#5
However, this result has been followed by many works.
To reevaluate the baseline, we add the result of OSCR (a better metric) in the paper and evaluate all baseline methods.
I hope it can help you.
[1] Open Set Learning with Counterfactual Images, ECCV 2018
Hi there! Thanks for your inspiring work and releasing the code. I have a small question regarding the baseline results. I did not modify the code and run it with command
python osr.py --dataset cifar10 --loss Softmax
. If I understand correctly, this would be the baseline method, and according to the Table 1 in your paper, the result AUROC should be67.7
for the CIFAR10 dataset. However, the log I obtained is as follows:And the average AUROC is about
86.09
, which is significantly higher than the results reported. I'd like to know if there is anything that I haven't done properly. Thanks in advance!The text was updated successfully, but these errors were encountered: