You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I ran the experiments for ood_loc_gpn_{10,16} to reproduce the results. The metrics are logged after training as follows:
val
test
accuracy
0.923077
0.889306
brier_score
0.280527
0.269254
ECE
0.10638
0.0702577
confidence_aleatoric_apr
0.966722
0.980251
confidence_epistemic_apr
0.95776
0.962611
confidence_structure_apr
nan
nan
confidence_aleatoric_auroc
0.706276
0.864282
confidence_epistemic_auroc
0.635031
0.772384
confidence_structure_auroc
nan
nan
CE
0.37979
0.339846
avg_prediction_confidence_aleatoric
0.816697
0.819579
avg_prediction_confidence_epistemic
14919.8
14410.5
avg_sample_confidence_aleatoric
0.816697
0.819579
avg_sample_confidence_epistemic
16556.9
15927.3
avg_sample_confidence_features
15835.7
16779.6
avg_sample_confidence_neighborhood
nan
nan
average_entropy
0.513569
0.501767
ood_detection_aleatoric_apr
0.801985
0.78415
ood_detection_aleatoric_auroc
0.833289
0.836685
ood_detection_epistemic_apr
0.80048
0.788122
ood_detection_epistemic_auroc
0.852889
0.855747
ood_detection_features_apr
0.75896
0.748484
ood_detection_features_auroc
0.81794
0.8237
ood_detection_neighborhood_apr
nan
nan
ood_detection_neighborhood_auroc
nan
nan
ood_detection_structure_apr
nan
nan
ood_detection_structure_auroc
nan
nan
ood_accuracy
0
0
ood_avg_prediction_confidence_aleatoric
0.556919
0.558297
ood_avg_prediction_confidence_epistemic
2879.92
2808.5
ood_avg_sample_confidence_aleatoric
0.556919
0.558297
ood_avg_sample_confidence_epistemic
4068.24
4109.26
ood_avg_sample_confidence_neighborhood
nan
nan
ood_avg_sample_confidence_features
2063.81
1759.3
ood_average_entropy
1.03177
1.03767
id_accuracy
0.923077
0.889306
id_avg_prediction_confidence_aleatoric
0.816697
0.819579
id_avg_prediction_confidence_epistemic
14919.8
14410.5
id_avg_sample_confidence_aleatoric
0.816697
0.819579
id_avg_sample_confidence_epistemic
16556.9
15927.3
id_avg_sample_confidence_features
15835.7
16779.6
id_average_entropy
0.513569
0.501767
Is this experiment exclusive for the With Effect case? I would like to reproduce 3 cases: aleatoric w/Net, Epist w/Net and Epist w/o Net as table 1 in the paper. Could you provide me with some guidance? Thank you.
The text was updated successfully, but these errors were encountered:
Hi, I ran the experiments for ood_loc_gpn_{10,16} to reproduce the results. The metrics are logged after training as follows:
Is this experiment exclusive for the With Effect case? I would like to reproduce 3 cases: aleatoric w/Net, Epist w/Net and Epist w/o Net as table 1 in the paper. Could you provide me with some guidance? Thank you.
The text was updated successfully, but these errors were encountered: