Repository navigation
Experiments
Tonio Weidler edited this page Jan 5, 2020
·
31 revisions
All strategies use the default circular padding without a self-connection. Second set of accuracies refers to results with random augmentations of the test set.
| Strategy | Accuracy | Uncertainty | Accuracy | Uncertainty | N |
|---|---|---|---|---|---|
| Baseline Alexnet | 83.33 | 0.05 | 81.04 | 0.05 | 180 |
| Baseline AlexCMap | 83.40 | 0.10 | 81.11 | 0.13 | 30 |
| SSI Frozen | 84.05 | 0.09 | 81.79 | 0.12 | 30 |
| SSI Adaptive | 84.20 | 0.11 | 81.97 | 0.10 | 30 |
| Converged Frozen | 84.09 | 0.12 | 81.76 | 0.12 | 30 |
| Converged Adaptive | 84.20 | 0.09 | 81.96 | 0.14 | 30 |
| Parametric | 83.98 | 0.09 | 81.85 | 0.14 | 30 |
| Strategy | Accuracy | Uncertainty | Accuracy | Uncertainty | N |
|---|---|---|---|---|---|
| Converged Frozen | 84.09 | 0.12 | 81.76 | 0.12 | 30 |
| Converged Adaptive | 84.20 | 0.09 | 81.96 | 0.14 | 30 |
| Parametric | 83.98 | 0.09 | 81.85 | 0.14 | 30 |
| Converged Frozen Self | 84.09 | 0.26 | 81.82 | 0.17 | 10 |
| Converged Adaptive Self | 83.99 | 0.15 | 82.00 | 0.15 | 10 |
| Parametric Self | 84.05 | 0.21 | 81.77 | 0.17 | 10 |
| Strategy | Accuracy | Uncertainty | Accuracy | Uncertainty | N |
|---|---|---|---|---|---|
| SSI Adaptive | 84.20 | 0.11 | 81.97 | 0.10 | 30 |
| SSI Frozen | 84.05 | 0.09 | 81.79 | 0.12 | 30 |
| Converged Frozen | 84.09 | 0.12 | 81.76 | 0.12 | 30 |
| Converged Adaptive | 84.20 | 0.09 | 81.96 | 0.14 | 30 |
| Parametric | 83.98 | 0.09 | 81.85 | 0.14 | 30 |
| SSI Adaptive Zero | 84.18 | 0.07 | 81.97 | 0.09 | 60 |
| SSI Frozen Zero | 84.10 | 0.19 | 81.75 | 0.25 | 10 |
| Converged Frozen Zero | 84.02 | 0.07 | 81.81 | 0.08 | 60 |
| Converged Adaptive Zero | 84.19 | 0.06 | 81.96 | 0.09 | 60 |
| Parametric Zero | 83.98 | 0.07 | 81.79 | 0.09 | 60 |
The Converged Adaptive Coverage Experiments used scope=27, width=3 and damp=0.1 in all layers.
| Strategy | Accuracy | Uncertainty | Accuracy | Uncertainty | N |
|---|---|---|---|---|---|
| Parametric full | 84.09 | 0.09 | 81.92 | 0.09 | 30 |
| Parametric @1 | 83.98 | 0.09 | 81.85 | 0.14 | 30 |
| Parametric @1,2 | 84.04 | 0.11 | 81.77 | 0.11 | 30 |
| Parametric @1,2,3 | 83.92 | 0.12 | 81.82 | 0.11 | 30 |
| Converged Adaptive full | 84.33 | 0.09 | 82.11 | 0.10 | 30 |
| Converged Adaptive @1 | 84.20 | 0.09 | 81.96 | 0.14 | 30 |
| Converged Adaptive @1,2 | 84.18 | 0.11 | 82.06 | 0.11 | 30 |
| Converged Adaptive @1,2,3 | 84.24 | 0.13 | 82.05 | 0.09 | 30 |
| Best Full |
| Strategy | Dataset | Accuracy | Uncertainty | N |
|---|---|---|---|---|
| VGG-19 | CIFAR-10 | |||
| VGG-19 + Best | CIFAR-10 | |||
| AlexNet | CIFAR-10 | |||
| AlexNet + Best | CIFAR-10 | |||
| VGG-19 | ImageNet | |||
| VGG-19 + Best | ImageNet | |||
| AlexNet | ImageNet | |||
| AlexNet + Best | ImageNet | |||
| VGG-19 | MNIST | |||
| VGG-19 + Best | MNIST | |||
| AlexNet | MNIST | |||
| AlexNet + Best | MNIST | |||
| CapsNet + Best | MNIST |