Skip to content
Experimental data for reproducibility of Confident Learning paper results
Shell Python
Branch: master
Clone or download
Latest commit e23b463 Nov 15, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cifar10 add examples of experiments Nov 8, 2019
LICENSE Update LICENSE Nov 8, 2019
README.md Update README.md Nov 14, 2019

README.md

confidentlearning-reproduce

Experimental data for reproducibility of CIFAR-10 experimental results in the confident learning paper.

This repo depends on the cleanlab Python package to implement confident learning.

Because GitHub limits filesizes to 100MB, I cannot upload trained ResNet-50 models (180MB each), but for every setting, I upload an out log file with the accuracy at every batch and test accuracy at every epoch. The file naming conventions are as follows

  • out -- the log files during training
  • train_mask.npy -- boolean vector for which examples where pruned during training
  • cifar10__train__model_resnet50__pyx.npy -- Cross-validation out of sample predicted probabilities for CIFAR-10 under the given noisy labels settings
  • cifar10_noisy_labels -- folder containing all the noisy labels settings
  • experiments.bash -- examples of the commands run to generate results
  • cifar10_train_crossval.py -- training script to perform all cifar-10 experiments (get cross-validated probabilities, evaluate on test set, train on a masked input to remove noisy examples)

Need out-of-sample predicted probabilities for CIFAR-10 train set?

You can obtain standard (no noise added to label) predicted probabilities here.

These are computed using four-fold cross-validation with a ResNet50 architecture. You can download the out-of-sample predicted probabilities for all training examples in CIFAR-10 for various noise and sparsities settings here:

  • Noise: 0% | Sparsity: 0% | [LINK]
  • Noise: 20% | Sparsity: 0% | [LINK]
  • Noise: 40% | Sparsity: 0% | [LINK]
  • Noise: 70% | Sparsity: 0% | [LINK]
  • Noise: 20% | Sparsity: 20% | [LINK]
  • Noise: 40% | Sparsity: 20% | [LINK]
  • Noise: 70% | Sparsity: 20% | [LINK]
  • Noise: 20% | Sparsity: 40% | [LINK]
  • Noise: 40% | Sparsity: 40% | [LINK]
  • Noise: 70% | Sparsity: 40% | [LINK]
  • Noise: 20% | Sparsity: 60% | [LINK]
  • Noise: 40% | Sparsity: 60% | [LINK]
  • Noise: 70% | Sparsity: 60% | [LINK]

License

Copyright (c) 2017-2020 Curtis Northcutt. Released under the MIT License. See LICENSE for details.

You can’t perform that action at this time.