Learning from Label Proportion by Learning from Label Noise

1. Implementation of LLPFC algorithms. Please Cite:

@inproceedings{
zhang2022learning,
title={Learning from Label Proportions by Learning with Label Noise},
author={Jianxin Zhang and Yutong Wang and Clayton Scott},
booktitle={Advances in Neural Information Processing Systems},
editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
year={2022},
url={https://openreview.net/forum?id=cqyBfRwOTm1}
}

2. Dependencies:
   • download and install Anaconda
   • create an environment and install the dependencies: conda env create -f LLPFC.yml. If failed, you may want to install the packages listed in LLPFC.yml manually.
   • activate the new environment: conda activate LLPFC
3. Generate LLP data:
   • run python make_data.py -h to check the usage
   • Example:

              python make_data.py -d cifar10 -c 10 -s cifar10_paper_BS256_BN160_T0 \
                    -l ./data/labeled_data/  \
                    -p ./data/llp_data/ \
                    -m dirichlet -b 256 -n 160 -r 15671628
   

   • -d, --dataset: Create an LLP dataset using one of the three datasets used in the paper. Must be one of ["cifar10", "svhn", "emnist_letters"].
   • -c, --num_classes: An integer. the number of classes.
   • -s, --data_save_name: The file name to save synthesiszed LLP dataset.
   • -l, --data_folder_labeled: The path to the folder of fully labele data. If the dataset does not exist already, it will be downloaded by torchvision.
   • -p, --data_folder_llp: Folder to save the systhesized LLP data.
   • -m, --method: Distribution to sample true label proportions. Must be one of ["dirichlet", "uniform"].
   • -a, --alpha: Decprepted. Update in future.
   • -b, --bag_size: Size of bag.
   • -n, --num_bags: Number of bags to generate; it too large, the dataset may have insufficient data points.
   • -r, --seed: An integer. The seed for all RNG.
4. Run Experiments:
   • run python main.py -h to check the usage
   • Example:

   python main.py -d cifar10 -c 10 -v 0 -a llpfc --seed 0 \  
                -p ./data/llp_data/cifar10_paper_BS256_BN160_T0 \  
                -f ./data/labeled_data/ \  
                -n wide_resnet_d_w -wrnd 16 -wrnw 4 -e 200 -r 20 -b 128 -dr 0.3 \  
                -o nesterov -l 1e-1 -w uniform -wd 5e-4 -m 0.9 -sc drop -ms 60 120 160 -ga 0.2 \  
                -log ./log_llpfc_BS256_BN160_T0 \  
                -s ./llpfc_BS256_BN160_T0.pth
   

   • -d, --dataset: Name of the dataset to be used. Must be one of ["cifar10", "svhn", "emnist_letters"].
   • -p, --path_lp: Path to the label proportion dataset generated by make_data.py.
   • -c, --num_classes: An integer. The number of classes.
   • -f, --data_folder_labeled: Folder of the test split of fully labeled data. If not exists, the dataset will be downloaded by torchvision.
   • -log, --logging_filename: Path to save the log file.
   • -a, --algorithm: The algorithm to solve the LLP problem. Must be one of ["llpfc", "kl", "llpvat", "llpgan"].
   • -n, --network: The neural networks to be used. Must be one of ["wide_resnet_d_w", "nin", "ResNet18", "vgg19_bn", "vgg16_bn", "densenet121"].
   • -wrnd, --WideResNet_depth: The depth of Wide ResNet. No effect on other networks.
   • -wrnw, --WideResNet_width: The width of Wide ResNet. No effect on other networks.
   • -dr, --drop_rate: The dropout rate in the dropout layers for Wide ResNet. No effect on other networks.
   • -o, --optimizer: The optimizer. Must be one of ["Adamax", "LBFGS", "Adagrad", "nesterov", "AdamW", "SGD"]. The "LBFGS" option is not fully implemented.
   • -ams, --amsgrad: Whether to use the AMSGrad variant of AdamW. No effect on other optimizers. Must be one of [0, 1].
   • -l, --lr: Learning Rate.
   • -m, --momentum: The momentum for SGD. No effect on other optimizers.
   • -wd, --weight_decay: The l_2 regularization paramter.
   • -e, --total_epochs: Total number of epochs to train.
   • -r, --num_epoch_regroup: Groups will be regenerated every this number of epochs. Effective only if the algorithm is llpfc.
   • -np, --noisy_prior_choice: The heuristics to estimate the noisy prior for each group. Must be one of ["approx", "uniform", "merge"]. "approx" implements LLPFC-approx, solving the constrained optimization for prior estimation. "uniform" implements LLPFC-uniform, assigning uniform noisy prior. "approx" and "uniform" circumvent Assumption 14 in the paper. "merge" merges all groups into one big group so Assumption 14 in the paper is always satisfied but in the sacrafice of performance.
   • -v, --validate: Decrypted as not implemented for all algorithms.
   • -b, --train_batch_size: Training batch size. Note for "llpfc", it sets the number of data points in a batch. For "kl", "llpvat", "llpgan", it sets the number of bags in a batch.
   • -t, --test_batch_size: Test batch size.
   • -s, --save_path: Path to save the trained model. Model will not be saved if it is None.
   • -dv, --device: Device to train network. Must be one of ["cuda", "cpu", "check"]. If "check", use cuda whenever it's available.
   • -w, --weights: Set the weights for groups in llpfc. Must be one of ["uniform", "ch_vol"]. "uniform" sets uniform weights. "ch_vol" sets the weights by volume of covex hull of label proportions in a group.
   • -sc, --scheduler: Set the scheduler of training lr. Must be one of ["drop", "CAWR"]. "drop" decreases the learning rate by a constant. "CAWR" uses the CAWR method.
   • -ms, --milestones: Number of epochs to drop lr. Effective only if --scheduler is 'drop'.
   • -ga, --gamma: Drop the learning rate by this factor, Effective only if --scheduler is 'drop'.
   • -T0, --T_0: Parameter of the CAWR scheduler. Effective only if --scheduler is 'CAWR'.
   • -Tm, --T_mult: Parameter of the CAWR scheduler. Effective only if --scheduler is 'CAWR'.
   • --seed: seed for all RNG.
   • -fr, --full_reproducibility: Disable all random algorithms in CUDA for reproducibility. Decrypted as this feature is in beta test by the time of the publication of this paper.
   • -xi, --vat_xi: Parameter for vat loss. Effective only if algorithm is 'llpvat'.
   • -eps, --vat_eps: Parameter for vat loss. Effective only if algorithm is 'llpvat'.
   • -ip, --vat_ip: Parameter for vat loss. Effective only if algorithm is 'llpvat'.
   • -nd, --noise_dim: Parameter for llpgan. The input dimension of the generator. Effective only if algorithm is 'llpgan'.
   • -js, --path_to_json: "Path to write training results. Write nothing if it is none