FedEditor is an efficient and effective federated unlearning (FU) framework. It minimizes degradation of the global model's predictive performance on un-forgotten data, while effectively eliminating unlearned data's influence from the global model without the participation of other clients and additional time-consuming retraining.
Fig. 1: The system model of FedEditor.
Fig. 2: The framework of the project.
Create a conda environment
git clone https://github.com/XXiaoY/Fededitor.git
conda create -n fededitor python=3.8.13
conda activate fededitor
pip install torch==2.0.0+cu118 torchvision==0.15.1+cu118 torchaudio==2.0.1+cu118 --index-url https://download.pytorch.org/whl/cu118
Install other dependencies
pip install -r requirements.txt
The training data can be easily split into IID and non-IID versions. We use the Fmnist dataset as an example.
# IID
nohup python -u gen_targetdata.py -data fmnist -poi True -ratio 0.1 -nc 10 -poi_num 2 > gen_fmnist.out 2>&1 &
# NON-IID
nohup python -u gen_targetdata.py -data fmnist -poi True -ratio 0.1 -nc 10 -poi_num 2 -iid False -dirichlet 1.0 > gen_fmnist_dir_1.0.out 2>&1 &
Basic parameters:
data: The name of experiment dataset.poi: Indicates whether the data is poisoned. Poisoned data can be considered as unlearned data.ratio: The proportion of unlearned data to the target client's training data.nc: The total number of clients.poi_num: The number of target clients that need to be unlearned.iid: The distribution of the data.dirichlet: Dirichlet distribution is used to model the data distribution.
We employ five non-member data scenarios with varying distribution shifts from local training distribution. We use the Cifar10 dataset as an example.
# OOD
nohup python -u gen_ood.py -data cifar10 -poi True -ratio 0.1 -nc 10 -poi_num 2 -style 0 > gen_cifar10_s0.out 2>&1 &
Basic parameters:
style: 0 constructs in-distribution non-member data using test data that shares a similar distribution with training data. 1 simulates label distribution shift using test data that is non-IID with the training data. 2 generates noise-corrupted data by applying common perturbations (e.g., Gaussian noise, fog, blur) to the local training data. 3 simulates the natural co-variate shift using CIFAR10.1 data as non-member data. 4 uses STL10 test data as out-of-distribution non-member data, which is a commonly used dataset in domain adaptation.
Federated unlearning (FU) removes the influence of a certain subset of clients' training data from the trained global model. Therefore, we first need to obtain the trained global model. Here is an example of running FedAvg on Fmnist dataset with LeNet model.
# IID
nohup python -u main.py -mode Fedavg -gr 100 -data fmnist -did 0 -ratio 0.1 -ls 1 -lr 0.01 -lbs 128 -nc 10 -poi_num 2 >> fmnist_0.out 2>&1 &
# NON-IID
nohup python -u main.py -mode Fedavg -gr 100 -data fmnist -did 3 -ratio 0.1 -ls 1 -lr 0.01 -lbs 128 -nc 10 -poi_num 2 -iid False -dirichlet 1.0 >> fmnist_d1.out 2>&1 &
Basic parameters:
mode: The FedAvg algorithm.gr: The communication rounds between clients and server.ls: The training epochs of each clients.lr: The learning rate.lbs: The batch size.
Here is an example to run FedEditor on Fmnist with LeNet:
(
datasets="fmnist"
muval="0.1 0.5 0.0 1.0 5.0 10.0"
alphaval="0.001 0.01 0.1 0.0 1.0"
learning_rates="0.001 0.005 0.01 0.05"
for dataset in $datasets; do
for mu in $muval; do
for alpha in $alphaval; do
for lr in $learning_rates; do
nohup python -u main.py -mode Fededitor -gr 100 -data "${dataset}" -did 3 -nc 10 -poi_num 2 -ratio 0.1 -ls 1 -lr "${lr}" -lbs 128 -mu "${mu}" -alpha "${alpha}" >> "${log_dir}/${dataset}_7.out" 2>&1
done
done
done
done
) &
Basic parameters:
mode: The unlearning algorithm, can choose Retrain, Contrain, GradientA, Randomlabel, and Fededitor.dataset: The name of experiment dataset.mu: The weight of knowledge unlearning loss.alpha: The weight of regularization term.
You can use the following evaluation methods to verify the unlearning effectiveness of FedEditor.
nohup python -u eval.py -mode Fedavg -gr 100 -data fmnist -did 3 -ratio 0.1 -ls 1 -lr 0.01 -lbs 128 -nc 10 -poi_num 2 >> fmnist.out 2>&1 &
For more detailed parameters setting, you can check the main.py. We offer a large number of parameter options.
Additionally, the run.sh offers some command line samples that can run directly for the fast simulation.
For the best practices of the baselines, please refer to the following code links:
- FedAvg. Basic federated learning framework.
- Model-Contrastive Federated Learning. Model-based contrastive learning loss.
- Federated Optimization in Heterogeneous Networks. Drift-mitigating regularization term.
- The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining. / The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining. Rapid Retraining scheme.
- Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learning. Complete retraining scheme.
- FedEraser: Enabling Efficient Client-Level Data Removal from Federated Learning Models. FedEraser scheme and membership inference attacks.
- An Empirical Study of Federated Unlearning: Efficiency and Effectiveness. Continue training scheme.
- Amnesiac Machine unLearning. Random labeling scheme.
- Backdoor Defense with Machine Unlearning. Gradient ascent scheme.
- REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data. Elastic weight consolidation loss.
- Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher. Output-level unlearning scheme via KL divergence, Jensen-Shannon divergence, and activation distance.
- Federated Unlearning: How to Efficiently Erase a Client in FL? Projected gradient descent scheme.
- DBA: Distributed Backdoor Attacks against Federated Learning. LOAN dataset and backdoor attacks.
- Membership Inference via Backdooring. Backdoor-assisted membership inference.
- Towards Unbounded Machine Unlearning. Distribution of entropy.
If you find the repo useful, please consider citing:
FedEditor