CmpE 491 Federated ML for Covid-19
Read the articles
Federated Learning Challenges, methods, and future directions https://arxiv.org/pdf/1908.07873.pdf, Learned about:
- Federated Learning Concepts
- Network Topologies used in FL
- Compression Schemes
- Difficulty that comes with non i.i.d. data
- Privacy for Federated Learning
Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data https://www.nature.com/articles/s41598-020-69250-1, Learned about
- Cyclic institutional incremental learning
- Institutional incremental learning
- Catastrophic forgetting and how FL becomes a solution for catastrophic forgetting
- Gained insight about experimenting methodologies used
Federated Learning: Collaborative Machine Learning without Centralized Training Data https://ai.googleblog.com/2017/04/federated-learning-collaborative.html
Differential Privacy for Everyone https://download.microsoft.com/download/D/1/F/D1F0DFF5-8BA9-4BDF-8924-7816932F6825/Differential_Privacy_for_Everyone.pdf
Read the article
- Communication-Efficient Learning of Deep Networks from Decentralized Data https://arxiv.org/pdf/1602.05629.pdf
- How cost functions of a Federated Learning differs from normal ML cost functions
- Federated Averaging (FedAvg) Algorithm
- System heterogenity and ways to tackle this problem (FedProx algorithm, weighted averaging)
Followed the Tutorials
- Federated Learning for Image Classification https://www.tensorflow.org/federated/tutorials/federated_learning_for_image_classification
- Custom Federated Algorithms, Part 2: Implementing Federated Averaging https://www.tensorflow.org/federated/tutorials/custom_federated_algorithms_2
Plans for the Upcoming Week
- I'm planning on getting more Familiar with TensorFlow by watching tutorials and making small models.
- After learning more about TensorFlow, I'm planning on familiarizing myself more with Tensor Flow Federated and making the model for MNIST (or maybe CIFAR) image classification from scratch.
- I'm planning on reading more articles about Federated Learning.
03.11.2020 - 09.11.2020 Done
- Learned aobut TensorFlow and TensorFlow Federated
- Worked on basic CIFAR10 model: https://colab.research.google.com/drive/1NFW_YXpptkM6O0lZ-xQN-PE7DOjo1k2M#scrollTo=DFSsHIeEV9k8
Plans for the upcoming week
Search for Covid-19 Databases that can be used.
Read at least 3 papers about Federated Learning
Practice more with tensor flow federated:
- with more complex models
- with different datasets
09.11.2020 - 16.11.2020 Done
- Implemented a Client Server architecture from scratch using keras, tf and python.
- Briefly searched for databases that could be used for the project: Federated Learning for Covid-19.
Plans for the upcoming week
- Implement a deeper model using ResNet or similar architecture.
- Learn more about TFF since the client-server architecture I implemented has overhead when distributing the model weights and reduces speed.
- Literature scan for
layer specific trainingusing Federated Learning.
16.11.2020 - 23.11.2020
- Read the paper :Averaging_Is_Probably_Not_the_Optimum_Way_of_Aggregating_Parameters_in_Federated_Learning https://www.researchgate.net/publication/339880491_Averaging_Is_Probably_Not_the_Optimum_Way_of_Aggregating_Parameters_in_Federated_Learning/fulltext/5e6a2cde92851c20f322812c/Averaging-Is-Probably-Not-the-Optimum-Way-of-Aggregating-Parameters-in-Federated-Learning.pdf
- Read the paper : Client-Edge-Cloud Hierarchical Federated Learning https://arxiv.org/pdf/1905.06641.pdf Read the paper:Dynamic FL https://www.researchgate.net/publication/339399233 Read a paper about Coded Federated Learning
- Tried to get more insight about using TensorFlow Federated