simulation code - step1, data distribution

[martinjaggi]

provide a simulated decentralized code (not using any p2p backend, but instead just running locally), which holds a communication graph, and distributes a standard/toy ML dataset among the nodes.

data distribution should support both random and heterogeneous / non-iid (for example different labels for each node).

we can use standard PyTorch code examples e.g. MNIST or Cifar

[Saipraneet]

We can base this code off of this federated learning simulation codebase: https://github.com/AshwinRJ/Federated-Learning-PyTorch

The main training loop occurs here and it just uses nested foor loops: https://github.com/AshwinRJ/Federated-Learning-PyTorch/blob/master/src/federated_main.py

Reach out if you're interested in working on this: there are some gotchas with this codebase (bugs etc.)

[martinjaggi]

cool, this looks nice and compact. probably easy to run in colab too?

[Saipraneet]

Yep...I can also extract out the federated learning code into a single colab from the personalization project with Mahmoud. It should serve as a useful starting point.

[Saipraneet]

A super simple colab code for decentralized training with a CNN on MNIST data.
https://colab.research.google.com/drive/1DT4EaeEk9AuaFWMRaNkFEyQ9e-SphlUG?usp=sharing

Currently the code tests on the average model across the clients. This should probably be changed.

[martinjaggi]

just pinging @ThomasDCepfl here as he's working on it (based on the above excellent colab code by praneeth). he will keep us updated later...

[Saipraneet]

Committed the notebook: 1e208c4

Please feel free to re-open issue/unroll commit.