Example Colab notebooks for collaborative training participants, from Appendix H1?

[cdleong]

I note that in Appendix H1 of the paper, it mentions Colab notebooks which participants can use to join in the training process. Can examples of these be released?

I am very fascinated by the potential of DeDLOC for decentralized organizations like Masakhane, so I'm curious how hard it might be to, say, setup the training of large translation models!

[mryab]

Hi, thank you for your interest in the project! Sorry for the delay; we've uploaded the example notebook here. If you need more details, we're happy to help.

[cdleong]

I had a look at the Notebook, and it looks really helpful! Thank you!

Plus it even links to a tutorial on how to get a group of volunteers together to train a language model: https://github.com/learning-at-home/hivemind/tree/master/examples/albert - lots of good info there as well. This is great, thanks for providing all this stuff.

[yhn112]

By the way, @cdleong, we've just made a discord server to communicate with ones who want to use our library for their experiments.
Perhaps it will be more convenient to discuss there.