Materials regarding decentralized AI research.
This includes literatures on consensus optimization (mainly from mainly control community). Hopefully one can borrow theory from them to build fully decentralized algorithms.
Dual averaging for distributed optimization: Convergence analysis and network scaling
2012 IEEE Transactions on Automatic control. Duchi, John C and Agarwal, Alekh and Wainwright, Martin J
On the convergence of decentralized gradient descent.
2016 SIAM Journal on Optimization. Yuan, Kun and Ling, Qing and Yin, Wotao
DSA: Decentralized double stochastic averaging gradient algorithm
2016 JMLR. Mokhtari, Aryan and Ribeiro, Alejandro.
Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent.
NIPS 2017. Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu
D^2: Decentralized training over decentralized data.
ICML 2018. Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, and Ji Liu.
Decentralized Linear Learning
- COLA: Decentralized Linear Learning
NIPS 2018. *Lie He, *An Bian, Martin Jaggi
Federated Learning & Optimization
This is one recent setting proposed to solve one existing decentralized data setting, but the algorithms therein are still centralized algorithms.
- Communication-Efficient Learning of Deep Networks from Decentralized Data
2017 AISTATS. McMahan, Brendan and Moore, Eider and Ramage, Daniel and Hampson, Seth and y Arcas, Blaise Aguera
This is one emerging setting where one wants to learn an intelligent model for mobile devices, such as mobile phones, tablets etc.
Randomized gossip algorithms
2006 EEE transactions on information theory. Boyd, Stephen and Ghosh, Arpita and Prabhakar, Balaji and Shah, Devavrat
2009 Foundations and Trends in Networking. Shah, Devavrat
Bitcoin: A Peer-to-Peer Electronic Cash System.
Satoshi Nakamoto. 2008.
When A Small Leak Sinks A Great Ship: Deanonymizing Tor Hidden Service Users Through Bitcoin Transactions Analysis.
Jawaheri, Husam Al and Sabah, Mashael Al and Boshmaf, Yazan and Erbad, Aimen. 2018.
- Ethereum: A Secure Decentralized Generalised Transaction Leger
Wood, Gavin. 2014.
Interplanetary File System (IPFS)
- IPFS-content addressed, versioned, P2P file system.
Benet, Juan. 2014.