decentralized-ai.net

Materials regarding decentralized AI research.

Consensus Optimization

This includes literatures on consensus optimization (mainly from mainly control community). Hopefully one can borrow theory from them to build fully decentralized algorithms.

• Dual averaging for distributed optimization: Convergence analysis and network scaling
  2012 IEEE Transactions on Automatic control. Duchi, John C and Agarwal, Alekh and Wainwright, Martin J

• On the convergence of decentralized gradient descent.
  2016 SIAM Journal on Optimization. Yuan, Kun and Ling, Qing and Yin, Wotao

• DSA: Decentralized double stochastic averaging gradient algorithm
  2016 JMLR. Mokhtari, Aryan and Ribeiro, Alejandro.

Decentralized SGD

• Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent.
  NIPS 2017. Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu

• D^2: Decentralized training over decentralized data.
  ICML 2018. Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, and Ji Liu.

Decentralized Linear Learning

• COLA: Decentralized Linear Learning
  NIPS 2018. *Lie He, *An Bian, Martin Jaggi

Federated Learning & Optimization

This is one recent setting proposed to solve one existing decentralized data setting, but the algorithms therein are still centralized algorithms.

• Communication-Efficient Learning of Deep Networks from Decentralized Data
  2017 AISTATS. McMahan, Brendan and Moore, Eider and Ramage, Daniel and Hampson, Seth and y Arcas, Blaise Aguera

Mobile AI

This is one emerging setting where one wants to learn an intelligent model for mobile devices, such as mobile phones, tablets etc.

Gossip Algorithm

• Randomized gossip algorithms
  2006 EEE transactions on information theory. Boyd, Stephen and Ghosh, Arpita and Prabhakar, Balaji and Shah, Devavrat

• Gossip algorithms
  2009 Foundations and Trends in Networking. Shah, Devavrat

Bitcoin

• Bitcoin: A Peer-to-Peer Electronic Cash System.
  Satoshi Nakamoto. 2008.
  https://bitcoin.org/bitcoin.pdf

• When A Small Leak Sinks A Great Ship: Deanonymizing Tor Hidden Service Users Through Bitcoin Transactions Analysis.
  Jawaheri, Husam Al and Sabah, Mashael Al and Boshmaf, Yazan and Erbad, Aimen. 2018.
  https://arxiv.org/pdf/1801.07501.pdf?utm_source=securitydailynews.com

Ethereum

• Ethereum: A Secure Decentralized Generalised Transaction Leger
  Wood, Gavin. 2014.
  http://gavwood.com/paper.pdf

Interplanetary File System (IPFS)

• IPFS-content addressed, versioned, P2P file system.
  Benet, Juan. 2014.
  https://ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ipfs.draft3.pdf