Experiments in trying to implement the Sub-linear Deep Learning Engine from [1] and the follow up paper MONGOOSE from [2]
References:
-
Chen, Beidi, Tharun Medini, James Farwell, Charlie Tai, and Anshumali Shrivastava. "Slide: In defense of smart algorithms over hardware acceleration for large-scale deep learning systems." Proceedings of Machine Learning and Systems 2 (2020): 291-306.
-
Chen, Beidi, et al. "MONGOOSE: A learnable LSH framework for efficient neural network training." International Conference on Learning Representations. 2020.