An ICML 2018 paper by Amirali Aghazadeh*, Ryan Spring*, Daniel LeJeune, Gautam Dasarathy, Anshumali Shrivastava, Richard G. Baraniuk
* These authors contributed equally and are listed alphabetically.
- Mission Logistic Regression
- Fine-Grained Mission Softmax Regression
- Coarse-Grained Mission Softmax Regression
- Feature Hashing Softmax Regression
- Mission streams in the dataset via Memory-Mapped I/O instead of loading everything directly into memory -
Necessary for Tera-Scale Datasets
- AVX SIMD optimization for fast Softmax Regression
- The code is currently optimized for the Splice-Site and DNA Metagenomics datasets.
Mission Softmax Regression
- Fine-Grained Feature Set - Each class maintains a separate feature set, so there is a top-k heap for each class.
- Coarse-Grained Feature Set - All the classes share a common set of features, so there is only one top-k heap. -
Each feature is measured by its L1 Norm for all classes.
- Data Parallelism - Each worker maintains a separate heap, while aggregating gradients in the same count-sketch.