Skip to content
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

stochs - fast stochastic solvers for machine learning


The stochs library provides efficient C++ implementations of stochastic optimization algorithms for common machine learning settings, including situations with finite datasets augmented with random perturbations (e.g. data augmentation or dropout). The library is mainly used from Python through a Cython extension. Currently, SGD, (S-)MISO and (N-)SAGA are supported, for dense and sparse data. See the following reference for details:

A. Bietti and J. Mairal. Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite-Sum Structure. NIPS, 2017.


The library requires Eigen >=3.3 (it will be downloaded automatically in the script unless the folder or symlink include/Eigen already exists) and glog. To install glog on Ubuntu, run:

sudo apt-get install libgoogle-glog-dev

The Python package can be built with the following command (this requires Cython and a compiler with OpenMP support such as gcc, you might need to change the CC and CXX environment variables on a mac):

python3 build_ext -if

By default, the library is built for double precision floating point numbers (np.float64), for single precision (np.float32) set USE_FLOAT = 1 in


Example usage with dropout perturbations:

import numpy as np
import stochs

X, y, Xtest, ytest = load_some_dataset()

solver = stochs.MISO(X.shape[1],  # number of features
                     X.shape[0],  # number of datapoints
                     alpha=1.0,   # initial step-size
                     lmbda=0.01,  # L2 regularization
                     loss=b'squared_hinge', # squared hinge loss
                     prox=b'l1',  # use L1 regularizer (by default 'none')
                     prox_weight=0.1, # L1 regularization weight
                     average=False) # no iterate averaging

n = X.shape[0]
for epoch in range(100):
    if epoch == 2:
        # start decaying the step-size after a few epochs
        # if average=True, this also starts iterate averaging

    # pick random indexes for one epoch
    idxs = np.random.choice(n, n)

    # apply 10% dropout
    Xperturbed = X[idxs] * np.random.binomial(1, 0.9, size=X.shape) / 0.9

    # run algorithm on batch of perturbed data
    solver.iterate(Xperturbed, y[idxs], idxs)
    # with no perturbations, use: solver.iterate_indexed(X, y, idxs)

    print(solver.compute_loss(Xtest, ytest))  # compute test loss
# access parameter vector with solver.w()

A more thorough example for sentiment analysis on the IMDB dataset is given in the examples folder, with sparse solvers and non-uniform sampling.


stochs: fast stochastic solvers for machine learning in C++ and Cython




No releases published
You can’t perform that action at this time.