Accelerated Stochastic Power Iteration with Momemtum
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
plots
Best_Ball.ipynb
Eigensolvers.jl
Inhomogeneous.ipynb
Mini-batches-no-acceleration.ipynb
Mini-batches.ipynb
Readme.md
Stability.ipynb
Util.jl

Readme.md

Stochastic Power Iteration with Momentum

This repository contains the Julia code that produces all the experimental results in the paper Accelerated Stochastic Power Iteration.

Usage

The main code of the implementation for different PCA algorithms is in the file eigensolvers.jl.

  • minibatch_sgd_m(data, x, beta, iters, u, s, seed=1): This function is the implementation for our algorithm Mini-batch Power Method with Momentum. data is the data matrix where eacho row represents a single point, x is the initial point, beta is the momentum parameter, iters is the maximum number of iterations, u is the true eigenvector that is used to compute the error for each iterate, s is the mini-batch size and seed is the random seed.

  • minibatch_svrg_m(data, x, beta, epoch, m, u,s, seed=1): This function is the implementation for our algorithm Variance Reduced Power Method with Momemtum. The argument m is the epoch length and epoch is the number of epochs. The rest arguments are the same as minibatch_sgd_m().

All the experimental results are generated from the Jupyter notebooks:

  • Mini-batches.ipynb: This notebook shows the acceleration for momentum stochastic power iteration with mini-batching and variance reduction.

  • Stability.ipynb: This notebook shows the instability of Lanczos method for finding multiple eigenvalues.

  • Best_Ball.ipynb: This notebook shows the performance of Best Heavy Ball method that auto-tune the momentum parameter in the power iteration.

  • Inhomogeneous.ipynb: This notebook show the better performance of Inhomogeneous Polynomial Recurrence than the constant momentum power method.

Reference