No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
HFD
nls
README.md

README.md

Second-Order Optimization for Non-Convex Machine Learning

This repository contains Matlab code that produces all the experimental results in the paper: Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study.

Specifically, multilayer perceptron(MLP) networks and non-linear least squares(NLS) are the two non-convex problems considered.

Usage

MLP networks

  • HFD: This folder contains all the source code for implementing the MLP problems that are considered in the paper.
  • HFD/algorithms contains the implementation of (sub-sampled) trust-region, gaussian-newton, momentum sgd algorithms.
  • HFD/mdoel constains the implementation of general neural network framework.
  • HFD/cifar_classification: a demo function that uses various algorithms to train a 1-hidden-layer network on cifar10.
  • HFD/mnist_autoencoder: a demo function that uses various algorithms to train autoencoder on mnist.

Example 1: Cifar10 Classification

Download Cifar10 dataset from here: https://www.cs.toronto.edu/~kriz/cifar-10-matlab.tar.gz

or run the command

bash download_cifar10.sh

In the Matlab Command Window, run

# check details of the function for different configurations
>> result = cifar_classifcation

Example 2: mnist Autoencoder

In the Matlab Command Window, run

# check details of the function for different configurations
>> result = mnist_autoencoder

NLS

  • nls: This folder contains all the source code for implementing the binary linear classification task using square loss (which gives a non-linear square problem).
  • nls/algorithms contains the implementation of (sub-sampled) TR, ARC, GN, GD, LBFGS algorithms for non-linear least squares.

Example 3: NLS on ijcnn1

Download 'ijcnn1' dataset from: https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html#ijcnn1

or run the command

bash download_ijcnn1.sh

In the Matlab Command Window, run

# this will generate the plots of all algorithms.
# check the details of the function for more options.
>> blc_demo('ijcnn1')

References