Skip to content
Proximal Newton OPTimizer
Matlab M
Find file
Latest commit 2dbd034 Mar 12, 2015 @yuekai edited README
Failed to load latest commit information.
private first commit Jan 18, 2014
README.md edited README Mar 12, 2015
pnopt.m first commit Jan 18, 2014
pnopt_optimset.m first commit Jan 18, 2014
pnopt_pn.m first commit Jan 18, 2014
pnopt_pqn.m first commit Jan 18, 2014
pnopt_sparsa.m first commit Jan 18, 2014

README.md

PNOPT: Proximal Newton OPTimizer

PNOPT (pronounced pee-en-opt) is a MATLAB package that uses proximal Newton-type methods to minimize composite functions. For details, please refer to Lee et al. (2014).

Installation

Unpack the archive and add the yuekai-PNOPT-xxxxxxx directory to your MATLAB path, e.g.

addpath /home/yuekai/matlab/yuekai-PNOPT-xxxxxxx/

We suggest users also install TFOCS (pronounced tee-fox), a MATLAB package that uses first-order methods to minimize composite functions (among other things).

Usage

PNOPT has the calling sequence:

[ x, f, output ] = pnopt( smoothF, nonsmoothF, x0, options );

The required input arguments are:

  • smoothF: a smooth function,
  • nonsmoothF: a nonsmooth function,
  • x0: a starting point for the solver.

The user can also supply an options structure created using pnopt_optimset to customize the behavior of PNOPT. pnopt_optimset shares a similar interface with MATLAB's optimset function:

options = pnopt_optimset( 'param1', val1, 'param2', val2, ... );

Calling pnopt_optimset with no inputs and outputs prints available options.

PNOPT returns:

  • x: an optimal solution,
  • f: the optimal value,
  • output: a structure containing information collected during the execution of PNOPT.

Creating smooth and nonsmooth functions

Smooth and nonsmooth functions must satisfy these conventions:

  • smoothF(x) should return the function value and gradient at x, i.e. [ fx, gradx ] = smoothF(x),
  • nonsmoothF(x) should return function value at x, i.e. f_x = nonsmoothF(x),
  • nonsmoothF(x,t) should return the proximal point y and the function value at y, i.e. [ f_y, y ] = nonsmoothF(x,t ).

PNOPT is compatible with the function generators included with TFOCS that accept vector arguments so users can use these generators to create commonly used smooth and nonsmooth functions. Please refer to section 3 of the TFOCS user guide for details.

Demo: sparse logistic regression

The demo requires LogisticLoss from PMTK and prox_l1 from TFOCS.

n = 100;
p = 200;
X = randn(n,p);
y = sign( X * ( (rand(p,1) > .5) .* randn(p,1) ) + randn(n,1) );

logistic_obj = @(w) LogisticLoss(w,X,y);
lambda = 10;
l1_pen  = prox_l1(lambda);
w0 = zeros(p,1);
pnopt_options = pnopt_optimset( 'optim', 1e-8 );

[ w, f ] = pnopt( logistic_obj, l1_pen, w0, pnopt_options );

================================================================
                 PNOPT v. 0.9.1 (Dec. 15, 2013)
================================================================
         Fun.    Prox     Step len.     Obj. val.        Optim.
----------------------------------------------------------------
   0 |      1       0                  6.9315e+01    4.4998e+00
  10 |     11      49    1.0000e+00    6.7911e+01    1.8744e-03
  13 |     14      66    1.0000e+00    6.7911e+01    2.0037e-04
----------------------------------------------------------------
Something went wrong with that request. Please try again.