Skip to content

smrfeld/l_bfgs_tutorial

main
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Simple implementation of L-BFGS (low-memory)

Following the description on Wikipedia, reproduced here for completeness:

drawing

Results

We try it on the six-hump camelback function:

drawing

Some sample trajectories during optimization, starting from uniformly sampled points:

drawing

The endpoint distribution is (counts indicate number of trajectories that end at that location from 100 trajectories):

drawing

drawing

Most points have converged to one of the local minima, but others are stuck at saddle points, in particular at (0,0).

Noisy gradients to move away from local maxima

We can add some Gaussian noise to the gradients (as is common in machine learning due to the use of a small batch size) to move away from the local maxima:

drawing

drawing

About

L-BFGS tutorial in Python

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages