Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Basic Optimization Suite #81

Open
3 of 9 tasks
cannontwo opened this issue Jan 5, 2021 · 6 comments
Open
3 of 9 tasks

Implement Basic Optimization Suite #81

cannontwo opened this issue Jan 5, 2021 · 6 comments
Assignees
@cannontwo
Labels
enhancement New feature or request ml

Comments

@cannontwo
Copy link
Owner

cannontwo commented Jan 5, 2021 •
edited
Loading

  • SGD
  • Brent's Algorithm in One Dimension
  • Generating Set Search (compass search/pattern search)
  • CMA-ES
  • Conjugate Gradient Descent
  • BFGS
  • Linear Programming
  • Quadratic Programming
  • SQP
@cannontwo cannontwo added enhancement New feature or request ml labels Jan 5, 2021
@cannontwo cannontwo self-assigned this Jan 5, 2021
@cannontwo
Copy link
Owner Author

May also want to define some canonical tasks in order to test/benchmark.

@cannontwo
Copy link
Owner Author

cannontwo commented Jan 12, 2021 •
edited
Loading

Nealder-Mead is empirically quick but may fail to find a stationary point on some problems. See here---specifically Algorithm 3.2---for information on Generating Set Search (compass search), which I'm implementing for derivative-free deterministic optimization instead. My implementation should use a minimal generating set (n+1 directions, defining a regular simplex in n dimensions) or coordinate search directions (2n sample directions). To accelerate search, periodically rotate the search directions by orienting a primary vector or a la Rosenbrock search (but use Palmer's innovation instead of Gram-Schmidt) respectively.

This was referenced Jan 12, 2021
Closed
Closed
cannontwo added a commit that referenced this issue Jan 23, 2021
@cannontwo
Implemented Wolfe Conditions line search and BFGS optimizer. Partiall…
627fc82 
…y addresses #81
@cannontwo
Copy link
Owner Author

For the SQP solver, implement algorithm 18.3 from Nocedal & Wright. For the QP solver, implement algorithm 16.4, solving the linear system using the projected CG algorithm 16.2.

cannontwo added a commit that referenced this issue Jan 24, 2021
@cannontwo
Implemented pattern search and tried applying it to lagrangian mechan…
ac7b1c1 
…ics. Still need to replace linear interpolation path representation with Lagrange interpolation. Partially addresses #81
@cannontwo
Copy link
Owner Author

For Linear Programming, implement interior point method in Ch. 14 of Nocedal and Wright

cannontwo added a commit that referenced this issue Jan 30, 2021
@cannontwo
Played around with implementing the simplex method, but after spendin…
c16d579 
…g some time I think that wrapping Gurobi is a better way to handle LP and QP solving. Added Gurobi integration and a test script. Relevant to #81
@cannontwo cannontwo mentioned this issue Jan 30, 2021
Closed
@cannontwo
Copy link
Owner Author

For Linear Programming, implement interior point method in Ch. 14 of Nocedal and Wright

Tried this, but implementation is too complex to be worthwhile right now. For now, just using Gurobi.

@cannontwo
Copy link
Owner Author

Linear Programming implemented as wrapper around Gurobi in 7626ce8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cannontwo cannontwo

Labels
enhancement New feature or request ml
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cannontwo