Skip to content
Global optimization without derivatives
Julia
Branch: master
Clone or download
Latest commit b42cdaa Mar 25, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
demo Update to julia-0.7 Oct 10, 2018
images Update README and fix image orientation Jan 21, 2018
src Fix another eps error Mar 24, 2019
test Update to julia-0.7 Oct 10, 2018
.codecov.yml QuadDIRECT.jl generated files. Jan 16, 2018
.gitignore Update iteration to 1.0 Dec 17, 2018
.travis.yml Update to julia-0.7 Oct 10, 2018
LICENSE.md QuadDIRECT.jl generated files. Jan 16, 2018
Project.toml
README.md Update the README and docstrings Feb 26, 2018
appveyor.yml Update to julia-0.7 Oct 10, 2018

README.md

QuadDIRECT

Build Status

codecov.io

QuadDIRECT is an algorithm for global optimization without requiring derivatives. It was inspired by DIRECT and MCS:

DIRECT: Jones, Donald R., Cary D. Perttunen, and Bruce E. Stuckman. "Lipschitzian optimization without the Lipschitz constant." Journal of Optimization Theory and Applications 79.1 (1993): 157-181.

MCS: Huyer, Waltraud, and Arnold Neumaier. "Global optimization by multilevel coordinate search." Journal of Global Optimization 14.4 (1999): 331-355.

There is no formal published description (yet), but it expands upon DIRECT by:

  • allowing boxes to be of different sizes, and supporting boxes that extend to infinity
  • not assuming that the coordinates form a metric space: each dimension is treated independently of the others
  • it splits boxes at points suspected of being minima as judged by a local one-dimensional quadratic model of the function
  • once enough boxes have been created, it attempts "local search," building a dense quadratic model and performing a quasi-Newton optimization.

Unlike MCS,

  • the geometry is more similar to DIRECT, in that each box is affiliated with a different function evaluation point
  • it uses the "Pareto front" concept of DIRECT, rather than a heuristic splitting scheme based on levels
  • the local search is not disconnected from the box-splitting: the function evaluations used in local search get entered into the tree structure, providing the opportunity to re-use those function evaluations for further improvements

As a simple demonstration, consider the "6-hump camel function":

julia> function camel(x)
            # 6-hump camel function. Typically evaluated over [-3,3] × [-2,2].
            x1, x2 = x[1], x[2]
            x1s = x1*x1
            x2s = x2*x2
            return (4 - 2.1*x1s + x1s*x1s/3)*x1s + x1*x2 + (-4 + 4*x2s)*x2s
        end

camel function

Here the value scale was cut off at 5 so that the structure of the minima can be seen. You can explore this function with

julia> using QuadDIRECT

julia> lower, upper = [-3,-2], [3,2]   # domain over which we allow solutions
([-3, -2], [3, 2])

julia> splits = ([-2, 0, 2], [-1, 0, 1])   # initial locations to evaluate function
([-2, 0, 2], [-1, 0, 1])

julia> root, x0 = analyze(camel, splits, lower, upper)
(BoxRoot@[NaN, NaN], [0.0, 0.0])

This creates a tree structure (currently) with a few hundred boxes, each corresponding to a single function evaluation:

boxes

You can see that it has concentrated its function evaluations near the local minima, and that all of the minima were explored. This plot was generated by utilities in QuadDIRECT/src/plotting.jl, which have to be loaded separatedly from the QuadDIRECT module.

You can inspect the minimum like this:

julia> box = minimum(root)
Box-1.0316284406055976@[0.0898781, -0.71269]

julia> value(box)
-1.0316284406055976

julia> position(box, x0)
2-element Array{Float64,1}:
0.0898781
-0.71269

These match the known global optima to reasonably high precision.

There's also a convenience function minimize which just returns the location and value of the minimum.

For more examples, see the demo directory.

Usage guidance, benchmarking, and convergence

In global optimization, "convergence" is a tricky subject: if the function might be non-convex, there is no principled way to say you're done---that your best minimum so far is the best minimum there is---without some additional information about the function. QuadDIRECT terminates if no further improvements have been made after ndims iterations. If you're benchmarking QuadDIRECT, here are a few tips:

  • if you know the value of the global minimum, you can use it to specify a termination criterion, e.g., within 1% relative or 0.01 absolute value of the global minimum. See the fvalue option for analyze.
  • You can count function evaluations by wrapping your function with fc = CountedFunction(f) and then use numevals(fc) to see how many evaluations were used. This is slightly more accurate than length(leaves(root)) because there are a few cases where the function value is not stored in the tree structure.
  • Use LoggedFunction to keep a record of every function value computed. The latter is particularly useful if you want to ask how many evaluations were required to reach a value of particular quality.
You can’t perform that action at this time.