Global optimization without derivatives

Switch branches/tags
Nothing to show

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

## Files

Failed to load latest commit information.
Type
Name
Commit time

This package is registered in HolyLabRegistry---you'll likely want to add that as an additional registry.

QuadDIRECT is an algorithm for global optimization without requiring derivatives. It was inspired by DIRECT and MCS:

DIRECT: Jones, Donald R., Cary D. Perttunen, and Bruce E. Stuckman. "Lipschitzian optimization without the Lipschitz constant." Journal of Optimization Theory and Applications 79.1 (1993): 157-181.

MCS: Huyer, Waltraud, and Arnold Neumaier. "Global optimization by multilevel coordinate search." Journal of Global Optimization 14.4 (1999): 331-355.

There is no formal published description (yet), but it expands upon DIRECT by:

• allowing boxes to be of different sizes, and supporting boxes that extend to infinity
• not assuming that the coordinates form a metric space: each dimension is treated independently of the others
• it splits boxes at points suspected of being minima as judged by a local one-dimensional quadratic model of the function
• once enough boxes have been created, it attempts "local search," building a dense quadratic model and performing a quasi-Newton optimization.

Unlike MCS,

• the geometry is more similar to DIRECT, in that each box is affiliated with a different function evaluation point
• it uses the "Pareto front" concept of DIRECT, rather than a heuristic splitting scheme based on levels
• the local search is not disconnected from the box-splitting: the function evaluations used in local search get entered into the tree structure, providing the opportunity to re-use those function evaluations for further improvements

As a simple demonstration, consider the "6-hump camel function":

```julia> function camel(x)
# 6-hump camel function. Typically evaluated over [-3,3] × [-2,2].
x1, x2 = x[1], x[2]
x1s = x1*x1
x2s = x2*x2
return (4 - 2.1*x1s + x1s*x1s/3)*x1s + x1*x2 + (-4 + 4*x2s)*x2s
end```

Here the value scale was cut off at 5 so that the structure of the minima can be seen. You can explore this function with

```julia> using QuadDIRECT

julia> lower, upper = [-3,-2], [3,2]   # domain over which we allow solutions
([-3, -2], [3, 2])

julia> splits = ([-2, 0, 2], [-1, 0, 1])   # initial locations to evaluate function
([-2, 0, 2], [-1, 0, 1])

julia> root, x0 = analyze(camel, splits, lower, upper)
(BoxRoot@[NaN, NaN], [0.0, 0.0])```

This creates a tree structure (currently) with a few hundred boxes, each corresponding to a single function evaluation:

You can see that it has concentrated its function evaluations near the local minima, and that all of the minima were explored. This plot was generated by utilities in `QuadDIRECT/src/plotting.jl`, which have to be loaded separatedly from the `QuadDIRECT` module.

You can inspect the minimum like this:

```julia> box = minimum(root)
Box-1.0316284406055976@[0.0898781, -0.71269]

julia> value(box)
-1.0316284406055976

julia> position(box, x0)
2-element Array{Float64,1}:
0.0898781
-0.71269```

These match the known global optima to reasonably high precision.

There's also a convenience function `minimize` which just returns the location and value of the minimum.

For more examples, see the `demo` directory.

### Usage guidance, benchmarking, and convergence

In global optimization, "convergence" is a tricky subject: if the function might be non-convex, there is no principled way to say you're done---that your best minimum so far is the best minimum there is---without some additional information about the function. QuadDIRECT terminates if no further improvements have been made after `ndims` iterations. If you're benchmarking QuadDIRECT, here are a few tips:

• if you know the value of the global minimum, you can use it to specify a termination criterion, e.g., within 1% relative or 0.01 absolute value of the global minimum. See the `fvalue` option for `analyze`.
• You can count function evaluations by wrapping your function with `fc = CountedFunction(f)` and then use `numevals(fc)` to see how many evaluations were used. This is slightly more accurate than `length(leaves(root))` because there are a few cases where the function value is not stored in the tree structure.
• Use `LoggedFunction` to keep a record of every function value computed. The latter is particularly useful if you want to ask how many evaluations were required to reach a value of particular quality.

Global optimization without derivatives

3 tags

## Packages 0

No packages published