Question❓
Hi, I'm working on the BBOB benchmarking PR for Extending PSO benchmarking and noticed something I wanted to clarify.
The L-BFGS refinement phase in hybrid.jl currently uses(https://github.com/SciML/ParallelParticleSwarms.jl/blob/main/src/hybrid.jl#L28-L31):
∇f = instantiate_gradient(_f, prob.f.adtype)
nlprob = SimpleNonlinearSolve.ImmutableNonlinearProblem{false}(∇f, prob.u0, prob.p)
nlalg = SimpleLimitedMemoryBroyden(; threshold = ..., linesearch = Val(true))
This finds stationary points (∇f = 0) rather than directly minimizing f. On the paper's benchmarks(https://openreview.net/pdf?id=nD10o1ge97) (Rosenbrock, Lotka-Volterra, FitzHugh-Nagumo) this works well since stationary points in those landscapes are mostly minima. But on the BBOB suite, which has many saddle-heavy and multimodal functions, I'm seeing the refinement phase converge to saddle points instead of minima.
Was the NonlinearProblem approach a deliberate choice for GPU kernel compatibility, or would switching to actual minimization be the preferred direction?
Question❓
Hi, I'm working on the BBOB benchmarking PR for Extending PSO benchmarking and noticed something I wanted to clarify.
The L-BFGS refinement phase in hybrid.jl currently uses(https://github.com/SciML/ParallelParticleSwarms.jl/blob/main/src/hybrid.jl#L28-L31):
This finds stationary points (∇f = 0) rather than directly minimizing f. On the paper's benchmarks(https://openreview.net/pdf?id=nD10o1ge97) (Rosenbrock, Lotka-Volterra, FitzHugh-Nagumo) this works well since stationary points in those landscapes are mostly minima. But on the BBOB suite, which has many saddle-heavy and multimodal functions, I'm seeing the refinement phase converge to saddle points instead of minima.
Was the NonlinearProblem approach a deliberate choice for GPU kernel compatibility, or would switching to actual minimization be the preferred direction?