Slow evaluation of nonlinear callbacks

[odow]

This example from discourse has some bad scaling behavior (OP tried to run with k = 8000, n = 1200, not 800 and 12):

using Ipopt, JuMP, AmplNLWriter, Ipopt_jll

function nlp_model(optimizer)
    k = 800
    n = 12
    p = rand(400:700, k, 1)
    c1 = rand(100:200, k, n)
    c2 = 0.9 .* c1
    b = rand(150:250, k, 1)
    model = Model(optimizer)
    set_optimizer_attribute(model, "print_level", 3)
    @variable(model, 0 <= x[i = 1:n] <= 1)
    @variable(model, 0 <= var1 <= 1)
    @variable(model, 0 <= var2 <= 1)
    @variable(model, 0 <= var3 <= 1)
    @objective(model, Max, var1 - var2 + var3)
    @NLexpression(model, expr, sum(x[i] * p[i] for i in 1:n))
    @NLexpression(model, expr_c1[j = 1:k], sum(x[i] * c1[j, i] for i in 1:n))
    @NLexpression(model, expr_c2[j = 1:k], sum(x[i] * c2[j, i] for i in 1:n))
    @NLconstraint(model, expr - sum(b[j] / (1 + var1)^j for j in 1:k) == 0)
    @NLconstraint(model, expr - sum(expr_c1[j] / (1 + var2)^j for j in 1:k) == 0)
    @NLconstraint(model, expr - sum(expr_c2[j] / (1 + var3)^j for j in 1:k) == 0)
    @NLconstraint(model, [j=1:k], expr_c1[j] >= b[j])
    optimize!(model)
end

using ProfileView
@profview nlp_model(Ipopt.Optimizer)
@profview nlp_model(Ipopt.Optimizer)

Profiling

Profiling suggests this is the bottleneck

JuMP.jl/src/nlp.jl

Lines 1108 to 1117 in 37eac58

	nzthis = _hessian_slice(
	d,
	ex,
	x,
	H,
	lambda[i],
	nzcount,
	recovery_tmp_storage,
	Val{chunk},
	)::Int

Comparison with AmplNLWriter

The settings aren't the same between the two so not directly comparable, but AMPL is significantly faster at evaluating the callbacks:

julia> nlp_model(Ipopt.Optimizer)
Total number of variables............................:       15
                     variables with only lower bounds:        0
                variables with lower and upper bounds:       15
                     variables with only upper bounds:        0
Total number of equality constraints.................:        3
Total number of inequality constraints...............:      800
        inequality constraints with only lower bounds:      800
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0


Number of Iterations....: 58

                                   (scaled)                 (unscaled)
Objective...............:  -1.9284143389601621e-01    1.9284143389601621e-01
Dual infeasibility......:   2.9577363271173456e-11    2.9577363271173456e-11
Constraint violation....:   3.4445205830034141e-15    2.1921096049482003e-09
Complementarity.........:   2.5059073564210140e-09   -2.5059073564210140e-09
Overall NLP error.......:   2.5059073564210140e-09    2.1921096049482003e-09


Number of objective function evaluations             = 59
Number of objective gradient evaluations             = 59
Number of equality constraint evaluations            = 59
Number of inequality constraint evaluations          = 59
Number of equality constraint Jacobian evaluations   = 59
Number of inequality constraint Jacobian evaluations = 59
Number of Lagrangian Hessian evaluations             = 58
Total CPU secs in IPOPT (w/o function evaluations)   =      1.404
Total CPU secs in NLP function evaluations           =      1.956

EXIT: Optimal Solution Found.

julia> nlp_model(() -> AmplNLWriter.Optimizer(Ipopt_jll.amplexe))
Ipopt 3.13.4: print_level=3


******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit https://github.com/coin-or/Ipopt
******************************************************************************

Total number of variables............................:       15
                     variables with only lower bounds:        0
                variables with lower and upper bounds:       15
                     variables with only upper bounds:        0
Total number of equality constraints.................:        3
Total number of inequality constraints...............:      800
        inequality constraints with only lower bounds:      800
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0


Number of Iterations....: 98

                                   (scaled)                 (unscaled)
Objective...............:  -1.9284143507388135e-01   -1.9284143507388135e-01
Dual infeasibility......:   1.7809677976059946e-09    1.7809677976059946e-09
Constraint violation....:   1.1235589064727368e-12    3.2688835667946093e-07
Complementarity.........:   2.6976428374560677e-09    2.6976428374560677e-09
Overall NLP error.......:   2.6976428374560677e-09    3.2688835667946093e-07


Number of objective function evaluations             = 101
Number of objective gradient evaluations             = 99
Number of equality constraint evaluations            = 101
Number of inequality constraint evaluations          = 101
Number of equality constraint Jacobian evaluations   = 99
Number of inequality constraint Jacobian evaluations = 99
Number of Lagrangian Hessian evaluations             = 98
Total CPU secs in IPOPT (w/o function evaluations)   =      2.898
Total CPU secs in NLP function evaluations           =      0.531

EXIT: Optimal Solution Found.

[odow]

I took another look at this. There isn't anything obviously bad. It has a lot of expressions, and the model is quite unusual with the 1 / (1 + x)^j form where j is quite large. it's not obvious that 1 / (1 + x)^8000 is something that anyone would actually write in practice.