exception_handling.py example gives incorrect output with Conda Forge's 3.13.3 Windows binary

[moorepants]

Text taken from #78:

This is the exception_handling.py output with Py 2.7, IPOPT 3.13.2 (official binary) on Windows:

python test/exception_handling.py
******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************
This is Ipopt version 3.13.2, running with linear solver pardiso.
Number of nonzeros in equality constraint Jacobian...:        4
Number of nonzeros in inequality constraint Jacobian.:        4
Number of nonzeros in Lagrangian Hessian.............:       10
Total number of variables............................:        4
                     variables with only lower bounds:        0
                variables with lower and upper bounds:        4
                     variables with only upper bounds:        0
Total number of equality constraints.................:        1
Total number of inequality constraints...............:        1
        inequality constraints with only lower bounds:        1
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.6109693e+01 1.12e+01 1.02e+00   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  1.7346108e+01 7.50e-01 2.37e+01  -0.3 7.22e-01    -  3.51e-01 1.00e+00f  1
   2  1.7922201e+01 7.70e-03 5.29e+00  -0.2 6.00e-02   2.0 9.96e-01 1.00e+00h  1
   3  0.0000000e+00 6.72e-02 1.66e+00  -0.8 1.37e+00    -  9.95e-01 6.98e-01f  1
Number of Iterations....: 3
                                   (scaled)                 (unscaled)
Objective...............:   0.0000000000000000e+00    0.0000000000000000e+00
Dual infeasibility......:   1.6551098674064002e+00    8.2755493370320010e-01
Constraint violation....:   6.7248402765482496e-02    6.7248402765482496e-02
Complementarity.........:   2.6927701887990801e-01    1.3463850943995401e-01
Overall NLP error.......:   1.6551098674064002e+00    8.2755493370320010e-01
Number of objective function evaluations             = 4
Number of objective gradient evaluations             = 4
Number of equality constraint evaluations            = 4
Number of inequality constraint evaluations          = 4
Number of equality constraint Jacobian evaluations   = 4
Number of inequality constraint Jacobian evaluations = 4
Number of Lagrangian Hessian evaluations             = 3
Total CPU secs in IPOPT (w/o function evaluations)   =      0.102
Total CPU secs in NLP function evaluations           =      0.003
EXIT: Stopping optimization at current point as requested by user.
Objective value at iteration #0 is - 16.1097
Objective value at iteration #1 is - 17.3461
Objective value at iteration #2 is - 17.9222
Caught ValueError: bum

And Python 3.8, IPOPT 3.13.3 (via conda forge), on Windows:

python test/exception_handling.py
******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit https://github.com/coin-or/Ipopt
******************************************************************************
This is Ipopt version 3.13.3, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).
Number of nonzeros in equality constraint Jacobian...:        4
Number of nonzeros in inequality constraint Jacobian.:        4
Number of nonzeros in Lagrangian Hessian.............:       10
Total number of variables............................:        4
                     variables with only lower bounds:        0
                variables with lower and upper bounds:        4
                     variables with only upper bounds:        0
Total number of equality constraints.................:        1
Total number of inequality constraints...............:        1
        inequality constraints with only lower bounds:        1
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.6109693e+01 1.12e+01 1.02e+00   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  1.7346108e+01 7.50e-01 2.37e+01  -0.3 7.22e-01    -  3.51e-01 1.00e+00f  1
   2  1.7922201e+01 7.70e-03 5.29e+00  -0.2 6.00e-02   2.0 9.96e-01 1.00e+00h  1
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Evaluation error during soft restoration phase step.
Warning: Evaluation error during soft restoration phase step.
Warning: Evaluation error during soft restoration phase step.
Sorry, things failed ?!?!
Number of Iterations....: 2
                                   (scaled)                 (unscaled)
Exception of type: Eval_Error in file "D:\bld\ipopt_1605616866017\work\Ipopt\src\Algorithm\IpOrigIpoptNLP.cpp" at line 487:
 Exception message: success && IsFiniteNumber(ret) evaluated false: Error evaluating the objective function
Number of objective function evaluations             = 45
Number of objective gradient evaluations             = 3
Number of equality constraint evaluations            = 41
Number of inequality constraint evaluations          = 41
Number of equality constraint Jacobian evaluations   = 4
Number of inequality constraint Jacobian evaluations = 4
Number of Lagrangian Hessian evaluations             = 3
Total CPU secs in IPOPT (w/o function evaluations)   =      0.016
Total CPU secs in NLP function evaluations           =      0.003
EXIT: Invalid number in NLP function or derivative detected.
Objective value at iteration #0 is - 16.1097
Objective value at iteration #1 is - 17.3461
Objective value at iteration #2 is - 17.9222
Caught ValueError: bum

Maybe there is a difference in IPOPT 3.13.2 and 3.13.3 that prevents this exception to be properly caught?

So 3.13.2 give the same results on Py 27 and Py 38 on Windows. Maybe this due to a change introduced in 3.13.3.

The file IpOrigIpoptNLP.cpp was edited between 3.13.2 and 3.13.3. See:

coin-or/Ipopt@releases/3.13.2...master#diff-9a3e51142bc8474657621b48b0490655b611422d8df80ffb35db280763789a0a

But the only changes are extra forwardslashes. Not sure what that does...

I tried the official Ipopt 3.13.3 binaries with Python 3.* on Windows and that works as expected. The oddity is only from the conda forge builds. So something is likely different in the conda forge builds. I think we could merge this as is and open an issue for this exception handling issue.

[traversaro]

It may be worth to try again once conda-forge/ipopt-feedstock#58 is merged. The bug conda-forge/ipopt-feedstock#57 could create quite subtle wrong behaviors.

[nrontsis]

In the output of your examples I see that there is a difference in the linear solver used (pardiso vs mumps). Could this be the source of the problem?

[traversaro]

Reading a bit more, I see that the error was:

Exception message: success && IsFiniteNumber(ret) evaluated false: Error evaluating the objective function

Given that, I think that probably this was due to conda-forge/ipopt-feedstock#57, that was indeed related to IsFiniteNumber not working correctly.

[moorepants]

In the output of your examples I see that there is a difference in the linear solver used (pardiso vs mumps). Could this be the source of the problem?

Interesting. I'm not sure why pardiso would be loaded. I didn't think we were installing that.