Assertion issue: isfinite(phi_d) && isfinite(gphi)

[ASaragga]

I am getting the following assertion issue: AssertionError: isfinite(phi_d) && isfinite(gphi). When running the data in euribor_ok there is no assertion error. However, when running euribor second line, containing data which is identical to euribor_ok, one gets the assertion error.

On: Optim v0.18.1 and Julia 1.1

using Optim

euribor_ok = [0.073 0.201 0.255 0.323 0.408 0.512 0.629 0.752 0.873 0.986 1.173 1.381 1.579 1.668 1.717]
euribor = [0.077 0.207 0.261 0.33 0.413 0.511 0.621 0.737 0.853 0.961 1.144 1.344 1.538 1.627 1.675; 
0.073 0.201 0.255 0.323 0.408 0.512 0.629 0.752 0.873 0.986 1.173 1.381 1.579 1.668 1.717]

function nss(p) 
    t = [1.0 , 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 12.0, 15.0, 20.0, 25.0, 30.0]; 
    p[1] .+ 
    p[2] *  (1 .- exp.(-t/p[5])) ./ (t/p[5]) .+ 
    p[3] * ((1 .- exp.(-t/p[5])) ./ (t/p[5]) .- exp.(-t/p[5])) .+
    p[4] * ((1 .- exp.(-t/p[6])) ./ (t/p[6]) .- exp.(-t/p[6]))   
end
function lsquares(p,observed) 
    sum((nss(p) .- observed) .^2) 
end

p0   = [0.0, 0.0, 0.0, 0.0, 1.0, 1.0];

ice(p) = lsquares(p, euribor_ok);
resp = optimize(ice, p0, BFGS()) 
for i =1:2
    println(i)
    ice2(p) = lsquares(p, euribor[i,:]);
    resp = optimize(ice2, p0, BFGS()); 
end

Output

Results of Optimization Algorithm
 * Algorithm: BFGS
 * Starting Point: [0.0,0.0,0.0,0.0,1.0,1.0]
 * Minimizer: [0.8353333384184507,-1.1386778170395966e-7, ...]
 * Minimum: 6.568619e+01
 * Iterations: 10
 * Convergence: true
   * |x - x'| ≤ 0.0e+00: false 
     |x - x'| = 7.99e-07 
   * |f(x) - f(x')| ≤ 0.0e+00 |f(x)|: true
     |f(x) - f(x')| = 0.00e+00 |f(x)|
   * |g(x)| ≤ 1.0e-08: false 
     |g(x)| = 2.77e-07 
   * Stopped by an increasing objective: false
   * Reached Maximum Number of Iterations: false
 * Objective Calls: 39
 * Gradient Calls: 39

1
2
ERROR: AssertionError: isfinite(phi_d) && isfinite(gphi)
Stacktrace:
 [1] bisect!(::getfield(LineSearches, Symbol("#ϕdϕ#6")){Optim.ManifoldObjective{OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1}}},Array{Float64,1},Array{Float64,1},Array{Float64,1}}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1}, ::Int64, ::Int64, ::Float64, ::Int64) at /Users/anton/.julia/packages/LineSearches/WrsMD/src/hagerzhang.jl:504
 [2] (::LineSearches.HagerZhang{Float64,Base.RefValue{Bool}})(::Function, ::getfield(LineSearches, Symbol("#ϕdϕ#6")){Optim.ManifoldObjective{OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1}}},Array{Float64,1},Array{Float64,1},Array{Float64,1}}, ::Float64, ::Float64, ::Float64) at /Users/anton/.julia/packages/LineSearches/WrsMD/src/hagerzhang.jl:202
 [3] HagerZhang at /Users/anton/.julia/packages/LineSearches/WrsMD/src/hagerzhang.jl:101 [inlined]
...

[pkofod]

What you do is not identical.

julia> (nss(p0) .- euribor_ok)
15×15 Array{Float64,2}:
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717
 -0.073  -0.201  -0.255  -0.323  -0.408  -0.512  -0.629  -0.752  -0.873  -0.986  -1.173  -1.381  -1.579  -1.668  -1.717

julia> (nss(p0) .- euribor[2,:])
15-element Array{Float64,1}:
 -0.073
 -0.201
 -0.255
 -0.323
 -0.408
 -0.512
 -0.629
 -0.752
 -0.873
 -0.986
 -1.173
 -1.381
 -1.579
 -1.668
 -1.717

However, to solve your problem, try changing the line search algorithm

for i =1:2
    println(i)
    ice2(p) = lsquares(p, euribor[2,:]);
    resp = optimize(ice2, p0, BFGS(linesearch=LineSearches.BackTracking())); 
end

[pkofod]

did you try to compare against LsqFit btw?

[ASaragga]

Thank you for your reply and for this wonderful package! Yes BFGS(linesearch=LineSearches.BackTracking()), or LBFGS(), solves the problem.

I have a few thousand observations not just two, and noticed that for my data the default HagerZhang() line search will arise this assertion error in about 1.7% of the observations.

I was not aware of LsqFit. Thank you again for mention it.

[pkofod]

I have a few thousand observations not just two, and noticed that for my data the default HagerZhang() line search will arise this assertion error in about 1.7% of the observations.

Yeah, it's a known "issue", but the solution is really to use something else. The thing is, that HagerZhang is often a good choice, but backtracking is often "safer". However, it gets tricky to pick a default, because on some problems HagerZhang will fail and on others BackTracking will fail. So... At least there are some different ones to try :)