Pros and cons of using other interfaces inside julia_interface.jl

[abelsiqueira]

We need to evaluate the pros and cons of using the core or specialized interfaces inside julia_interface.jl.

From #40:

The call chain would be

• objcons(nlp, x) in the Julia interface
• ufn(nlp, x) in the specialized interface
• ufn(io_err, n, x, f, libname) in the core interface
• cutest_ufn() in libcutest.

Right now, objcons calls cutest_ufn directly. But the intermediate functions are just passing scalars
and pointers around, there's no copy of arrays.

[dpo]

Perhaps we could just run some benchmarks on a few large problems to determine what the penalty would be.

[dpo]

I'm wondering if the core interface should be exported at all. Most users will use the julia interface (I think) and if you don't want to work with NLPModels, the specialized interface is as friendly as it could be. What do you think?

As a side comment, the non! methods in the specialized interface should call the ! methods. That would shorten the code and ensure better coverage.

[abelsiqueira]

I think not exporting it is best.

Also, the specialized interface has a reasonable amount of functions defined.

• Two using nlp (one inplace and one not)
• Two not using nlp
• Sometimes duplicates using Int32 for arrays.

Maybe we should remove some? I feel using nlp should be enough for most, and I'm not sure about Int32 arrays.

This might take some time.

[dpo]

Ok, I agree with not exporting the core interface.

Do you think a user is likely to use the specialized interface with nlp? If they use an nlp, won't they just use the julia interface?

This is by no means urgent.

[abelsiqueira]

There might be (hopefully insignficant) speed differences between julia and specialized, but I think the main reason to choose one over the other are the specific functions CUTEst provides, like ccifsg.
It that's the case, people might still use nlp, but use the specialized interface.
Furthermore, I don't think they will want to manually call sifdecoder and csetup, so they'll probably create a nlp anyway.

[dpo]

We should benchmark the different interfaces but I agree that the difference is probably not significant. I'm happy to expose the Julia and specialized interfaces.

[abelsiqueira]

I made some benchmarks on the NLPModels interface vs the specialized interface (here).
I only used two problems: ROSENBR and DIXMAANJ with M=30, and tested only objective and gradient with a steepest descent method.
The NLPModels version seems to take about 10% more time and memory than the specialized interface.

Do you think we should invest more time on this? The NLPModels interface is already using ccalls.

[dpo]

Did you by any chance plot a flame graph to try and see where the difference comes from? Would it help if the Julia interface called the specialized interface?

[abelsiqueira]

Okay, so, investigating did help.
obj is actually objgrad which calls uofg, while I was testing against ufn.
So I tested objgrad vs objcons and uofg vs ufn, with false for gradient, and found out that these can have different times. objcons and ufn are a little bit faster than objgrad and uofg (~10%) in ROSENBR, and similar for DIXMAANJ.

For both problems, ufn and objcons are comparable, but uofg and objgrad are not, because in uofg the gradient is allocated regardless of the boolean argument. I'll fix this.

What's strange though, is the difference in objgrad and objcons. So I did a core interface call for uofg and ufn for ROSENBR, and there is still about 10% difference in time. I allocated everything beforehand, so it's only the call for the core interface which implies only a ccall. I did the same for DIXMAANJ (nvar=3000) and OSCIGRAD (nvar=100000) and the calls have similar time, so it's a ROSENBR problem, possibly.

Did you by any chance plot a flame graph to try and see where the difference comes from? Would it help if the Julia interface called the specialized interface?

I didn't find anything very useful with the flame graph. Creating array-like variables (io_err, f, c and g) take some time and most time is in the ccall, namely dlsym.
Using the specialized in the Julia interface would be cleaner, but probably not faster.

[dpo]

I suppose the problem is that objcons() and objgrad() (and therefore obj()) are not type stable. We should probably recode obj() so it is type stable.

[abelsiqueira]

It's possible that there is a slow down because of type-stability, I'll have to check, but for larger problems, the times of equivalent functions is equivalent. Try the following after the fix on uofg:

using BenchmarkTools
using CUTEst

function foo()
  nlp = CUTEstModel("DIXMAANJ")
  try
    x = nlp.meta.x0
    t = @benchmark objcons($nlp, $x)
    println(t)
    io_err = Cint[0]
    n = Cint[nlp.meta.nvar]
    f = Cdouble[0]
    g = Cdouble[]
    grad = Cint[0]
    t = @benchmark uofg($io_err, $n, $x, $f, $g, $grad)
    println(t)
    n = nlp.meta.nvar
    t = @benchmark uofg($n, $x, false)
    println(t)
  finally
    finalize(nlp)
  end
end

foo()

However, for some problems uofg and ufn from core don't have equivalent times, at least for Rosenbrock-like (?) problems:

using BenchmarkTools
using CUTEst

function foo()
  nlp = CUTEstModel("GENROSE", "param", "N=10000")
  try
    io_err = Cint[0]
    n = Cint[nlp.meta.nvar]
    x = nlp.meta.x0
    f = Cdouble[0]
    g = Cdouble[]
    grad = Cint[0]
    t = @benchmark ufn($io_err, $n, $x, $f)
    println(t)
    t = @benchmark uofg($io_err, $n, $x, $f, $g, $grad)
    println(t)
  finally
    finalize(nlp)
  end
end

foo()

[dpo]

I just noticed that the in-place functions in the specialized interface don't return anything. I believe that's an error, e.g., push!() returns the argument that it modifies. I think we should have something like

function usetup(input::Int, out::Int, io_buffer::Int, n::Int)
  x = Array{Cdouble}(n)
  x_l = Array{Cdouble}(n)
  x_u = Array{Cdouble}(n)
  usetup!(input, out, io_buffer, n, x, x_l, x_u)
end

function usetup!(input::Int, out::Int, io_buffer:Int, n::Int, x::Vector{Float64}, x_l::Vector{Float64}, x_u::Vector{Float64})
  io_err = Cint[0]
  usetup(io_err, Cint[input], Cint[out], Cint[io_buffer], Cint[n], x, x_l, x_u)
  @cutest_error
  return x, x_l, x_u
end

instead of what we have now. Also, usetup!() should not require n since it's the common length of x, x_l and x_u.

Better yet would be to phase out either the specialized or core interface. It's too much code to maintain and it's essentially duplicated.

[abelsiqueira]

I agree on phasing out the specialized interface. The core interface is just a wrapper for ccalls using the internal cutest_lib, so it's good to have around.

[dpo]

Can this be closed now?

[abelsiqueira]

Yes, thank you.