Fix logic in cost_of_bridging

[odow]

@blegat should this be a + or a -? Part of #1698

With the -:

julia> solver = MOI.instantiate(
           MOI.OptimizerWithAttributes(
               MOIPajarito.Optimizer,
               "oa_solver" => MOI.OptimizerWithAttributes(GLPK.Optimizer),
               "conic_solver" => MOI.OptimizerWithAttributes(ECOS.Optimizer),
           );
           with_bridge_type = Float64
       );

julia> MOI.Utilities._cost_of_bridging(solver, MOI.GreaterThan{Float64})
(0.0, false)

julia> MOI.Utilities._cost_of_bridging(solver, MOI.Integer)
(0.0, false)

The cost for GreaterThan is 1 - 1 but the cost for Integer is 0 - 0. Or perhaps we need to return a triplet of (x - y, x + y, false).

[blegat]

It should be -, it's checking what will be the difference of cost if they are added as constrained-on-creation vs added free and then constrained so the - takes this difference.

[odow]

Okay, but we should probably prefer sets that don't require bridging over those that do? So we should distinguish 1 - 1 from 0 - 0?

[odow]

Because for Pajarito the cost of bridging GreaterThan is greater than Integer.

[odow]

@blegat I'm not convinced of the current ordering of the sets. For example, the comment says "give priority to vector sets" but the ordering actually does the opposite (they come last).

One issue with the costs is that they don't take into account modified costs if we choose one of the variable bridges. i.e., the act of bridging one set can invalidate the other bridges that we would have chosen. In this case, if we bridge on GreaterThan first, then the cost of Integer is no longer 0.

I think we need to construct some test cases first, and then modify the behavior to match the test-cases.

struct Optimizer1698 <: MOI.AbstractOptimizer end

function MOI.supports_constraint(
    ::Optimizer1698,
    ::Type{MOI.VariableIndex},
    ::Type{MOI.Integer},
)
    return true
end

function MOI.supports_constraint(
    ::Optimizer1698,
    ::Type{<:Union{MOI.VectorOfVariables,MOI.VectorAffineFunction{Float64}}},
    ::Type{<:Union{MOI.Nonnegatives,MOI.SecondOrderCone}},
)
    return true
end

function test_sorted_variable_sets_by_cost()
    src = MOI.Utilities.Model{Float64}()
    x = MOI.add_variable(src)
    y = MOI.add_variables(src, 2)
    MOI.add_constraint(src, x, MOI.GreaterThan(1.0))
    MOI.add_constraint(src, x, MOI.Integer())
    MOI.add_constraint(src, y, MOI.SecondOrderCone(2))
    dest = MOI.Bridges.full_bridge_optimizer(Optimizer1698(), Float64)
    @test MOI.Utilities.sorted_variable_sets_by_cost(dest, src) == [
        MOI.Integer,
        MOI.GreaterThan{Float64},
        MOI.SecondOrderCone,
    ]
    return
end

What about this one?

struct Optimizer1698_2 <: MOI.AbstractOptimizer end

function MOI.supports_constraint(
    ::Optimizer1698_2,
    ::Type{<:Union{MOI.VectorOfVariables,MOI.VectorAffineFunction{Float64}}},
    ::Type{<:Union{MOI.Nonnegatives,MOI.Nonpositives}},
)
    return true
end

function test_sorted_variable_sets_by_cost()
    src = MOI.Utilities.Model{Float64}()
    MOI.add_constrained_variables(src, MOI.Nonnegatives(2))
    MOI.add_constrained_variables(src, MOI.Zeros(2))
    dest = MOI.Bridges.full_bridge_optimizer(Optimizer1698_2(), Float64)
    @test MOI.Utilities.sorted_variable_sets_by_cost(dest, src) == [
        MOI.Nonnegatives,
        MOI.Zeros,
    ]
    return
end

I think it makes sense for Nonnegatives to come first. But at the moment Zeros does.

[odow]

@chriscoey you can try MOIPajarito with this branch.

julia> using JuMP, GLPK, ECOS, MOIPajarito

julia> solver = MOI.OptimizerWithAttributes(
           MOIPajarito.Optimizer,
           "oa_solver" => MOI.OptimizerWithAttributes(GLPK.Optimizer),
           "conic_solver" => MOI.OptimizerWithAttributes(ECOS.Optimizer),
       )
MathOptInterface.OptimizerWithAttributes(MOIPajarito.Optimizer, Pair{MathOptInterface.AbstractOptimizerAttribute, Any}[MathOptInterface.RawOptimizerAttribute("oa_solver") => MathOptInterface.OptimizerWithAttributes(GLPK.Optimizer, Pair{MathOptInterface.AbstractOptimizerAttribute, Any}[]), MathOptInterface.RawOptimizerAttribute("conic_solver") => MathOptInterface.OptimizerWithAttributes(ECOS.Optimizer, Pair{MathOptInterface.AbstractOptimizerAttribute, Any}[])])

julia> model = Model(solver)
A JuMP Model
Feasibility problem with:
Variables: 0
Model mode: AUTOMATIC
CachingOptimizer state: EMPTY_OPTIMIZER
Solver name: Pajarito

julia> @variable(model, z >= 1, Int)
z

julia> optimize!(model)
no conic constraints need outer approximation
OA solver finished with status OPTIMAL, after 0.13000798225402832 seconds and 0 cuts
one tree method used 0 lazy callbacks and 0 heuristic callbacks

[blegat]

Indeed, there is something I missed when writing this cost. For Integer, it is natively supported both if it's constrained on creation or not so it seems it will never need any bridging but it can also be forced-bridged into ScalarAffineFunction if the variable is constrained on creation.
It seems to me that we should first give priority to the variables such that constraining on creation is natively supported to try to have no variable bridge.
So the order should be

1. Bool equal to supports_add_constrained_variable(s) giving priority to true
2. Difference of bridge cost
3. Bool indicating whether it is vector, giving priority to true

For example, the comment says "give priority to vector sets" but the ordering actually does the opposite (they come last).

Indeed, I'm surprised, I thought it was tested. This should be fixed.

[odow]

That order makes sense.

[odow]

How's this?

[blegat]

I'd like something along these lines to be either in the docstring or a comment in the code as I feel I will forget about that subtlety but I can add in a separate PR:

We favor adding first variables that won't use variables bridges because then the variable constraints on the same variable can still be added as VariableIndex or VectorOfVariables.
For the variable that do need variable bridges. If they are part of other variable constraint then they will be force-bridged into affine constraints so there is a hidden cost in terms of number of additional number of bridges that will need to be used. In fact, if the order matter, it means the variable is at least in one other variable constraint so in a sense we could do x - y + sign(x) but I think !iszero(x), x - y is fine.

[odow]

I added a big chunk to the docstring, including the Pajarito example. An open research question for MOI 2.0 could be how to update the bridging costs in the presence of variable bridges. Getting that right would fix all of this.