Working MIGP

[bqpd]

Closes #414

potential extensions:

• allow for choosing between sets of substitutions
  • allow "freeing" variables to be between eps and 1/eps if they're only in some of the sets of substitutions
• allow for choosing between sets of ConstraintSets

@rileyjmurray @erling-d-andersen, this implements a binary variable technique for discretizing logspace variabes, but it's a bit...slow. Any recommendations? Is this just because solving integer problems has a constant-speed slowdown? (if there's only one "choice" there's no slowdown, but I presume that's the presolver optimizing it away...)

[1ozturkbe]

I believe you are adding a new binary var for each possible integer solution to do this, so the slowdown is is somewhat expected @bqpd. There are two possible formulations here.

Rn you do this (at least from what I can discern).
new binary variable z[i=1:number of options]
sum(z[i]) = 1
choice = sum(choices * z)

The following is likely faster, since it introduces hierarchy into binary vars. (Just so you know, I rushed this so something might be slightly off about the formulation.)
new binary variable z[i=1:number of options]
z[j] >= z[j+1] for all j=1:number_of_options-1
choice = choices[0]*z[0] + sum(choice_differences[1:] * z[1:])
where the choice differences are the logspace differences between choices[0] and choices[1] etc.
I hope that helps.

[bqpd]

oh indeed, that's a much better way to do it!

[bqpd]

unfortunately that didn't make it any faster on a large problem with a single discrete variable with several choices (which takes ~15-20 times longer than without the binary variable, which is obvs less efficient than just solving each case separately...), but it does reduce the number of auxiliary binary variables by 1.

[rileyjmurray]

@bqpd different mixed-integer solvers will use different branching strategies depending on the integer formulation. So even if MOSEK isn't running much faster, it's possible that other solvers will (should any be added in the future). I specifically suspect that simpler (open-source) mixed-integer solvers will do better with the new formulation.

[1ozturkbe]

@bqpd three questions.

1. What is the size of your integer domain? Don't want to crush your hopes, but I don't think MIGP will be tractable in the scale that MILP is, at least with current solvers. I think most MICP algorithms are lightweight and don't do anything much smarter than B&B under the hood (eg. cutting planes etc.). And a naive approach to B&B will mostly enumerate...
2. If your problem is a GP, have you tried solving therelaxed problem, then only allowing integer solutions "in the vicinity" of the relaxation and resolving? If the user wants a higher quality solution, you could run the more exhaustive MIGP approach, but I assume the first will work optimally in most cases. This is what a lot of MICP solvers do.
3. Have you tried passing an initial guess to the second formulation? I know this is not ideal, but good guesses dramatically speed up B&B, much more so than IP solves.

(edited by @bqpd to add numbers)

[bqpd]

in re: @1ozturkbe

1. ha it's very small right now, just a few options! haven't even tried multiple discrete variables yet.

2. I'm letting MOSEK do their conic integer alg rather than implementing my own branch & bound, so I presume they're handling that (and given the results upthread they're doing something a little more complex than what I would have implemented on my own!)

3. I haven't! I've been thinking what an interface for that might be; my feeling rn is that because they're discrete sets rather than integers this is much less of a problem, but I could either allow for a guess or automatically choose the median?

[bqpd]

test models please

[bqpd]

test models please

[bqpd]

test models please

[bqpd]

retest this please

[bqpd]

test models please