CBC from python mip library is way slower than cvxpy+CBC

[Joanna-Wojdylo]

I have the following problem, I need to use CBC solver for my optimization problem and from some business reasons I cannot use regular cvxpy and CBC installed as 3rd party application. That is why I used python mip library (mip docs) which includes pre-compiled libraries of the MIP Solver CBC. When using external CBC problem gets solved in couple minutes, however when using mip CBC it gets stuck:

 CVXPY                                     
                                    v1.1.11                                    
===============================================================================
(CVXPY) Mar 09 09:03:08 AM: Your problem has 182166 variables, 5 constraints, and 0 parameters.
(CVXPY) Mar 09 09:03:08 AM: It is compliant with the following grammars: DCP, DQCP
(CVXPY) Mar 09 09:03:08 AM: (If you need to solve this problem multiple times, but with different data, consider using parameters.)
(CVXPY) Mar 09 09:03:08 AM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.
-------------------------------------------------------------------------------
                                  Compilation                                  
-------------------------------------------------------------------------------
(CVXPY) Mar 09 09:03:08 AM: Compiling problem (target solver=PYTHON_MIP).
(CVXPY) Mar 09 09:03:08 AM: Reduction chain: FlipObjective -> Dcp2Cone -> CvxAttr2Constr -> ConeMatrixStuffing -> PYTHON_MIP
(CVXPY) Mar 09 09:03:08 AM: Applying reduction FlipObjective
(CVXPY) Mar 09 09:03:08 AM: Applying reduction Dcp2Cone
(CVXPY) Mar 09 09:03:08 AM: Applying reduction CvxAttr2Constr
(CVXPY) Mar 09 09:03:08 AM: Applying reduction ConeMatrixStuffing
(CVXPY) Mar 09 09:03:08 AM: Applying reduction PYTHON_MIP
(CVXPY) Mar 09 09:03:09 AM: Finished problem compilation (took 1.194e+00 seconds).
(CVXPY) Mar 09 09:03:09 AM: (Subsequent compilations of this problem, using the same arguments, should take less time.)
-------------------------------------------------------------------------------
                                Numerical solver                               
-------------------------------------------------------------------------------
(CVXPY) Mar 09 09:03:09 AM: Invoking solver PYTHON_MIP to obtain a solution.
INFO:mip.model:Using Python-MIP package version 1.13.0

FYI: I am using custom solver interface to run mip library with cvxpy syntax as described here: coin-or/python-mip#182

I am aware, that through the documentation it seems that python-mip CBC and binaries of CBC are the same but I would like to confirm it because as said: using python mip CBC and the mip_conif.py interface, the problem gets stuck (and it works correctly when using CBC binaries). Can you please confirm that using CBC via mip library leads to the exact same CBC as in traditional cvxpy way?

[tkralphs]

When you say it "gets stuck," what do you mean? How long did you wait for a solution?

The algorithms that are used to solve MILPs have a high performance variability in general. Even permuting the rows and columns of an instance (which results in a mathematically equivalent instance) can have a dramatic effect on the time required to solve the instance. This unpredictability is somewhat unavoidable. Over a large enough collection of instances, it's likely you would not see much performance difference, provided the algorithmic parameters are the same, but on a single instance, different ways of feeding the same problem to the solver could cause performance differences. If you are seeing this on a single instance, I wouldn't focus on it too much. Provided you have reason to believe the parameters settings are the same in each case (this would need to be verified), the overall performance should not be different.

[Joanna-Wojdylo]

Thank you @tkralphs for the answer. Replying to your questions, so by "gets stuck" I mean that the after calling the mip-python interface to use built-in CBC I am waiting for some time (max I was waiting 2h) and nothing happens, no errors, no information about not converging, it just hangs. Also, I am running it on a single instance, but I cannot not focus on that because that is crucial business problem and it is not acceptable to wait for the optimization solution for so long (I don't even know if this would finally compute ever). What interested me most in your comment, could you say something more about parameters settings? Which parameters do you mean? At this moment I am sure that I am running optimization problem using external CBC+cvxpy and mip_python+interface on the exactly same python env, with exactly the same optimization objective and constraints.

I will try to go over this freezing program with debugger, and in case of any more specific findings I will post it here. Meanwhile I would really appreciate you comment on these parameters settings, thank you so much!

[odow]

This issue is not a bug in Cbc.

On one hand, you are comparing "CVXPY + CBC", on the other, you are comparing "CVXPY + PYTHON_MIP + a different Cbc binary".

The time difference could be for a number of reasons;

1. The binaries are different
2. The row orders are different
3. The CVXPY -> PYTHON_MIP interface is slow.

From coin-or/python-mip#182

Keep in mind the code I've written here is by no means robust, tested, or well thought out.

I suggest you either

1. Talk to your IT and use the external Cbc
2. Rewrite your code to python-mip.