Trying to link mip to Debian's CBC package throws undefined error #165

sjferc · 2020-11-30T09:37:49Z

I installed coinor-cbc to my armhf system, and did this:

export PMIP_CBC_LIBRARY="/usr/lib/arm-linux-gnueabihf/libCbcSolver.so.3.9.9"
export LD_LIBRARY_PATH="/usr/lib/arm-linux-gnueabihf/"

as established here.

Then, when I try to execute it on Python's console, I do:

>>> from mip import Model, MAXIMIZE, CBC, INTEGER, OptimizationStatus
>>> model = Model(sense=MAXIMIZE, solver_name=CBC)

And I inmediately get the error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/mip/model.py", line 93, in __init__
    import mip.cbc
  File "/usr/local/lib/python3.7/dist-packages/mip/cbc.py", line 600, in <module>
    Osi_getNumCols = cbclib.Osi_getNumCols
  File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 912, in __getattr__
    make_accessor(name)
  File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 908, in make_accessor
    accessors[name](name)
  File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 838, in accessor_function
    value = backendlib.load_function(BType, name)
AttributeError: function/symbol 'Osi_getNumCols' not found in library '/usr/lib/arm-linux-gnueabihf/libCbcSolver.so.3.9.9': /usr/lib/arm-linux-gnueabihf/libCbcSolver.so.3.9.9: undefined symbol: Osi_getNumCols

I am using

Python 3.7.3
mip 1.12.0
coinor-cbc 2.9.9+repack1-1
Debian GNU/Linux 10 (buster)
Kernel: Linux 4.14.78-g8e54a4b
Architecture: armv7l

The text was updated successfully, but these errors were encountered:

h-g-s · 2020-11-30T12:03:11Z

Hi, I think that Debian probably includes the last stable (API stable) version of CBC. Python-MIP requires the development version of CBC. From: sjferc <notifications@github.com> Reply-To: coin-or/python-mip <reply@reply.github.com> Date: Monday, November 30, 2020 at 6:38 AM To: coin-or/python-mip <python-mip@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: [coin-or/python-mip] Trying to link mip to Debian's CBC package throws undefined error (#165) I installed coinor-cbc to my armhf system, and did this: export PMIP_CBC_LIBRARY="/usr/lib/arm-linux-gnueabihf/libCbcSolver.so.3.9.9" export LD_LIBRARY_PATH="/usr/lib/arm-linux-gnueabihf/" as established here. Then, when I try to execute it on Python's console, I do:

>> from mip import Model, MAXIMIZE, CBC, INTEGER, OptimizationStatus >> model = Model(sense=MAXIMIZE, solver_name=CBC)

And I inmediately get the error: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.7/dist-packages/mip/model.py", line 93, in __init__ import mip.cbc File "/usr/local/lib/python3.7/dist-packages/mip/cbc.py", line 600, in <module> Osi_getNumCols = cbclib.Osi_getNumCols File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 912, in __getattr__ make_accessor(name) File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 908, in make_accessor accessors[name](name) File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 838, in accessor_function value = backendlib.load_function(BType, name) AttributeError: function/symbol 'Osi_getNumCols' not found in library '/usr/lib/arm-linux-gnueabihf/libCbcSolver.so.3.9.9': /usr/lib/arm-linux-gnueabihf/libCbcSolver.so.3.9.9: undefined symbol: Osi_getNumCols I am using Python 3.7.3 mip 1.12.0 coinor-cbc 2.9.9+repack1-1 Debian GNU/Linux 10 (buster) Kernel: Linux 4.14.78-g8e54a4b Architecture: armv7l — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

sjferc · 2020-12-02T17:54:08Z

Okay, so I did install it with coinbrew like this:

export CFLAGS="-Ofast -fPIC -flto -DNDEBUG -fprefetch-loop-arrays -I/opt/gcc/include/"
export FFLAGS="-Ofast -fPIC -flto -DNDEBUG -I/opt/gcc/include/"
export CXXFLAGS="-Ofast -fPIC -flto -DNDEBUG -I/opt/gcc/include/"
export LDFLAGS="-Ofast -fPIC -L/opt/gcc/lib -flto -static-libgcc -static-libstdc++ -static-libgfortran"

coinbrew build Cbc -j 1 --no-third-party --tests none

Which installed latest release, 2.10.5, and properly linked the libraries again but got:

>>> from mip import Model, MAXIMIZE, CBC, INTEGER, OptimizationStatus
>>> model = Model(sense=MAXIMIZE, solver_name=CBC)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/mip/model.py", line 93, in __init__
    import mip.cbc
  File "/usr/local/lib/python3.7/dist-packages/mip/cbc.py", line 603, in <module>
    Osi_getIntegerTolerance = cbclib.Osi_getIntegerTolerance
  File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 912, in __getattr__
    make_accessor(name)
  File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 908, in make_accessor
    accessors[name](name)
  File "/usr/local/lib/python3.7/dist-packages/cffi/api.py", line 838, in accessor_function
    value = backendlib.load_function(BType, name)
AttributeError: function/symbol 'Osi_getIntegerTolerance' not found in library '/downloads/dist/lib/libCbcSolver.so': /downloads/dist/lib/libCbcSolver.so: undefined symbol: Osi_getIntegerTolerance

Is this still an incorrect version of CBC?

tkralphs · 2020-12-02T17:59:49Z

You need to do

coinbrew build Cbc@master -j 1 --no-third-party --tests none

to get the development version, but you should remove the build and dist directories first.

sjferc · 2020-12-02T18:46:49Z

Thank you! I will test that out.
Another thing I noticed... from the flags, there is -I/opt/gcc/include/, but actually that directory does not exist locally.
Do you know how could I check where is my equivalent?

It also happens with: -L/opt/gcc/lib.

tkralphs · 2020-12-03T19:01:00Z

In general, you shouldn't need to set any flags manually like you have. I assumed you had some specific reason for doing that and didn't look too closely at them. You usually would not need any flag such as -I/opt/gcc/include/ except in very specific circumstances.

If you want to build static executables, which it looks like you are trying to do, just add --static to coinbrew. Generally speaking, appropriate flags are set during configuration for each project. If you try to set flags on your own, there's a good chance you will mess something up by, e.g., missing out a flag which is necessary for defining an important symbol.

If necessary, you can augment the pre-defined flags using ADD_CXXFLAGS, etc.

gewesp · 2022-03-04T08:29:26Z

coinbrew build Cbc@master -j 1 --no-third-party --tests none

I tried this today and it worked, however it seems that the library has been renamed in the Cbc master
and Python-MIP has not been updated (yet).

To make it work with Python-MIP you need to do the following after the coinbrew command:

cd dist/lib
ln -s libCbc.dylib -> libCbcSolver.dylib

This is for MacOS, for Linux probably replace .dylib by .so.

Using this command, I got Python-MIP working on a Mac M1 ('Apple silicon').

In the long run, hopefully Python-MIP will distribute appropriate binaries and 'freeze' the Cbc version it is using.

EDIT Mar 9, 2022: It also works to point PMIP_CBC_LIBRARY directly to the libCbc dynamic library (as opposed to libCbcSolver), e.g.

export PMIP_CBC_LIBRARY="$CBC_LIBRARIES_DIR/libCbc.dylib"

(Define CBC_LIBRARYES_DIR appropriately).

tkralphs · 2022-03-04T17:03:53Z

I tried this today and it worked, however it seems that the library has been renamed in the Cbc master
and Python-MIP has not been updated (yet).

Yes, there used to be a libCbc and a libCbcSolver, but this separation did not serve much purpose, so the libraries were merged as part of the on-going refactoring being undertaken in the master branch. I'm not sure why you need to do the dynamic linking , though. You should be able to just set PMIP_CBC_LIBRARY to point directly to libCbc, unless I'm missing something.

We are still trying to get to a formal release of the code in Cbc master branch, but it still needs some work and developer bandwidth is severely limited. The work to be done doesn't need that much knowledge of Cbc internals, volunteers are welcome. See coin-or/Cbc/discussions/465.

gewesp · 2022-03-08T07:17:38Z

I'm not sure why you need to do the dynamic linking , though. You should be able to just set PMIP_CBC_LIBRARY to point directly to libCbc, unless I'm missing something.

Much appreciated Ted! I was following the Python-MIP documentation which mentions dynamic linking:

"Python-MIP uses the CbcSolver shared library [...] In [...] MacOS the extension should be .dll and .dylp, respectively. To force Python-MIP to use your [...] CBC binaries, you can set the PMIP_CBC_LIBRARY environment variable, indicating the full path to this shared library." (emphasis mine)

tkralphs · 2022-03-08T18:10:09Z

Sorry, I misspoke. I didn't mean to say dynamic linking, I meant symbolic linking. You created a symbolic link from libCbc to libCbcSolver when I think you could just directly point python-mip to libCbc by setting PMIP_CBC_LIBRARY.

gewesp · 2022-03-09T05:18:04Z

Indeed, that works as well, thanks!

jurasofish · 2022-05-28T04:17:51Z

Leaving this here for reference

I've just gotten an M1 Mac and I found that using CBC built from today's master failed some tests in my application - mostly just extreme edge cases, so I guess it's just normal variance between different versions of CBC. But I wanted to have CBC as identical as possible to what's in python-mip v1.13.0

So I wanted to build CBC for M1 Mac using the exact same version of CBC that's used in python-mip 1.13.0.
The python-mip CBC binary was last updated around 2020-11-15 which seems likely to be built from this commit of CBC. That is, CBC master @ 2020-11-15T22:40:00Z

When building CBC master version using coinbrew it seems that coinbrew checks out master versions of all the dependencies, if I'm interpreting the contents of the .coin-or folder correctly

So, I've put together a little script that checks out all the dependent repositories to master at the time that I want and then builds CBC. And it seems to work, just need env var PMIP_CBC_LIBRARY=/...../libCbcSolver.dylib
@tkralphs does this seem kosher to you?

# see https://stackoverflow.com/a/6990682 for git checkout to specific date
bash coinbrew fetch Cbc@master
cd Cbc
git checkout `git rev-list -n 1 --before="2020-11-15T22:41:00Z" --date=iso master`
cd ../Cgl
git checkout `git rev-list -n 1 --before="2020-11-15T22:41:00Z" --date=iso master`
cd ../Clp
git checkout `git rev-list -n 1 --before="2020-11-15T22:41:00Z" --date=iso master`
cd ../CoinUtils
git checkout `git rev-list -n 1 --before="2020-11-15T22:41:00Z" --date=iso master`
cd ../Osi
git checkout `git rev-list -n 1 --before="2020-11-15T22:41:00Z" --date=iso master`
cd ../ThirdParty/ASL
git checkout `git rev-list -n 1 --before="2020-11-15T22:41:00Z" --date=iso master`
cd ../Glpk
git checkout `git rev-list -n 1 --before="2020-11-15T22:41:00Z" --date=iso master`
# mumps doesn't seem to have a master branch
# cd ../Mumps
# git checkout `git rev-list -n 1 --before="2020-11-15T22:41:00Z" --date=iso master`
cd ../..
bash coinbrew build Cbc -j 1 --no-third-party --tests none

tkralphs · 2022-05-29T17:20:45Z

Yes, in fact, coinbrew has an option for doing exactly what you're trying to do. From the help:

--time check out project and all dependencies at a time stamp

It's admittedly not been used (by me) since when I actually needed it and added it as an option. If you have a chance to try it, that would be great.

gewesp mentioned this issue Mar 4, 2022

wheel on PyPI is marked as the "any" platform but is actually platform specific #199

Open

jurasofish mentioned this issue May 28, 2022

How to provide your own CBC binaries #279

Closed

jurasofish mentioned this issue Sep 7, 2022

Latest Cbc build doesn't work with python-mip from pip #239

Open

nessdoor mentioned this issue Dec 4, 2022

python3Packages.mip: init at 1.14.1 NixOS/nixpkgs#197222

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trying to link mip to Debian's CBC package throws undefined error #165

Trying to link mip to Debian's CBC package throws undefined error #165

sjferc commented Nov 30, 2020

h-g-s commented Nov 30, 2020 via email

sjferc commented Dec 2, 2020 •

edited

Loading

tkralphs commented Dec 2, 2020

sjferc commented Dec 2, 2020 •

edited

Loading

tkralphs commented Dec 3, 2020 •

edited

Loading

gewesp commented Mar 4, 2022 •

edited

Loading

tkralphs commented Mar 4, 2022

gewesp commented Mar 8, 2022

tkralphs commented Mar 8, 2022 •

edited

Loading

gewesp commented Mar 9, 2022

jurasofish commented May 28, 2022 •

edited

Loading

tkralphs commented May 29, 2022

Trying to link mip to Debian's CBC package throws undefined error #165

Trying to link mip to Debian's CBC package throws undefined error #165

Comments

sjferc commented Nov 30, 2020

h-g-s commented Nov 30, 2020 via email

sjferc commented Dec 2, 2020 • edited Loading

tkralphs commented Dec 2, 2020

sjferc commented Dec 2, 2020 • edited Loading

tkralphs commented Dec 3, 2020 • edited Loading

gewesp commented Mar 4, 2022 • edited Loading

tkralphs commented Mar 4, 2022

gewesp commented Mar 8, 2022

tkralphs commented Mar 8, 2022 • edited Loading

gewesp commented Mar 9, 2022

jurasofish commented May 28, 2022 • edited Loading

tkralphs commented May 29, 2022

sjferc commented Dec 2, 2020 •

edited

Loading

sjferc commented Dec 2, 2020 •

edited

Loading

tkralphs commented Dec 3, 2020 •

edited

Loading

gewesp commented Mar 4, 2022 •

edited

Loading

tkralphs commented Mar 8, 2022 •

edited

Loading

jurasofish commented May 28, 2022 •

edited

Loading