Installation broken on Ubuntu 22.04 with Conda 23.1.0

[Time0o]

I have installed CyLP according to the instructions under "On Linux/macOS with conda: Installation from source".

Now, when trying to import cylp.cy I get the following error:

>>> import cylp.cy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/timo/miniconda3/envs/cbc/lib/python3.11/site-packages/cylp/cy/__init__.py", line 1, in <module>
    from .CyCoinIndexedVector import CyCoinIndexedVector
ImportError: /home/timo/miniconda3/envs/cbc/lib/python3.11/site-packages/cylp/cy/CyCoinIndexedVector.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN9CoinError12printErrors_E

There seems to be a version mismatch between cylp and coin-or-cbc.

[tkralphs]

I can replicate and it's strange. It's not a version mismatch, it's that CyCoinIndexedVector.cpython-311-x86_64-linux-gnu.so doesn't seem to have a dependence on the COIN-OR libraries after the build, as it seems it should. It doesn't find the symbol simply because it doesn't look for the library.

(cylp) ~/Projects/CyLP/build/lib.linux-x86_64-cpython-39/cylp/cy > ldd CyCoinIndexedVector.cpython-39-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffe5d354000)
        libstdc++.so.6 => /home/ted/Projects/miniforge3/envs/cylp/lib/libstdc++.so.6 (0x00007fe6461dc000)
        libgcc_s.so.1 => /home/ted/Projects/miniforge3/envs/cylp/lib/libgcc_s.so.1 (0x00007fe6461c3000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe646187000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe645f95000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe645e46000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe6463a1000)

The last time I touched the build was about a year ago and at that time, we had changed some things to support building binary wheels for Linux and OS X. As such ,there are binary wheels for Linux available, but only for Python 3.7-3.9.

It looks like conda is defaulting to Python 11, but if you create a conda environment with Python 3.9

conda create -n cylp python=3.9 ...

and then pip install cylp, you will get a binary wheel that works for me.

(cylp) ~/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy > ldd CyCoinIndexedVector.cpython-39-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffcfa943000)
        libCbcSolver-0461f6dc.so.3.10.7 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libCbcSolver-0461f6dc.so.3.10.7 (0x00007fa251dd9000)
        libCbc-a2fe6983.so.3.10.7 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libCbc-a2fe6983.so.3.10.7 (0x00007fa251bee000)
        libCgl-b42445a7.so.1.10.5 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libCgl-b42445a7.so.1.10.5 (0x00007fa251a71000)
        libOsiClp-ea1dbdb6.so.1.14.7 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libOsiClp-ea1dbdb6.so.1.14.7 (0x00007fa2519ea000)
        libClpSolver-21b6c261.so.1.14.7 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libClpSolver-21b6c261.so.1.14.7 (0x00007fa25196c000)
        libClp-e9aede7a.so.1.14.7 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libClp-e9aede7a.so.1.14.7 (0x00007fa25175e000)
        libOsi-1bfe6771.so.1.13.7 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libOsi-1bfe6771.so.1.13.7 (0x00007fa2516ec000)
        libCoinUtils-3ae710b8.so.3.11.6 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libCoinUtils-3ae710b8.so.3.11.6 (0x00007fa251583000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fa25154e000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa251544000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa251362000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa251211000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa2511f6000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa2511d3000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa250fe1000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa251f87000)

In the meantime, I'll see if I can chase down this issue.

[tkralphs]

The problem traces back to the addition of this patch, which resulted in dependent libraries not being explicitly linked when building. The patch itself was actually added to prevent over-linking, which is the opposite problem. Everything seems to work now. Could you try by cloning the repo and pip installing from source? Feel free to re-open if there is a problem and thanks for reporting it! Once you confirm, I'll make a new release and also set up creation of wheels for Python 3.10 and 3.11.

[Time0o]

@tkralphs Seems to work fine now, thank you!

[tkralphs]

By the way, there is now a new release with binary wheels for Linux with up to Python 3.11.

[neumeyerx]

Fyi @tkralphs , I faced the same issue on M1 mac.

brew install cbc pkg-config
poetry add cylp

(Also tried pip install cylp even though it should not make a difference.)

import cylp worked, but importing something else triggered this error (anonymized):

Python 3.10.13 (main, Nov  1 2023, 16:34:31) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from cylp.py.modeling.CyLPModel import IndexFactory
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/<username>/Projekte/<project_folder>/.venv/lib/python3.10/site-packages/cylp/py/__init__.py", line 1, in <module>
    from . import pivots
  File "/Users/<username>/Projekte/<project_folder>/.venv/lib/python3.10/site-packages/cylp/py/pivots/__init__.py", line 3, in <module>
    from .PositiveEdgePivot import PositiveEdgePivot
  File "/Users/<username>/Projekte/<project_folder>/.venv/lib/python3.10/site-packages/cylp/py/pivots/PositiveEdgePivot.py", line 9, in <module>
    from cylp.cy import CyCoinIndexedVector
  File "/Users/<username>/Projekte/<project_folder>/.venv/lib/python3.10/site-packages/cylp/cy/__init__.py", line 1, in <module>
    from .CyCoinIndexedVector import CyCoinIndexedVector
ImportError: dlopen(/Users/<username>/Projekte/<project_folder>/.venv/lib/python3.10/site-packages/cylp/cy/CyCoinIndexedVector.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__ZN9CoinError12printErrors_E

I resolved this going down this path (using poetry)

brew install cbc pkg-config
poetry add Cython
git clone https://github.com/coin-or/CyLP.git
cd CyLP
pip install -e .

Now, imports work and CBC solver executes. But for some reason following the "On Linux/macOS with pip: Installation from source" cylp setup did not work for me 😕.