-
Notifications
You must be signed in to change notification settings - Fork 33
Description
At the moment we are building our kalign-python wheels with OpenMP support and shipping our own copy own copy of libomp.dylib, bundled next to the extension.
otool -L _core.cpython-313-darwin.so
_core.cpython-313-darwin.so:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.120.2)
@loader_path/.dylibs/libomp.dylib (compatibility version 5.0.0, current version 5.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1700.255.5)This makes the precompiled wheel not combine well with anything that loads a different version of OpenMP, particularly making it fail with conda environments as soon as any dependency builds against OpenMP.
As an example, given the following environment.yml
name: kalign-test
channels:
- conda-forge
- nodefaults
dependencies:
- python<3.14
- pip
- numpy
- pip:
- kalign-pythonand test file:
import kalign
SEQUENCES = [
'MLNSFKLSLQYILPKLWLTRLAGWGASKRAGWLTKLVIDLFVKYYKVDMKEAQKPDTASYRTFNEFFVRPLRDEVRPIDTDPNVLVMPADGVISQLGKIEEDKILQAKGHNYSLEALLAGNYLMADLFRNGTFVTTYLSPRDYHRVHMPCNGILREMIYVPGDLFSVNHLTAQNVPNLFARNERVICLFDTEFGPMAQILVGATIVGSIETVWAGTITPPREGIIKRWTWPAGENDGSVALLKGQEMGRFKLG',
'LQLKGMSYSMCTGKFKVVKEIAETQHGTIVIRVQYEGDGSPCKIPFEIMDLEKRHVLGRLITVNPIVTEk',
'kLQLKGMSYSMCTGKFKIVKEIAETQHGTIVIRVQYEGDGSPCKIPFEITDLEKRHVLGRLITVNPIVTEk',
]
print(kalign.align(SEQUENCES))We get:
OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/
[1] 20026 abort python ktest.py
Setting KMP_DUPLICATE_LIB_OK=TRUE let's the example run, but it is discouraged and might have side effects down the road.
Would it be possible to compile the wheels without OpenMP, at least for OSX? What would be the performance impact? Is there an alternative to build more generic wheels?
As a note, we are trying to bring a conda package for kalign-python here, which should help for the conda ecosystem:
conda-forge/staged-recipes#32505