error running compiled sklearn on anaconda with accelerate and mkl

[satra]

i'm running into a library issue. on osx, with anaconda and accelerate

the setup process works fine but then when i try to execute something it breaks here:

from ..pairwise import pairwise_distances
     11 
     12 

/software/anaconda/envs/devpype/lib/python2.7/site-packages/sklearn/metrics/pairwise.py in <module>()
     53 from ..externals.joblib.parallel import cpu_count
     54 
---> 55 from .pairwise_fast import _chi2_kernel_fast, _sparse_manhattan
     56 
     57 

ImportError: dlopen(/software/anaconda/envs/devpype/lib/python2.7/site-packages/sklearn/metrics/pairwise_fast.so, 2): Library not loaded: libmkl_intel_lp64.dylib
  Referenced from: /software/anaconda/envs/devpype/lib/python2.7/site-packages/sklearn/metrics/pairwise_fast.so
  Reason: image not found

is there a known workaround? i guess i can change the DYLD libraries, but didn't see this issue in search.

$ conda info
Current conda install:

             platform : osx-64
        conda version : 3.6.2
  conda-build version : not installed
       python version : 2.7.8.final.0
     requests version : 2.3.0
     root environment : /software/anaconda  (writable)
  default environment : /software/anaconda/envs/devpype
     envs directories : /software/anaconda/envs
        package cache : /software/anaconda/pkgs
         channel URLs : http://repo.continuum.io/pkgs/free/osx-64/
                        http://repo.continuum.io/pkgs/pro/osx-64/
          config file : None
    is foreign system : False

[satra]

the DYLD_library option didn't work.

i can compile by forcing these lines in _build_utils.py - but a better solution is probably needed.

    if (not blas_info) or atlas_not_found(blas_info) or True:
        cblas_libs = ['cblas']
        blas_info.pop('libraries', None)

[ogrisel]

the DYLD_library option didn't work.

Can you give more details? What did you do and what was the error message? Have you tried setting DYLD_FALLBACK_LIBRARY_PATH?

http://stackoverflow.com/questions/3146274/is-it-ok-to-use-dyld-library-path-on-mac-os-x-and-whats-the-dynamic-library-s

[satra]

the error message was a clash with the JPEG library with the ipython notebook when i simply set DYLD_LIBRARY_PATH

dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libTIFF.dylib
  Expected in: /software/anaconda/envs/devpype/lib//libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libTIFF.dylib

however: DYLD_FALLBACK_LIBRARY_PATH works fine.

for the moment i had fixed it with the following based on a suggestion from the anaconda list:

find . -name "*.so" | xargs -I fname ../change_libs.sh fname

where change_libs.sh is

#!/bin/bash

echo "Changing $1"
install_name_tool -change libmkl_intel_lp64.dylib /software/anaconda/envs/devpype/lib/libmkl_intel_lp64.dylib $1
install_name_tool -change libmkl_intel_thread.dylib /software/anaconda/envs/devpype/lib/libmkl_intel_thread.dylib $1
install_name_tool -change libmkl_core.dylib /software/anaconda/envs/devpype/lib/libmkl_core.dylib $1
install_name_tool -change libiomp5.dylib /software/anaconda/envs/devpype/lib/libiomp5.dylib $1

given there are two ways to solve it, i'm closing it for now.