libmxnet.so load path error

[bzcheeseman]

Description

After not being able to run a project that links against mxnet because of the following error:
dyld: Library not loaded: lib/libmxnet.so, I discovered that
otool -L libmxnet.so produces

lib/libmxnet.so (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.5.0)

and the mxnet/lib directory contains libmxnet.a and libmxnet.so.

EDIT: I realized that I'm an idiot and the first line should be there. I also realized, however, that it should likely be an rpath or something, and the solution will likely be with install_name_tool.

Environment info (Required)

----------Python Info----------
Version      : 3.6.2
Compiler     : GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)
Build        : ('default', 'Jul 20 2017 13:14:59')
Arch         : ('64bit', '')
------------Pip Info-----------
Version      : 9.0.1
Directory    : /Users/Aman/anaconda3/lib/python3.6/site-packages/pip
----------MXNet Info-----------
Version      : 1.0.0
Directory    : /Users/Aman/code/mxnet/python/mxnet
Hashtag not found. Not installed from pre-built package.
----------System Info----------
Platform     : Darwin-16.7.0-x86_64-i386-64bit
system       : Darwin
node         : Amans-MacBook-Pro.local
release      : 16.7.0
version      : Darwin Kernel Version 16.7.0: Mon Nov 13 21:56:25 PST 2017; root:xnu-3789.72.11~1/RELEASE_X86_64
----------Hardware Info----------
machine      : x86_64
processor    : i386
b'machdep.cpu.extfeatures: SYSCALL XD EM64T LAHF RDTSCP TSCI'
b'machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS'
b'machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC POPCNT AES PCID XSAVE OSXSAVE TSCTMR AVX1.0 RDRAND F16C'
b'machdep.cpu.brand_string: Intel(R) Core(TM) i7-3740QM CPU @ 2.70GHz'
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0167 sec, LOAD: 0.7192 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.2221 sec, LOAD: 0.2984 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.3973 sec, LOAD: 0.2155 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.1764 sec, LOAD: 0.2118 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0351 sec, LOAD: 0.3496 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0566 sec, LOAD: 0.1383 sec.

Package used (Python/R/Scala/Julia): C++

Build info (Required if built from source)

make USE_GPERFTOOLS=1 USE_CPP_PACKAGE=1 USE_OPENMP=0 USE_OPENCV=0 LDFLAGS="-L/usr/local/lib -framework Accelerate"

I should also note that I tried building with opencv3 and the build errors out with being unable to find the cv::Mat and other symbols:

cv::Mat::deallocate()", referenced from:
      mxnet::io::DefaultImageAugmenter::DefaultImageAugmenter() in image_aug_default.o
      mxnet::io::DefaultImageAugmenter::Process(cv::Mat const&, std::__1::vector<float, std::__1::allocator<float> >*, std::__1::mersenne_twister_engine<unsigned int, 32ul, 624ul, 397ul, 31ul, 2567483615u, 11ul, 4294967295u, 7ul, 2636928640u, 15ul, 4022730752u, 18ul, 1812433253u>*) in image_aug_default.o
      cv::Mat::operator=(cv::Mat const&) in image_aug_default.o
      mxnet::io::DefaultImageAugmenter::~DefaultImageAugmenter() in image_aug_default.o
      mxnet::io::DefaultImageDetAugmenter::Process(cv::Mat const&, std::__1::vector<float, std::__1::allocator<float> >*, std::__1::mersenne_twister_engine<unsigned int, 32ul, 624ul, 397ul, 31ul, 2567483615u, 11ul, 4294967295u, 7ul, 2636928640u, 15ul, 4022730752u, 18ul, 1812433253u>*) in image_det_aug_default.o
      mxnet::io::DefaultImageDetAugmenter::~DefaultImageDetAugmenter() in image_det_aug_default.o
      mxnet::io::DefaultImageDetAugmenter::~DefaultImageDetAugmenter() in image_det_aug_default.o
      ...
  "cv::Mat::create(int, int const*, int)", referenced from:
      mxnet::io::DefaultImageAugmenter::DefaultImageAugmenter() in image_aug_default.o
      mxnet::io::DefaultImageAugmenter::Process(cv::Mat const&, std::__1::vector<float, std::__1::allocator<float> >*, std::__1::mersenne_twister_engine<unsigned int, 32ul, 624ul, 397ul, 31ul, 2567483615u, 11ul, 4294967295u, 7ul, 2636928640u, 15ul, 4022730752u, 18ul, 1812433253u>*) in image_aug_default.o
  "cv::Mat::copySize(cv::Mat const&)", referenced from:
      cv::Mat::operator=(cv::Mat const&) in image_aug_default.o
      mxnet::io::DefaultImageDetAugmenter::Process(cv::Mat const&, std::__1::vector<float, std::__1::allocator<float> >*, std::__1::mersenne_twister_engine<unsigned int, 32ul, 624ul, 397ul, 31ul, 2567483615u, 11ul, 4294967295u, 7ul, 2636928640u, 15ul, 4022730752u, 18ul, 1812433253u>*) in image_det_aug_default.o
      cv::Mat::operator=(cv::Mat const&) in image_det_aug_default.o
  "cv::Mat::Mat(cv::Mat const&, cv::Rect_<int> const&)", referenced from:
      mxnet::io::DefaultImageAugmenter::Process(cv::Mat const&, std::__1::vector<float, std::__1::allocator<float> >*, std::__1::mersenne_twister_engine<unsigned int, 32ul, 624ul, 397ul, 31ul, 2567483615u, 11ul, 4294967295u, 7ul, 2636928640u, 15ul, 4022730752u, 18ul, 1812433253u>*) in image_aug_default.o
      mxnet::io::DefaultImageDetAugmenter::Process(cv::Mat const&, std::__1::vector<float, std::__1::allocator<float> >*, std::__1::mersenne_twister_engine<unsigned int, 32ul, 624ul, 397ul, 31ul, 2567483615u, 11ul, 4294967295u, 7ul, 2636928640u, 15ul, 4022730752u, 18ul, 1812433253u>*) in image_det_aug_default.o
  "cv::flip(cv::_InputArray const&, cv::_OutputArray const&, int)", referenced from:
      mxnet::io::DefaultImageDetAugmenter::Process(cv::Mat const&, std::__1::vector<float, std::__1::allocator<float> >*, std::__1::mersenne_twister_engine<unsigned int, 32ul, 624ul, 397ul, 31ul

MXNet branch v0.12.0

Error Message:

dyld: Library not loaded: lib/libmxnet.so

Steps to reproduce

find_library(MXNET_LIBRARY
        NAMES libmxnet.so
        PATHS $ENV{MXNET_ROOT}/lib
)
add_library(my_lib SHARED ${sources})
target_link_libraries(my_lib PUBLIC ${MXNET_LIBRARY})

and then try to build with usual cmake commands.

What have you tried to solve it?

I've tried using install_name_tool to change the linkage manually to libmxnet.a but that doesn't work. I've also tried setting my LD_LIBRARY_PATH and PATH to the mxnet root

I also just tried checking out the v0.11.0 tag, that doesn't work either.

[bzcheeseman]

I managed to fix the problem with
install_name_tool -id /path/to/mxnet/lib/libmxnet.so libmxnet.so

I still think this is an issue as I don't think it should be necessary to do this. However, this does seem to fix the problem, I'll investigate if I can make a change to the makefile to fix this.

[kalyc]

Thanks for updating the issue with a an associated fix
@bzcheeseman could you please close the issue?