Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem compiling CasADi on Mac OS X 10.6.7 Snow Leopard #201

Closed
casadibot opened this issue Oct 31, 2012 · 0 comments
Closed

Problem compiling CasADi on Mac OS X 10.6.7 Snow Leopard #201

casadibot opened this issue Oct 31, 2012 · 0 comments

Comments

@casadibot
Copy link
Member

This ticket reports problems I have hade with compiling CasADi on my MacBook Pro, in particular to get the Python interface working.

  1. I installed packages using macports:
  2. python26
  3. py26-ipython
  4. gcc45
  5. py26-numpy
  6. py26-scipy
  7. py26-wxpython
  8. py26-matplotlib
  9. py26-lxml
  10. apache-ant
  11. subversion
  12. swig
  13. cmake
  14. py26-cython don't work out of the box, see https://trac.macports.org/ticket/28682
    Check the log file for the failed python install, on my system located at:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py26-cython/main.log
    and copy the lines below "The correct checksum line may be…" into the port file:
    /opt/local/var/macports/sources/rsync.macports.org/release/ports/python/py26-cython/Portfile
    to replace the incorrect checksums.
  15. How to select a Python and gcc distribution
    http://superuser.com/questions/35256/how-can-i-change-the-default-python-version-on-snow-leopard
> sudo port select --set ipython ipython26
> sudo port select --set python python26
> sudo port select --set gcc gcc45
  1. Compilation of Ipopt 3.10.0
> ../configure --prefix=/Users/jakesson/packages/Ipopt/Ipopt-3.10.0-MA27-install --enable-loadable-library ADD_CXXFLAGS="-mmacosx-version-min=10.4 -fPIC -fexceptions -g" ADD_CFLAGS="-mmacosx-version-min=10.4 -fPIC  -fexceptions -g" ADD_FFLAGS="-mmacosx-version-min=10.4 -fPIC -g" CXX=g++-mp-4.5 CC=gcc-mp-4.5 F77=gfortran-mp-4.5
> make install
> make test

No errors reported in testing of Ipopt

  1. Compile CasADi

Use the following script to run cmake to set up the makefiles:


IPOPT=/Users/jakesson/packages/Ipopt/Ipopt-3.10.0-MA27-install \
SUNDIALS=/Users/jakesson/packages/Sundials/sundials-2.4.0-install-non-shared/ \
cmake ../ -DEXTRA_LIBRARIES:STRING=-lgfortran \
-DNUMPY_LIBS_1:FILEPATH="/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/multiarray.pyd" \
-DNUMPY_PATH:PATH="/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include/numpy/;/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include/
" \
-DNUMPY_PATH_DEP1:PATH="/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include/numpy/" \
-DPYTHON_EXECUTABLE:FILEPATH="/opt/local/bin/python2.6" \
-DPYTHON_LIBRARY:FILEPATH="/opt/local/lib/libpython2.6.dylib"

Then build and install CasADi:

> sudo make install_python
> cd examples
> make
> cd ../bin
> ./rocket_ipopt

The rocket_ipopt test works without problems - Ipopt is invoked and no errors are reported.

Now try a Python example:

ipython
Python 2.6.7 (r267:88850, Jul 28 2011, 18:02:40)
Type "copyright", "credits" or "license" for more information.

IPython 0.10.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: run vdp_collocation.py
Python(50482) malloc: *** error for object 0x7fff70714500: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
zsh: abort      ipython

The resulting trace reported is:

Process:         Python [68191]
Path:            /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python
Identifier:      Python
Version:         ??? (???)
Code Type:       X86-64 (Native)
Parent Process:  sh [68190]

Date/Time:       2011-07-29 08:18:26.075 +0200
OS Version:      Mac OS X 10.6.7 (10J4139)
Report Version:  6

Interval Since Last Report:          1394264 sec
Crashes Since Last Report:           138
Per-App Crashes Since Last Report:   44
Anonymous UUID:                      99540F56-9C7D-40E1-9043-61B6A1625D2D

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Application Specific Information:
abort() called

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   libSystem.B.dylib               0x00007fff839f55d6 __kill + 10
1   libSystem.B.dylib               0x00007fff83a95cde abort + 83
2   libSystem.B.dylib               0x00007fff839ad6b5 free + 128
3   libipopt.1.dylib                0x0000000116aa40e9 Ipopt::AdaptiveMuUpdate::RegisterOptions(Ipopt::SmartPtr<Ipopt::RegisteredOptions>) + 4441
4   ???                             0x00007fff5fbfb410 0 + 140734799787024

Thread 1:  Dispatch queue: com.apple.libdispatch-manager
0   libSystem.B.dylib               0x00007fff839c012a kevent + 10
1   libSystem.B.dylib               0x00007fff839c1ffd _dispatch_mgr_invoke + 154
2   libSystem.B.dylib               0x00007fff839c1cd4 _dispatch_queue_invoke + 185
3   libSystem.B.dylib               0x00007fff839c17fe _dispatch_worker_thread2 + 252
4   libSystem.B.dylib               0x00007fff839c1128 _pthread_wqthread + 353
5   libSystem.B.dylib               0x00007fff839c0fc5 start_wqthread + 13

Compiling IPOPT with gcc-4.4 gives a slightly more informative error message:

Process:         Python [94810]
Path:            /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python
Identifier:      Python
Version:         ??? (???)
Code Type:       X86-64 (Native)
Parent Process:  zsh [9305]

Date/Time:       2011-08-03 10:49:16.644 +0200
OS Version:      Mac OS X 10.6.7 (10J4139)
Report Version:  6

Interval Since Last Report:          1497533 sec
Crashes Since Last Report:           150
Per-App Crashes Since Last Report:   55
Anonymous UUID:                      99540F56-9C7D-40E1-9043-61B6A1625D2D

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Application Specific Information:
abort() called

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   libSystem.B.dylib               0x00007fff839f55d6 __kill + 10
1   libSystem.B.dylib               0x00007fff83a95cde abort + 83
2   libSystem.B.dylib               0x00007fff839ad6b5 free + 128
3   libipopt.1.dylib                0x0000000102ca96ac Ipopt::IpoptApplication::RegisterOptions(Ipopt::SmartPtr<Ipopt::RegisteredOptions>) + 5436 (basic_string.h:236)
4   libipopt.1.dylib                0x0000000102ca5ac8 Ipopt::RegisterOptions_Interfaces(Ipopt::SmartPtr<Ipopt::RegisteredOptions> const&) + 120 (IpSmartPtr.hpp:513)
5   libipopt.1.dylib                0x0000000102caa534 Ipopt::IpoptApplication::IpoptApplication(bool, bool) + 724 (IpIpoptApplication.cpp:1098)
6   _casadi.so                      0x00000001027d2eb4 CasADi::IpoptInternal::init() + 2276
7   _casadi.so                      0x00000001028727bf CasADi::SharedObject::init() + 31
8   _casadi.so                      0x000000010255bfd3 _wrap_SharedObject_init + 99
9   ???                             0x00000001051cc990 0 + 4380739984

Given that the error seems to be related to strings (basic_string.h) and the string problems I previously encountered in CasADi's XML import, I did some searching on string related problems and came up with the following posts:

http://stackoverflow.com/questions/2234557/c-using-getline-prints-pointer-being-freed-was-not-allocated-in-xcode
http://gcc.gnu.org/ml/gcc-bugs/2009-10/msg00807.html
http://lists.apple.com/archives/cocoa-dev/2009/Sep/msg01199.html
http://lists.macosforge.org/pipermail/macports-dev/2011-June/014877.html

However, this issue should have been fixed in the macports compilers:

https://trac.macports.org/ticket/22234

The problem may be that the Python-distribution installed by macports is compiled by the XCode compiler (gcc-4.2.1), which is compiled in a way that may cause problems, as described in the posts above. So, when CasADi and Ipopt are compiled using macport's gcc-4.5 or gcc-4.4, there may be an incompatibility between the system libraries of these compilers?

I checked the dependencies of _casadi.so to see if there may be conflicting libraries involved, and it seems that here is:

otool -L ~/svn_projects/CasADi/build/lib/_casadi.so
/Users/jakesson/svn_projects/CasADi/build/lib/_casadi.so:
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.0)
/usr/lib/libmpi_cxx.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libmpi.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libopen-rte.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libopen-pal.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/jakesson/packages/Ipopt/Ipopt-3.10.0-MA27-install/lib/libipopt.1.dylib (compatibility version 10.0.0, current version 10.0.0)
/Users/jakesson/packages/Ipopt/Ipopt-3.10.0-MA27-install/lib/libcoinhsl.1.dylib (compatibility version 5.0.0, current version 5.0.0)
/Users/jakesson/packages/Ipopt/Ipopt-3.10.0-MA27-install/lib/libcoinlapack.1.dylib (compatibility version 5.0.0, current version 5.0.0)
/Users/jakesson/packages/Ipopt/Ipopt-3.10.0-MA27-install/lib/libcoinblas.1.dylib (compatibility version 4.0.0, current version 4.0.0)
/opt/local/lib/gcc45/libgfortran.3.dylib (compatibility version 4.0.0, current version 4.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.10)
/Users/jakesson/packages/Sundials/sundials-2.4.0-install-non-shared/lib/libsundials_cvodes.2.dylib (compatibility version 3.0.0, current version 3.0.0)
/Users/jakesson/packages/Sundials/sundials-2.4.0-install-non-shared/lib/libsundials_idas.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/jakesson/packages/Sundials/sundials-2.4.0-install-non-shared/lib/libsundials_kinsol.1.dylib (compatibility version 2.0.0, current version 2.0.0)
/Users/jakesson/packages/Sundials/sundials-2.4.0-install-non-shared/lib/libsundials_nvecserial.0.dylib (compatibility version 1.0.0, current version 1.2.0)
/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib (compatibility version 1.0.0, current version 219.0.0)
/opt/local/lib/gcc45/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.14.0)
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 830.0.0)
/opt/local/lib/gcc45/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

Notice that libgcc_s.1.dylib is referenced both in the macport version of the compiler (/opt/local/lib) and by the XCode version (/opt/local/lib)

  1. I also tried to compile Python (2.6.7) from sources. In order to make this work, gcc-4.6 is needed, for the reasons described in http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44981, so I installed gcc46 using macports and then compiled Python using
> ../configure --prefix=/Users/jakesson/packages/python2.6/python/python2.6 --enable-shared CXX=g++-mp-4.6 CC=gcc-mp-4.6 > F77=gfortran-mp-4.6
> make
> make install

This installation, too, however, seems to be dependent on the XCode version of libgcc_s.1.dylib:

otool -L ~/packages/python2.6/python/python2.6/lib/libpython2.6.dylib
/Users/jakesson/packages/python2.6/python/python2.6/lib/libpython2.6.dylib:
/Users/jakesson/packages/python2.6/python/python2.6/lib/libpython2.6.dylib (compatibility version 2.6.0, current version 2.6.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 550.42.0)
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 830.0.0)
/opt/local/lib/gcc46/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.10)

In order to compile Numpy, the following patch needs to be applied:
https://trac.macports.org/browser/trunk/dports/python/py26-numpy/files/patch-system_info.py.diff?rev=62626.
After this change it works fine to install numpy in the newly compiled Python installation, and to then compile CasADi and install it in the same installation.

Unfortunately, the very same problem occurs with this Python distro as with the one installed by macports.

Created by None at: 2011-08-04T10:21:35
Last updated at: 2012-10-16T15:55:38


Comment 1 by jaeandersson at 2011-08-04T10:43:43

Yes, this is a really tricky issue. Maybe a good idea is to check with the Ipopt community, maybe someone else has experienced similar problems when using Ipopt from Python on Mac.


Comment 2 by jaeandersson at 2011-12-15T11:52:44

I agreed with Sebastien to sit down and try to get CasADi working in his Mac (which is using Mac OS X 10.7.8). We will do it in January.


Comment 3 by jaeandersson at 2011-12-15T11:54:49

Replying to [comment:2 joelandersson]:

I agreed with Sebastien to sit down and try to get CasADi working in his Mac (which is using Mac OS X 10.7.8). We will do it in January.

I mean 10.6.8.


Comment 4 by jaeandersson at 2012-04-11T15:02:09

We made an attempt to install CasADi on a Mac here and ended up with pretty much the same problems. Did you try any of the following:

  • Using the Python binaries for Mac instead of the Python you get from macports
  • Compiling CasADi and dependencies using gcc4.2

Comment 5 by None at 2012-04-11T15:13:51

Replying to [comment:4 joelandersson]:

We made an attempt to install CasADi on a Mac here and ended up with pretty much the same problems. Did you try any of the following:

\* Using the Python binaries for Mac instead of the Python you get from macports

Nope, I tried only with the macports distros.

\* Compiling CasADi and dependencies using gcc4.2

I tried this, but at least with that version of CasADi I got compilation errors in the generated SWIG code for the Python interface with gcc-4.2. Can you verify that this is still the case?


Comment 6 by jaeandersson at 2012-04-11T15:58:23

Replying to [comment:5 jakesson6]:

Replying to [comment:4 joelandersson]:

We made an attempt to install CasADi on a Mac here and ended up with pretty much the same problems. Did you try any of the following:

\* Using the Python binaries for Mac instead of the Python you get from macports

Nope, I tried only with the macports distros.

\* Compiling CasADi and dependencies using gcc4.2

I tried this, but at least with that version of CasADi I got compilation errors in the generated SWIG code for the Python interface with gcc-4.2. Can you verify that this is still the case?

I didn't get that far. I didn't manage to compile the dependencies (Ipopt) using gcc4.2, since gfortran was not available, and it wasn't possible to install it either. I will make an attempt with gcc4.4. Starting from gcc4.5, the names of gcc in macports changed, so maybe that is an indicator that other things changed as well.


Comment 7 by jaeandersson at 2012-04-11T17:17:40

With gcc4.4 CasADi's Python interface appears to work if Ipopt is not used. As soon as an Ipopt instance is created, Python crashes. I will try to recompile Ipopt with only static libraries, as I suspect that the DLL loader in Ipopt might be what is causing the problems.


Comment 8 by jaeandersson at 2012-04-11T17:51:37

Good news. It works to generate the Python interface if Ipopt has been compiled with disable-shared. Some unrelated problems remain though, as some examples and the test suite fails. Here are the instructions we used:

  • Install Macports from the website website
  • Update macports using sudo port -v selfupdate
  • Install dependencies (takes a long time!):
  • sudo port install python27 py27-ipython gcc44 py27-numpy py27-scipy py27-matplotlib subversion swig cmake py27-spyder
  • Select Python and GCC installation:
  • sudo port select --set ipython ipython27
  • sudo port select --set python python27
  • sudo port select --set gcc mp-gcc44
  • Copy the following lines to ~/.profile in order to get Spyder working
  • export LANG=en_US.UTF-8
  • export LC_CTYPE=en_US.UTF-8
  • Download and install IPOPT from SVN:
  • svn co https://projects.coin-or.org/svn/Ipopt/stable/3.10 CoinIpopt
  • ../configure --prefix=/usr/local --disable-shared
  • Download and install Sundials
  • Download and install CasADi from SVN
  • svn co https://casadi.svn.sourceforge.net/svnroot/casadi/trunk casadi
  • mkdir build; cd build; cmake ..; make all python; sudo make install_python

Comment 9 by jaeandersson at 2012-04-11T17:54:03

It probably works also with a newer GCC, I will try gcc46.


Comment 10 by jaeandersson at 2012-05-15T15:13:02

This should be resolved until the next release.


Comment 11 by jaeandersson at 2012-06-19T09:38:36

There appears to be some problems with the xcode compiler. See:
http://stackoverflow.com/questions/1854113/ostringstream-problem-with-int-in-c and http://stackoverflow.com/questions/2478648/c-stringstream-int-to-string-but-returns-null.

A workaround solution seems to be to switch xcode from debug mode to release mode.


Comment 12 by jaeandersson at 2012-06-19T09:44:23

Some problems related to the error handling was detected, see #395.


Comment 13 by jaeandersson at 2012-06-19T10:17:14

It should be possible to use cmake to generate XCode projects. I tried this but ran into the following issue: http://www.itk.org/Bug/view.php?id=9739


Comment 14 by jaeandersson at 2012-06-19T12:09:36

Tried gcc47, but it fails as follows:

In [1]: from casadi import *

In [2]: a = ssym("a",2)
Python(30232,0x7fff70daecc0) malloc: *** error for object 0x1057dc7e0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap

Comment 15 by jaeandersson at 2012-06-19T17:07:17

gcc46 fails just like gcc47. gcc45 fails like gcc44.


Comment 16 by jaeandersson at 2012-06-20T07:58:28

Came across the following instructions on how to change the compiler in Macports: http://trac.macports.org/wiki/UsingTheRightCompiler. You really don't want to compile modern numerical software with gcc4.2.


Comment 17 by jaeandersson at 2012-06-28T00:05:35

Great news! Finally managed to install CasADi (including Python interface, Sundials and Ipopt) on a Mac OS X Lion (10.7).

The way that finally worked was

  • Uninstall macports completely
  • Install gfortran from http://hpc.sourceforge.net/
  • Updated Xcode to latest version (4.3) using App Store and installed "Command Line Tools" from Xcode -> Preferences -> Downloads
  • Install CMake from website
  • Compile Ipopt from sources, setting only prefix to /usr/local/
  • Compile Sundials from sources (with -fPIC)
  • Installed SWIG from sources (macports version was corrupted). Needed to configure with --without-pcrc
  • Install easy_install
  • Make sure Apple's default Python installation works (in our case it was Python 2.7)
  • Install PySide from website (needed for spyder) which in turn needed the installation of QT4
  • Install numpy, scipy, ipython, readline, sphinx (needed for spyder) and spyder from easy_install
  • Download CasADi from SVN
  • Remove ${CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES} in the file <CASADI_ROOT>/cmake_modules/FindIPOPT.cmake
  • Configured and installed CasADi with default options (mkdir build, cd build, cmake .., make, make python, sudo make install_python. After uploading Xcode, clang was chosen as the default C/C++ compiler. If this is not automatic, export CC=/usr/bin/clang, export CXX=/usr/bin/clang++ will probabably work also. Compiling CasADi with Clang is fast but gives quite a few warnings. No errors though, except for an irrelevant one in the documentation subdirectory.

All tests in CasADi's testsuite passed after the installation. The only thing that didn't work was matplotlib. It appears that the matplotlib that you get from easy_install is corrupted and the matplotlib binaries from the website only work with the Python installation from the [www.python.org], and not Apple's default one, which was the only one we got working together with CasADi. Installing matplotlib from sources should work, but we didn't have time to test that.

I will try this recipe on Sebastien's Mac (with Snow Leopard) and then put together installation instructions. Since we tried quite a lot of different variants and installed and uninstalled several software packages, it is likely that I forgot something above.


Comment 18 by jaeandersson at 2012-06-28T00:08:19

Ah, also note that the gcc version used to compile Ipopt and SWIG was the default one (4.2). Apple does not appear interested in updating to a newer gcc version and instead concentrate on clang. Therefore it also makes sense to use clang to compile CasADi.


Comment 19 by jaeandersson at 2012-06-28T01:12:36

Created a ticket for better clang-support, #402.


Comment 20 by jaeandersson at 2012-07-05T11:06:36

I've updated the installation instructions and will close the ticket since it now basically works.


Comment 21 by jaeandersson at 2012-07-26T16:55:39

I was a bit too fast in closing the ticket. Though it appears to work on 10.7, 10.6 still appears to have problems.


Comment 22 by jaeandersson at 2012-09-12T15:01:57

Got everything working on OS X 10.6, after manually changing the ipopt linker flags, see #447.


Comment 23 by jaeandersson at 2012-09-29T12:48:31

After major changes to CasADi's build system and several successful builds on Macs, I consider this ticket solved.


Comment 24 by None at 2012-10-16T15:55:38

Hi, I am currently a Mac OS X Lion user and I have tried both installing processes described in the previous response and there:https://sourceforge.net/apps/trac/casadi/wiki/InstallationMac.
Unfortunately, none of these methods helps me to get a suitable installation. I have always a problem with Ipopt which is not recognized by the system even if the installation seems to be done successfully.
Could you kindly post a described and suitable installation process so that we could install Casadi simply?
Thank you in advance for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants