MacPort

MeqTrees on the Mac

After a lot of blood, sweat and tears, Mike Sipior has managed to get MeqTrees to build and run on OS X. The necessary patches currently (01/03/2009) reside on a branch of the repository, which you can check out as follows:

• svn co svn://lofar9.astron.nl/var/svn/repos/branches/macport/Timba Then, you will enter the sheer hell of getting together the dependency chain on the Mac. Mike's summary of his travails follows:

Gentlemen,

As promised, a more detailed primer on the vagaries of Meq-building on OS X. First, a few general caveats:

1. Binaries are of the type "Mach-O", not the ELF format of Linux and Solaris. To indicate this, beginning in 10.4 (or thereabouts), OS X designates shared libraries with the ".dylib" suffix, instead of ".so". The system linker knows what is going on, but some configure scripts will get confused when trying to verify whether a given library is present, as they are looking for a library with the wrong filename suffix. Judicious use of symbolic links pointing to the correct .dylib files circumvents the problem.

2. OS X is BSD-based, whereas Linux principally descends from System V. Certain commands behave very differently between these two UNIX cousins. In particular, don't assume that you have the GNU version of a given tool (I'm looking at you, sed...more on that later).

3. There are three principal means of installing third-party open-source software on OS X machines. First is MacPorts, an OS X version of the BSD ports system, which is source-based. Second is Fink, more similar to rpm or dpkg from Linuxland, and centered around pre-built binaries. Third of course is to build each package from source using the provided system compilers. Why this is important: if you choose either of the first two methods for installing the prerequisites for MeqTrees, you need to keep in mind that there will be multiple, parallel versions of gcc, g++ and python on your system, and you must be very careful to ensure that you are using a consistent build environment, else you will get some very interesting linking errors (and generally only after the bulk of the software has already been installed...)

I personally used MacPorts to grab some basic dependencies, and was bitten by this problem on a number of occasions. So my recommendation for building a proper installation package for MeqTrees is to simply use the default gcc, g++ and python (2.5) provided by XCode: much more work tracking a million dependencies, but much less agonising over why your build dies with linker and "incompatible version" errors...

4. The default filesystem on OS X is case-insensitive. I hope whoever thought that was a clever idea is enjoying their new flat in the City of Dis. In particular, certain files in MeqTrees include "Time.h". You can imagine the fun that provides if you can't differentiate between that and "time.h".

So, I grabbed most of the dependencies listed on the Timba wiki from MacPorts. This includes qdbm and Qt3, Blitz, fftw and NumPy. Extra things I had to get:

--- An older version of automake (1.9.6), which the MeqTrees configure system requires, as more modern versions cause the build to choke and die. This really needs to be fixed, btw, unless the hope is that Linux distributions (and everybody else) will keep providing ancient versions of autoconf and friends for ever and ever. I also installed a new autoconf (2.61) and libtool (2.2.6) to go with it in /usr/local.

--- GNU sed. The MeqTrees build scripts assume that a GNU version of the stream editor is in the path. That turned out not to be the case, and I built version 4.1.5 and also stuck it in /usr/local.

--- A recent version of scons (1.2). The version currently provided by MacPorts does not work with the latest release (0.4.0) of casacore, much to my great sadness.

Notes on some problematic dependencies:

--- Qt3. The version designed for use with X11 builds fine using MacPorts. The native Aqua version, however, will not build on any system more recent than OS X 10.4. So, we don't get the pretty Mac widgets until you move to Qt4.

--- PyQt. What a royal pain in the ass. I grabbed QScintilla-1.71, sip-4.7.9 and PyQt-mac-gpl-3.17.6 off the Riverbank Computing website. QScintilla built without incident (be sure that QTDIR is specified before running qmake, however). sip also gave no trouble. PyQt itself required changing one line in each of two files. Here's a patch:

--- qt/sipqtQCursor.cpp 2009-02-06 14:34:59.000000000 +0100 +++ qt/sipqtQCursor.osx.cpp 2009-02-06 11:15:18.000000000 +0100 @@ -179,7 +179,7 @@ static PyObject *meth_QCursor_handle(PyO

        sipRes = sipCpp->handle();

•        return sipConvertFromVoidPtr(sipRes);
  

•        return sipConvertFromVoidPtr((void *)sipRes); /*MSS*/
    }
  

  }

--- qt/sipqtQThread.cpp 2009-02-06 14:34:39.000000000 +0100 +++ qt/sipqtQThread.osx.cpp 2009-02-06 11:15:31.000000000 +0100 @@ -140,7 +140,7 @@ static PyObject *meth_QThread_currentThr

        sipRes = QThread::currentThread();

•        return sipConvertFromVoidPtr(sipRes);
  

•        return sipConvertFromVoidPtr((void *)sipRes); /*MSS*/
    }
  

  }

No point running that through patch: just add the explicit cast to void* in the two indicated locations. Be sure to do this after running "python configure.py" though, as those files are autogenerated (sigh).

--- Qwt4. I just used the Qwt-4.2.0 source provided in the PyQwt-4.2.3 tarball. Again, ensure that QTDIR is set before running qmake.

--- PyQwt4. The SIP directive "template %MappedType QMemArray" must be commented out in the file sip/qwt4qt3/common/QwtMemArray.sip.

--- Boost. The MacPorts version must be explicitly built with python support, with "port install boost +python25" Make sure that it's enabled if you're building from source (and using the correct version of python!)

--- casacore. Well, this is infinitely easier to build than the slagheap of AIPS++ ever was. I installed wcslib in /usr/local, and grabbed the atlas package from MacPorts to get LAPACK and BLAS. Here's the batchbuild line we used:

sudo ./batchbuild.py enable_hdf5=false prefix=/usr/local wcsroot=/usr/local cfitsioroot=/opt/local/ lapackroot=/opt/local blasroot=/opt/local FORTRAN=gfortran extralinkflags="-L/usr/lib/gcc/i686-apple-darwin9/4.2.1" f2clib=gfortran

--- pyrap. More recent versions of NumPy (1.2.1, at the very least) have changed the name of the config.h file to numpyconfig.h, thus breaking the detection from the pyrap configure. Using MacPorts 1.7, python 2.5 and NumPy 1.2.1, the problem is fixed by making a link:

cd /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/core/include/numpy ln -s numpyconfig.h config.h

and the configuration command line:

sudo ./batchbuild.py --casacoreroot=/usr/local --boostroot=/opt/local/ --numpyincdir=/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/core/include/

You'll also need to make a link from the actual boost_python library so that the pyrap configuration can find it:

cd /opt/local/lib sudo ln -s libboost_python-mt.dylib libboost_python.dylib

Another amusing problem with pyrap and MacPorts: there is a bug in the MacPorts version of python 2.5 which will prevent pyrap from linking properly at run time, giving all sorts of "incompatible version" errors. We never solved this problem properly, but it's not an issue if you use the Apple-supplied compilers for everything. Python 2.6 works in principle, but there are problems with NumPy under 2.6 at the moment, closing that option off.

After all this, MeqTrees can be built using the instructions on the wiki. There remain two problems with the OS X version of MeqTrees: pyrap doesn't work, and casarest doesn't either. The latter has apparently just been fixed under Linux, so it's probably worth a svn up to see if it's working under OS X as well. pyrap should also work properly if it (and the rest of MeqTrees and its python dependencies) are built using the Apple-supplied python 2.5. We just haven't had time to test this hypothesis.

So, that was a long message. Please let me know if there is anything that needs further clarification. I think the packaging can best proceed by using "hdiutil create" to make a disk image which can then have MeqTrees and every dependency installed into it. OS X mounts all disk images under /Volume, so symbolic link paths shouldn't be an issue from one machine to another.

By the way, here are the relevant lines from the variants file I used to build MeqTrees on my machine:

gnu4.compiler.conf: CXX=ccache\ g++ --with-cppflags="-m32 -Wno-deprecated -Wno-u nused-parameter" --enable-shared --with-threads --with-sse --with-ldflags="--ena ble-new-dtags" --with-blitz=/opt/local --with-cfitsio=/opt/local --with-fftw3=/o pt/local --with-cxxflags="-DDARWIN -I/opt/local/include" --with-cflags="-DDARWIN -I/opt/local/include" --with-ldflags="-L/opt/local/lib" --with-lapack=/usr/lib gnu4.compiler.aipspp.var: --with-casacore=/usr/local --with-wcs=/usr/local

python.var: --with-python-version=2.5 --with-python=/opt/local/include/python2. 5 --with-python-libdir=/opt/local/lib

Cheers,

M.