Skip to content

Commit

Permalink
use LLVM for mac-os builds to avoid gcc openmp issue
Browse files Browse the repository at this point in the history
with gcc, we kept getting an error:
ImportError: dlopen(
    /private/...lib.macosx-10.9-x86_64-3.6/pycraf/
    pathprof/cyprop.cpython-36m-darwin.so, 2
    ): Symbol not found: _GOMP_parallel
  • Loading branch information
bwinkel committed Nov 21, 2017
1 parent 58c6512 commit dce2123
Show file tree
Hide file tree
Showing 2 changed files with 99 additions and 40 deletions.
52 changes: 37 additions & 15 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,26 +87,48 @@ matrix:
- os: osx
osx_image: xcode8.3
env:
- CC='gcc-7'
- CXX='g++-7'
- CPP='g++-7'
- LD='gcc-7'
- CC='/usr/local/opt/llvm/bin/clang'
- LDFLAGS='-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib'
- CPPFLAGS='-I/usr/local/opt/llvm/include'
- COMPILER='llvm'
# compiler: gcc

# Try MacOS X
- os: osx
osx_image: xcode7.2
env:
- COMPILER='conda'
# compiler: gcc

# Try MacOS X
# produces ImportError: dlopen(/private/.../pycraf/pathprof/cyprop.cpython-36m-darwin.so, 2): Symbol not found: _GOMP_parallel
# - os: osx
# osx_image: xcode8.3
# env:
# - CC='gcc-7'
# - CXX='g++-7'
# - CPP='g++-7'
# - LD='gcc-7'
# - COMPILER='gcc'

# We have to make the MacOS release based on rather old numpy version
# otherwise users will keep getting "incompatible API" errors
# (For more information on ABI problems with numpy see:
# https://stackoverflow.com/questions/17709641/valueerror-numpy-dtype-has-the-wrong-size-try-recompiling/18369312)
- os: osx
osx_image: xcode8.3
env:
- CC='gcc-7'
- CXX='g++-7'
- CPP='g++-7'
- LD='gcc-7'
# - CC='gcc-7'
# - CXX='g++-7'
# - CPP='g++-7'
# - LD='gcc-7'
# - COMPILER='gcc'
- DO_MAC_DIST=YES
- NUMPY_VERSION=1.11
# compiler: gcc
- CC='/usr/local/opt/llvm/bin/clang'
- LDFLAGS='-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib'
- CPPFLAGS='-I/usr/local/opt/llvm/include'
- COMPILER='llvm'

# Do a coverage test.
- os: linux
Expand Down Expand Up @@ -148,12 +170,10 @@ matrix:

before_install:
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew update; fi
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install gcc; brew link --overwrite gcc; fi
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then gcc --version; fi
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then gcc-7 --version; fi
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then gfortran --version; fi
# - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then echo "LD_LIBRARY_PATH" $LD_LIBRARY_PATH ; fi
# - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then echo "DYLD_LIBRARY_PATH" $DYLD_LIBRARY_PATH ; fi
- if [[ "$TRAVIS_OS_NAME" == "osx" ]] && [[ $COMPILER == "gcc" ]]; then brew install gcc; brew link --overwrite gcc; fi
- if [[ "$TRAVIS_OS_NAME" == "osx" ]] && [[ $COMPILER == "gcc" ]]; then gcc --version; fi
- if [[ "$TRAVIS_OS_NAME" == "osx" ]] && [[ $COMPILER == "gcc" ]]; then gcc-7 --version; fi
- if [[ "$TRAVIS_OS_NAME" == "osx" ]] && [[ $COMPILER == "llvm" ]]; then brew install llvm; fi

install:

Expand All @@ -173,6 +193,8 @@ install:
- git clone --depth 1 git://github.com/astropy/ci-helpers.git
- source ci-helpers/travis/setup_conda.sh

- if [[ "$TRAVIS_OS_NAME" == "osx" ]] && [[ $COMPILER == "conda" ]]; then conda install -c conda-forge openmp; fi

# As described above, using ci-helpers, you should be able to set up an
# environment with dependencies installed using conda and pip, but in some
# cases this may not provide enough flexibility in how to install a
Expand Down
87 changes: 62 additions & 25 deletions docs/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,8 @@ To install pycraf with `pip <http://www.pip-installer.org/en/latest/>`__, simply
.. note::

You may need a C compiler (``gcc``) to be installed for the installation
to succeed. Since `pycraf` needs OpenMP, ``clang`` is currently not
supported.
You may need a C compiler (``gcc``) with OpenMP support to be installed
for the installation to succeed.

.. note::

Expand All @@ -78,7 +77,7 @@ To install pycraf with `pip <http://www.pip-installer.org/en/latest/>`__, simply
.. _source_install:

Installation from source
~~~~~~~~~~~~~~~~~~~~~~~~
------------------------

There are two options, if you want to build pycraf from sources. Either, you
install the tar-ball (`*.tar.gz` file) from `PyPI
Expand All @@ -100,11 +99,20 @@ Again, consider the ``--user`` option or even better use a python distribution
such as `Anaconda <https://www.continuum.io/downloads>`_ to avoid messing up
the system-wide Python installation.

.. note::

If you use `Anaconda` and want to install the `sgp4` and `pyproj`
packages, you'll have to use a different channel

.. code-block:: bash
conda install -c conda-forge sgp4 pyproj
.. _windows_install:

Installation on Windows
~~~~~~~~~~~~~~~~~~~~~~~
-----------------------

Note that for Windows machines we provide a binary wheel (Python 3.5+ only).
However, the `pyproj <https://pypi.python.org/pypi/pyproj>`_ package is a
Expand Down Expand Up @@ -141,43 +149,72 @@ If you're using `Anaconda <https://www.continuum.io/downloads>`__
.. _macos_install:

Installation on MacOS
~~~~~~~~~~~~~~~~~~~~~
---------------------

Even though we also provide a binary wheel for MacOS, it seems you'll still
have to install a C-compiler, in particular *gcc-6*. The clang compiler
seems not to support OpenMP in a way that is needed by pycraf:
Installation on MacOS can be a bit tricky, because the standard C compiler
does not support OpenMP. We provide wheels on PyPI, such that you can

.. code-block:: bash
brew install gcc6
brew link --overwrite gcc@6
pip install pycraf
however, you need to have the LLVM C compiler (see below), otherwise you'll
likely get an error message that a library (such as "libgomp") is not
found, when you import pycraf in Python.

Also, if you want to install from source, you must have a C compiler. There
are basically two options, using LLVM or the gcc suite.

Then proceed as usual with
LLVM
~~~~

.. code-block:: bash
# if on Anaconda, install pyproj the easy way:
conda install -c conda-forge pyproj
brew update
brew install llvm
# then
pip install pycraf
export CC="/usr/local/opt/llvm/bin/clang"
export LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"
export CPPFLAGS="-I/usr/local/opt/llvm/include"
Then follow the instructions in :ref:`source_install`.

gcc
~~~

.. code-block:: bash
brew install gcc6 # or gcc7
brew link --overwrite gcc@6 # or gcc@7
Then follow the instructions in :ref:`source_install`.

.. note::

The MacOS wheel, which we provide on PyPI (for pip installation)
was built using LLVM. So it may happen that you run into binary
incompatibilities if you use a different compiler suite on your computer.
In such cases it may be necessary to build pycraf from source using
your own compiler. Sometimes even different compiler versions
(e.g. gcc 6.3 instead of gcc 6.4) can lead to problems.
Please write a ticket, if you run into trouble.

.. note::

Again, if you're on Anaconda, things get (often) much simpler:

.. code-block:: bash
.. warning::
conda install -c conda-forge openmp
Currently, we only provide MacOS wheels for *gcc-6* (version 6.4).
It seems that if you have a different version of *gcc-6* on your machine
importing pycraf will fail with some ``libgomp`` error, which is probably
due to binary inconsistencies between the various *gcc* versions.
Please, consider updating to the 6.4-version of *gcc*, or install from
source with your own *gcc* compiler.
.. _testing_installed_pycraf:

Testing an installed pycraf
----------------------------

The easiest way to test your installed version of pycraf is running
correctly is to use the `~pycraf.test()` function::
The easiest way to test if your installed version of pycraf is running
correctly, is to use the `~pycraf.test()` function::

import pycraf
pycraf.test()
Expand Down

0 comments on commit dce2123

Please sign in to comment.