Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Conda packaging #328

Merged
merged 15 commits into from
Feb 13, 2018
15 changes: 15 additions & 0 deletions .conda/bin/build
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env bash

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

export BRAINIAK_HOME=$DIR/../../

# See run-tests.sh
export MKL_THREADING_LAYER=GNU

if [ -z ${CONDA_HOME+dummy} ]
then
CONDA_HOME="$HOME"/miniconda3
fi

"$CONDA_HOME"/bin/conda build "$DIR"/..
6 changes: 6 additions & 0 deletions .conda/bin/docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

# Need to run apt update && apt install -y build-essential curl git
docker run --rm -w /src -v $DIR/../..:/src -ti ubuntu:trusty /bin/bash
28 changes: 28 additions & 0 deletions .conda/bin/install-miniconda
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/usr/bin/env bash

# Install Miniconda.
echo ""
echo "Installing a fresh version of Miniconda."
MINICONDA_URL="https://repo.continuum.io/miniconda"

MINICONDA_FILE="Miniconda3-latest-Linux-x86_64.sh"
if [[ "$OSTYPE" == "darwin"* ]]; then
MINICONDA_FILE="Miniconda3-latest-MacOSX-x86_64.sh"
fi

curl -L "${MINICONDA_URL}/${MINICONDA_FILE}" -o ~/${MINICONDA_FILE}

# Update an existing miniconda installation, if it exists
bash ~/$MINICONDA_FILE -b -u

# Clean up
rm -f ~/${MINICONDA_FILE}

# Configure conda.
echo ""
echo "Configuring conda."
source ~/miniconda3/bin/activate root
conda config --add channels conda-forge --add channels defaults
conda config --set show_channel_urls true
conda install --yes --quiet conda-build
source deactivate
15 changes: 15 additions & 0 deletions .conda/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash

if [[ "$OSTYPE" == "darwin"* ]]; then
export MACOSX_DEPLOYMENT_TARGET=10.9
export CC=$(which clang)
export CXX=$(which clang++)
fi

echo $PREFIX

# Install pymanopt via pip because there isn't a conda package
$PYTHON -m pip install pymanopt

# NOTE: This is the recommended way to install packages
$PYTHON setup.py install --single-version-externally-managed --record=record.txt
67 changes: 67 additions & 0 deletions .conda/meta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
{% set conda_package_nonexistent = (
"pymanopt",
) %}
{% set data = load_setup_py_data() %}

package:
# Repeating name because of the following issue:
# https://github.com/conda/conda-build/issues/2475
name: brainiak
version: {{ GIT_DESCRIBE_TAG[1:] }}

about:
home: {{ data.get('url') }}
license: {{ data.get('license') }}
summary: {{ data.get('description') }}

source:
path: ../

build:
number: {{ GIT_DESCRIBE_NUMBER }}
script_env:

# We need the source directory because conda removes source / working
# directories by the time we reach the test phase. We can optionally pass in
# the --keep-old-work flag later on to use the $PREFIX/work directory
- BRAINIAK_HOME
- MKL_THREADING_LAYER

requirements:
build:
- {{ compiler('cxx') }}

host:
- python
- setuptools
- pip
- openmp
{% for req in data.get('setup_requires', [])
if req not in conda_package_nonexistent -%}
- {{req}}
{% endfor %}
{% for req in data.get('install_requires', [])
if req not in conda_package_nonexistent -%}
- {{req}}
{% endfor %}

run:
- python
- openmp
{% for req in data.get('install_requires', [])
if req not in conda_package_nonexistent -%}
- {{req}}
{% endfor %}

test:
commands:
- find $BRAINIAK_HOME/tests | grep pycache | xargs rm -rf
- mpiexec -n 2 pytest $BRAINIAK_HOME

# Known issue: https://github.com/travis-ci/travis-ci/issues/4704#issuecomment-348435959
- python -c 'import os,sys,fcntl; flags = fcntl.fcntl(sys.stdout, fcntl.F_GETFL); fcntl.fcntl(sys.stdout, fcntl.F_SETFL, flags&~os.O_NONBLOCK);'
- conda inspect linkages -p $PREFIX brainiak # [not win]
- conda inspect objects -p $PREFIX brainiak # [osx]
requires:
- pytest
- numdifftools
11 changes: 11 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,17 @@ matrix:
env: *macos_env
before_install: *macos_before_install

# Build and test conda packages
- os: linux
dist: trusty
install: ./.conda/bin/install-miniconda
script: ./.conda/bin/build

- os: osx
osx_image: xcode8
install: ./.conda/bin/install-miniconda
script: ./.conda/bin/build

install:
- python3 -m pip install -U pip codecov

Expand Down
30 changes: 29 additions & 1 deletion CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,22 @@ Making a release
This information is only of interest to the core contributors who have the
right to make releases.

Before making a release, ensure that:

1. The following environment variables are set::

export $CONDA_HOME=/path/to/miniconda3

2. The following Conda channels are enabled::

conda config --add channels conda-forge --add channels defaults

3. The following Conda packages are installed::

conda install conda-build anaconda-client

To make a release:

1. Choose a release number, ``v``. We follow `Semantic Versioning
<http://semver.org>`_, although we omit the patch number when it is 0.

Expand All @@ -300,7 +316,7 @@ right to make releases.

git tag v<v>

4. Create and test the distribution package::
4. Create and test the source distribution package::

python3 setup.py sdist
tar -xf dist/brainiak-<v>.tar.gz
Expand All @@ -313,3 +329,15 @@ right to make releases.

git tag --push
twine upload dist/brainiak-<v>.tar.gz


7. Create and test Conda packages (repeat command for all OSes and Python
versions); requires the ``conda-build`` Conda package::

.conda/bin/build

8. Upload the built Conda package (repeat command for all OSes and Python
versions); requires the ``anaconda-client`` Conda package::

anaconda upload -u brainiak \
$CONDA_HOME/conda-bld/<OS>/brainiak-<v>-<python_version>.tar.bz2
92 changes: 9 additions & 83 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,93 +23,19 @@ To reduce verbosity, we may refer to the Brain Imaging Analysis Kit using the
package name), we use ``brainiak``.


Requirements
============

We support Linux and MacOS with Python version 3.4 or higher. Most of the
dependencies will be installed automatically. However, a few need to be
installed manually.


Linux
-----

Install the following packages (Ubuntu 16.04 is used in these instructions)::

apt install build-essential libgomp1 libmpich-dev mpich python3-dev \
python3-pip python3-venv

Install updated version of the following Python packages::

python3 -m pip install --user -U pip

Note the ``--user`` flag, which instructs Pip to not overwrite system
files. You must add ``$HOME/.local/bin`` to your ``$PATH`` to be able to run
the updated Pip, e.g., by adding the following line to ``$HOME/.profile``
and launching a new login shell (e.g., logout or execute ``bash -l``)::

PATH="$HOME/.local/bin:$PATH"


MacOS
-----

Install the Xcode Command Line Tools::

xcode-select --install

Install ``brew`` from https://brew.sh. If you already have ``brew``, examine
the output of the following command to make sure it is working::

brew doctor

Then install the following::

brew install llvm cmake mpich python3

Note that if your MacOS does not have ``gcc`` (a prerequisite of ``mpich``),
it will take a long time (perhaps over an hour) to install ``gcc`` from source for you.

You must instruct programs to use this ``clang`` version at ``/usr/local/opt/llvm/bin``.
One way to do this, which
works for most programs, is setting the ``CC`` environment variable. You can
add the following lines to ``$HOME/.profile`` (or ``$HOME/.bash_profile``, if
you have one). For them to take effect, you must logout or launch a new login
shell, e.g., ``bash -l``::

export CC=/usr/local/opt/llvm/bin/clang
export CXX=/usr/local/opt/llvm/bin/clang++

In addition, you also need to specify the directories that the newly installed `clang`
will seek for compiling and linking::

export LDFLAGS="-L/usr/local/opt/llvm/lib "\
"-Wl,-rpath,/usr/local/opt/llvm/lib $LDFLAGS"
export CPPFLAGS="-I/usr/local/opt/llvm/include $CPPFLAGS"

Install updated versions of the following Python packages::

python3 -m pip install -U pip


Installing and upgrading
========================
Quickstart
==========

The Brain Imaging Analysis Kit is available on PyPI. You can install it (or
upgrade to the latest version) using the following command::
If you have `Conda <conda.io>`_::

python3 -m pip install -U brainiak
conda install -c brainiak -c defaults -c conda-forge brainiak

Note that you may see a ``Failed building wheel for brainiak`` message (`issue
#61`_). Installation will proceed despite this failure. You can safely ignore it
as long as you see ``Successfully installed`` at the end.
Otherwise, or if you want to compile from source, install the requirements (see
`docs/installation`) and then install from PyPI::

.. _issue #61:
https://github.com/brainiak/brainiak/issues/61
python3 -m pip install brainiak

Until we reach version 1.0, we will only support the latest released version.
Therefore, if you have a problem with an older version, please upgrade to the
latest version before creating an issue on GitHub.
Note that we do not support Windows.


Docker
Expand Down Expand Up @@ -179,4 +105,4 @@ http://brainiak.org." Additionally, if you use `RRIDs
<https://scicrunch.org/resolver>`_ to identify resources, please mention
BrainIAK as "Brain Imaging Analysis Kit, RRID:SCR_014824". Finally, please cite
the publications referenced in the documentation of the BrainIAK modules you
use, e.g., `SRM <http://pythonhosted.org/brainiak/brainiak.funcalign.html>`_.
use, e.g., `SRM <http://brainiak.org/docs/brainiak.funcalign.html>`_.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Table of contents

.. toctree::

installation
release_notes
api
contributing
Expand Down
Loading