Skip to content

Commit

Permalink
Some extra documentation on building wheels on OSX (#2415)
Browse files Browse the repository at this point in the history
  • Loading branch information
heerener committed Jul 5, 2023
1 parent 8082e1c commit 65efbdd
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 11 deletions.
39 changes: 29 additions & 10 deletions docs/install/python_wheels.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Current NEURON Linux image is based on `manylinux2014`.
### Setting up Docker

[Docker](https://en.wikipedia.org/wiki/Docker_(software)) is required for building Linux wheels.
You can find instructions on how to setup Docker on Linux [here](https://docs.docker.com/engine/install/).
You can find instructions on how to setup Docker on Linux [here](https://docs.docker.com/engine/install/).


### NEURON Docker Image Workflow
Expand All @@ -31,7 +31,7 @@ All wheels built on Azure are:
* `neuron-x.y.z` -> when the pipeline is manually triggered for release `x.y.z`
* Stored as `Azure artifacts` in the Azure pipeline for every run.

Refer to the following image for the NEURON Docker Image workflow:
Refer to the following image for the NEURON Docker Image workflow:
![](images/docker-workflow.png)


Expand All @@ -43,7 +43,7 @@ cd nrn/packaging/python
docker build -t neuronsimulator/neuron_wheel:<tag> .
```
where `<tag>` is:
* `latest-x86_64` or `latest-aarch64` for official publishing on respective platforms. For `master`, we are using `latest-gcc9-x86_64` and `latest-gcc9-aarch64` (see [Use GCC9 for building wheels #1971](https://github.com/neuronsimulator/nrn/pull/1971)).
* `latest-x86_64` or `latest-aarch64` for official publishing on respective platforms. For `master`, we are using `latest-gcc9-x86_64` and `latest-gcc9-aarch64` (see [Use GCC9 for building wheels #1971](https://github.com/neuronsimulator/nrn/pull/1971)).
* `feature-name` for updates (for local testing or for PR testing purposes where you can temporarily publish the tag on DockerHub and tweak Azure CI pipelines to use it - refer to
`Job: 'ManyLinuxWheels'` in [azure-pipelines.yml](../../azure-pipelines.yml) )

Expand Down Expand Up @@ -80,7 +80,7 @@ We can conveniently mount the local NEURON repository inside docker, by using th
docker run -v $PWD/nrn:/root/nrn -w /root/nrn -it neuronsimulator/neuron_wheel bash
```
where `$PWD/nrn` is a NEURON repository on the host machine that ends up mounted at `/root/nrn`.
This is how you can test your NEURON updates inside the NEURON Docker image.
This is how you can test your NEURON updates inside the NEURON Docker image.
Note that `-w` sets the working directory inside the container.

### MPI support
Expand All @@ -105,27 +105,46 @@ In order to have the wheels working on multiple macOS target versions, special c


Taking Azure macOS `x86_64` wheels for example, `readline` was built with `MACOSX_DEPLOYMENT_TARGET=10.9` and stored as secure file on Azure.
For `arm64` we need to set `MACOSX_DEPLOYMENT_TARGET=11.0`. The wheels currently need to be built manually, using `universal2` Python installers.
For `arm64` we need to set `MACOSX_DEPLOYMENT_TARGET=11.0`. The wheels currently need to be built manually, using `universal2` Python installers.
For upcoming `universal2` wheels (targeting both `x86_64` and `arm64`) we will consider leveling everything to `MACOSX_DEPLOYMENT_TARGET=11.0`.


You can use [packaging/python/build_static_readline_osx.bash](../../packaging/python/build_static_readline_osx.bash) to build a static readline library.
You can have a look at the script for requirements and usage.
You can have a look at the script for requirements and usage.

### Installing macOS prerequisites

Install the necessary Python versions by downloading the universal2 installers from https://www.python.org/downloads/macos/
You'll need several other packages installed as well (brew is fine):

```
brew install --cask xquartz
brew install flex bison mpich cmake
brew unlink mpich && brew install openmpi
brew uninstall --ignore-dependencies libomp || echo "libomp doesn't exist"
```

Bison and flex installed through brew will not be symlinked into /opt/homebrew (installing it next to the version provided by OSX can cause problems). To ensure the installed versions will actually be picked up:

```
export BREW_PREFIX=$(brew --prefix)
export PATH=/opt/homebrew/opt/bison/bin:/opt/homebrew/opt/flex/bin:$PATH
```

## Launch the wheel building

### Linux
Once we've cloned and mounted NEURON inside Docker(c.f. `-v` option described previously), we can proceed with wheels building.
Once we've cloned and mounted NEURON inside Docker(c.f. `-v` option described previously), we can proceed with wheels building.
There is a build script which loops over available pythons in the Docker image under `/opt/python`, and then builds and audits the generated wheels.
Wheels are generated under `/root/nrn/wheelhouse` and also accessible in the mounted NEURON folder from outside the Docker image.

```
# Working directory is /root/nrn
bash packaging/python/build_wheels.bash linux
bash packaging/python/build_wheels.bash linux
ls -la wheelhouse
```

You can build the wheel for a specific python version:
You can build the wheel for a specific python version:
```
bash packaging/python/build_wheels.bash linux 38 # 38 for Python v3.8
```
Expand Down Expand Up @@ -158,7 +177,7 @@ bash packaging/python/test_wheels.sh python3.8 "-i https://test.pypi.org/simple/

### MacOS considerations

On MacOS, launching `nrniv -python` or `special -python` can fail to load `neuron` module due to security restrictions.
On MacOS, launching `nrniv -python` or `special -python` can fail to load `neuron` module due to security restrictions.
For this specific purpose, please `export SKIP_EMBEDED_PYTHON_TEST=true` before launching the tests.

### Testing on BB5
Expand Down
2 changes: 1 addition & 1 deletion packaging/python/build_wheels.bash
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ pip_numpy_install() {
esac

# older version for apple m1 as building from source fails
if [[ `uname -m` == 'arm64' ]]; then
if [[ `uname -m` == 'arm64' && "$py_ver" == "39" ]]; then
numpy_ver="numpy==1.21.3"
fi

Expand Down

0 comments on commit 65efbdd

Please sign in to comment.