Installing iris with pip is cumbersome (with fix at the bottom)

[valeriupredoi]

Hey guys, been a while since I spammed you with nasty stuff 😁 I am trying to think carefully how to migrate ESMValTool deps from conda to PyPi since PyPi is so much more awesome, and in the process, as a natural starter, I started with iris>=2.2. Since some of our deps are impossible to install from PyPi we'll still require a conda environment; installing iris with pip is actually harder and more cumbersome than I thought (sorry, am being nasty 😁 ). Here is the conda env file that I used for a successful installation:

 ---
name: test_basic
channels:
  - conda-forge

dependencies:
  #- matplotlib<3
  - pip=18   # revert from 19 due to PEP517 error with cartopy
  - six      # will not install automatically via pip
  - proj4    # will not install automatically via pip
  - pyke     # will not install automatically via pip
  - cartopy  # pip install: error: command 'gcc' failed with exit status 1 
  - udunits2 # libudunits2.so.0: cannot open shared object
  - pip:
    - scitools-iris>=2.2

As you can see there are a number of iris deps that will not be automatically picked up and installed by pip and need to be installed in advance by conda; also there is this issue that forces pip to be retrograded to v18.1; and there is the issue of udunits2 that manifests itself after the environment has successfuly been created and iris installed (creeps up at import iris stage).

Anything that can be done for a much smoother installation of the package? Will buy beer 🍺

NOTE: the PEP517 issue is not only for MacOSX, I used Debian/Ubuntu and ran into it

[ocefpaf]

I'll try to summarize the problem here:

• proj4 and udunits2 are external, non-python dependencies, and pip does not handle those. You have to either use a "complete" package manager like apt-get, yum, conda, etc, and make sure you have all the dependencies in the right place before installing iris.

• The gcc error for cartopy is similar b/c pip does not handle compilers needed for cartopy's installation. Again, either install a compiler with your preferred package manager, or use a pre-packaged binary for cartopy. (I don't think there are PyPI wheels for it but there are packages for Fedora, conda, etc.)

• pyke is a different beast b/c it is pure python, does not require compilation, but the authors never published it on PyPI. So... no pip install for that one as well.

All I can say about these problems is: welcome to scientific python packaging! These problems are not exclusive of iris at all, they are a problem with the ecosystem.

[bjlittle]

@valeriupredoi You're being hit by the weaknesses of using pip over conda... just as @ocefpaf summarised so well

Purging pyke from iris has been on my todo list since... well, forever, so at least we can tackle that headache

[bjlittle]

@valeriupredoi Let's make sure to talk about this at the DLR tech workshop in Oberpfaffenhofen this week, okay?

[valeriupredoi]

cheers for the absolutely valid answers guys! I will try talk to @bjlittle about this very soon since we are actually in the same room all thru this week 😁 Then I'll post the update and close the issue.

[ocefpaf]

cheers for the absolutely valid answers guys! I will try talk to @bjlittle about this very soon since we are actually in the same room all thru this week Then I'll post the update and close the issue.

Note that building an iris wheel would be great to the community but the amount of work there is not trivial at all due to the external C-dependencies. Not to mention the problem with the name iris being taken on PyPI, there iris is named scitools-iris, creating a lot of confusion for users.

PS: just a minor pet peeve, the problem here has nothing to do with a conda environment and the title should be "Installing iris with pip is cumbersome" 😉 . In fact a conda environment may help you get all the dependencies so you can pip install it. (Although in that case is only valid when installing the dev of iris version, otherwise just install iris itself with conda.)

[valeriupredoi]

guys, adding anaconda as option channel made things much easier: cheers @bjlittle for the suggestion!

---
name: iris_pypi
channels:
  - anaconda
  - conda-forge
  # for iris
  - pip
  - pyke
  - cartopy
  - udunits2
  - pip:
    - scitools-iris>=2.2

is the much nicer recipe to install it. Many thanks and I'm gonna close this 🍺

[ocefpaf]

When using multiple channels it is recommend to use strict channel option:

conda config --set channel_priority strict

You will need conda version >=4.6.9 if I'm not mistaken. Also, none of the problems you reported above are channel related. I'm guessing there were other problems that you had to solve offline. I'm saying this b/c recommending to add the anaconda channel may be misleading. (Also, I guess you should use defaults instead of anaconda, not sure the the latter is aliased to the former.)

[valeriupredoi]

cool man, we are using 4.6.14 and always opt for the latest conda version anyway. Cheers muchly for the help! I am going to close this since my problem is now fixed 🍺

[bhushanbrb]

I did not see a fix to install it from pypi , I get lot of error when trying to install from pip. such as those mentioned above