JOSS: Issues installing

[skyreflectedinmirrors]

@bjodah I'm having some issues installing pyodesys for the JOSS review:

First, let me note that installing the main package via conda worked without a hitch, pip not so much (but that's a more difficult nut to crack). However:

1. There needs to be a list of all dependencies that are needed to run tests / examples / etc. A lot of this can be inferred from either the setup.py or the other optional packages, but I tried that route and I'm still running into errors on tests / examples (more on this later). Preferably this list would be the docs, and there would be a link in the install section along the lines of "Optional Dependencies...". Even if this links to the various subpackages (e.g. pygslodeiv2) which then tell you how to install themselves, that would be an improvement.

2. This list needs to include any non-python packages required such that a pip install will be possible.
   Conda is obviously an easier choice, but if you're going to offer pip installation I need to the base packages required. Again using pygslodeiv2 as an example, it's not immediately obvious where GSL should be installed to (other than "default" locations?). Or for odeint do I need Boost? ode-int itself?

3. Even with (I think) all the sub-dependencies installed, most of the tests skip and the examples give erros. My conda environment is here

e.g.: my test output:

pyodesys/pyodesys/native/tests/test_cvode.py sssssssssssssssssssssssssssss [ 11%]
pyodesys/pyodesys/native/tests/test_gsl.py sssssFsssssssss [ 16%]
pyodesys/pyodesys/native/tests/test_odeint.py ssssFsssssss [ 21%]
pyodesys/pyodesys/tests/test_core.py s.ssssssss....s.....s..s..s..ssss. [ 34%]
pyodesys/pyodesys/tests/test_plotting.py ss [ 35%]
pyodesys/pyodesys/tests/test_results.py ..s [ 36%]
pyodesys/pyodesys/tests/test_robertson.py ssssssssss [ 40%]
pyodesys/pyodesys/tests/test_symbolic.py ....ssssssssss.................ssssssssssssssssssssssssssssssssssssssssss...sssssssssss.sss.s.s....ss..ssssssssssssssss [ 86%]
sssssssssssssssssssssss.ssssssssss [ 99%]
pyodesys/pyodesys/tests/test_util.py ..

Or running chemical_kinetics.ipynb:

~/miniconda3/envs/pyodesys/lib/python3.6/site-packages/pycvodes/init.py in ()
7 import numpy as np
8
----> 9 from ._cvodes import adaptive, predefined, requires_jac, steppers, fpes
10 from ._util import _check_callable, _check_indexing
11 from ._release import version

ImportError: libsundials_cvodes.so.2: cannot open shared object file: No such file or directory

edit: link back to joss review:
openjournals/joss-reviews#490 (comment)

[bjodah]

@arghdos, thank you for your thorough feedback on the installation issues. Right now for installation using pip I've taken an "all or nothing" approach (where pip install pyodesys installs the bare minimum and pip install pyodesys[all] installs everything). The original reason for this was that the 3rd party libraries were not always easily accessible on windows (or even OS X) until recently (now they are all available for all platforms via conda).

Arguably I could extend what the "bare minimum" is (e.g. have it include also sympy, scipy, matplotlib & pytest) for pyodesys. Then a basic "pip-install" of the package will be more useful. That would also lead to fewer skipped tests due to missing imports (maybe I should also instruct the user to pass the -rs flag to pytest?). Do you think that would be an improvement?

• 1/2) I'll add an "Optional dependencies" subsection to the installation section. I'll also add a note about what 3rd party libraries those include (sundials 2.7.0/gsl >=1.16/boost >=1.65.0) for pycvodes/pygslodeiv2/pyodeint respectively), in addition I can add the required package names in e.g. debian/ubuntu.

• 3) I could add a link to a binder (which would use a Dockerimage)? That would then serve both as an example of setup as well as an easy way for prospect users to explore the package.

[skyreflectedinmirrors]

@bjodah Ok, that helped a lot (the sundials version in particular was a hangup for me, as by default conda installs 3.x.x). Additionally, the -rs option is also helpful as it lists some other packages that might be missing in the setup dependencies (or possibly sub-dependencies?) --symcxx, quantities, symengine (and matplotlib, but it looks like you fixed that already).

The docker image is a great idea as well! Particularly for complicated mixed python / c packages I think it's critical to make the installation as well explained as possible, it's very easy (as we're showing here!) for small details to result in broken installs. Also an entry point into exploring the package / examples would be excellent.

I still end up with failed tests however). It appears to be only for tests that require pycompilation or pycodeexport. g++ --version on this system reports:

g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

which as far as I can tell should support c++14 as required for these examples, any thoughts here?

[bjodah]

@arghdos thanks.

g++5.4 is new enough (that's the version I use myself). Looking at your latest failed test log I see:

...
nvector/nvector_serial.h: No such file or directory

this suggests that sundials' headers are not installed in any of the default include directories. One way to work around that is setting e.g. CPLUS_INCLUDE_PATH=/path/to/sundials-2.7/include

Regarding sundials-2.7.0 vs. 3.x: yes, right now pycvodes only works with 2.7.0, they broke both API and ABI moving from 2.7.0 -> 3.x. It is "just" a matter of adding a bunch of #ifdef blocks to pycvodes, I will do so as soon as I find some time.

I have now added both an environment.yml file for use with binder (link in README) and a script for hosting a docker-based jupyter notebook with pip-installed dependencies (with instructions for how to host in README). That Dockerfile then also serves as an example of how to setup a system with the required 3rd party libraries (the Dockerfile is based on the current debian stable).

Regarding symcxx, quantities and python-symengine:
I added the first two to the pip-installed environment in the Dockerfile. I'd rather not add symengine to that list since their official way of installing is through conda. So any tests depending on symengine (currently there is 1 such test) will be skipped in the Docker environment (it is still being run on the CI server).

EDIT: symcxx support is really "beta", and it's an experimental package of mine and not "production ready" by any means.

[skyreflectedinmirrors]

...
nvector/nvector_serial.h: No such file or directory
this suggests that sundials' headers are not installed in any of the default include directories.

One way to work around that is setting e.g. CPLUS_INCLUDE_PATH=/path/to/sundials-2.7/include

Aaahhh, I had missed that in the noise of the log. I had gotten pycvode to install correctly previously by passing the miniconda directories to pip but it makes sense that it needs them at compile time as well. I'll give that a shot

[skyreflectedinmirrors]

@bjodah that did the trick (along with a LIBRARY_PATH to point the library search in the right direction). I'll give the docker a test as well, but I think this issue can be closed