New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.4 Travis fail due to unmet dependencies #2066

Open
orbeckst opened this Issue Sep 8, 2018 · 26 comments

Comments

Projects
None yet
6 participants
@orbeckst
Member

orbeckst commented Sep 8, 2018

The Python 3.4 tests are all failing now:

UnsatisfiableError: The following specifications were found to be in conflict:
  - numpy=1.15 -> *[track_features=blas_openblas]
  - numpy=1.15 -> python[version='>=3.7,<3.8.0a0'] -> tk[version='>=8.6.7,<8.7.0a0']
  - python=3.4
Use "conda info <package>" to see the dependencies for each package.

Examples:

Do we need to pin numpy or should we deactivate 3.4 tests?

cc: @richardjgowers @jbarnoud @tylerjereddy

@zemanj

This comment has been minimized.

Member

zemanj commented Sep 9, 2018

We could pin the numpy version here as yet another band-aid. Though, what we really want is something like "take the highest possible version that is compatible with this python version". Is there a sane way to achieve this without doing nasty things like parsing the output of conda info?

@richardjgowers

This comment has been minimized.

Member

richardjgowers commented Sep 9, 2018

I thought figuring this stuff out was something conda was meant to figure out for itself. Maybe we're not giving conda enough information initially to make these decisions. Pinning on the 3.4 build is an easy fix which wil allow PRs to continue though

@orbeckst

This comment has been minimized.

Member

orbeckst commented Sep 10, 2018

We will have to do something soon because this holds up merging of all PRs.

I thought figuring this stuff out was something conda was meant to figure out for itself. Maybe we're not giving conda enough information initially to make these decisions.

Yes, I though so, too. Perhaps this is actually a bug in one of the build recipes (maybe @kain88-de sees immediately what's wrong) or in the ci-helper script (???) but that doesn't help us right now.

What is the last version of numpy that works for us in 3.4? We then pin that version, using ci-helper's NUMPY_VERSION

PYTHON_VERSION=3.4 NUMPY_VERSION=1.14 CODECOV="true" SETUP_CMD="${PYTEST_FLAGS} --cov=MDAnalysis" 

(or whatever is needed)

orbeckst added a commit that referenced this issue Sep 10, 2018

pin numpy version for Python 3.4
- NUMPY_VERSION=1.14 (last version known to be compatible with
  Python 3.4 when installed via conda and conda-forge channel)
- fix #2066
- This is a band-aid and should be reverted if the dependencies
  in the conda-forge packages have been corrected.

@orbeckst orbeckst added this to the 0.19.0 milestone Sep 10, 2018

@orbeckst orbeckst referenced this issue Sep 10, 2018

Merged

pin numpy version for Python 3.4 #2067

1 of 4 tasks complete
@kain88-de

This comment has been minimized.

Member

kain88-de commented Sep 10, 2018

This looks like a conda error. Numpy 1.15 has not been build for python3.4. This is coming up now because the 1.15 version has been published 6 days ago.

Ci-helpers uses a default pin of 1.15 for numpy as the latest stable version. This pin causes a problem for python3.4 so using a different pinned version is a good solution.

@kain88-de kain88-de closed this Sep 10, 2018

@kain88-de kain88-de reopened this Sep 10, 2018

@orbeckst

This comment has been minimized.

Member

orbeckst commented Sep 10, 2018

Should we open an issue with ci-helpers?

@kain88-de

This comment has been minimized.

Member

kain88-de commented Sep 10, 2018

@zemanj zemanj closed this in #2067 Sep 10, 2018

@zemanj zemanj reopened this Sep 10, 2018

@zemanj

This comment has been minimized.

Member

zemanj commented Sep 10, 2018

For the record:

  • The quick fix from PR #2067 is in place
  • More thorough investigation needed for what's going wrong with conda / ci-helpers
  • Issues regarding error handling in ci-helpers should go into an issue/PR there.
@orbeckst

This comment has been minimized.

Member

orbeckst commented Sep 11, 2018

I removed the critical label because we have at least the band-aid #2067 .

@zemanj

This comment has been minimized.

Member

zemanj commented Sep 11, 2018

I ran some tests locally and figured out that for the python 3.4 build there are two possible solutions that don't result in a gazillion of slow pip installs:

  1. Pin numpy version: NUMPY_VERSION=1.11
  2. Do not require a specific numpy version: NUMPY_VERSION=''

Both solutions result in numpy 1.11.3 being installed.
Which solution do you think is better?

@zemanj

This comment has been minimized.

Member

zemanj commented Sep 11, 2018

I investigated the issue further:

  • Python 3.4 builds fail since the "stable" numpy version in ci-helpers has been bumped from 1.14 to 1.15 on September 5. This is because before trying to install other dependencies, ci-helpers starts by setting up the test env with
conda create -n test python=$PYTHON_VERSION numpy=$NUMPY_VERSION

and that fails because in the conda-forge channel, there is no python 3.4 build for numpy 1.15.

  • Before the stable version bump, the python 3.4 builds only passed because of the pip fallback. So the numerous pip installs we're currently seeing when pinning NUMPY_VERSION=1.14 are in fact nothing new, they just went unnoticed. See, e.g., this build from August 24.
  • The biggest problem with pip installs is that they're especially slow for scipy, and the latest python 3.4 build of scipy is version 0.18.1 with numpy 1.11.3. The same is true for biopython.

Conclusion:
Having a conda-forge numpy 1.15 build for python 3.4 will only help in conjunction with builds of scipy and biopython (and maybe further dependencies) for python 3.4 and numpy 1.15.

Until that happens (I don't think it ever will), we can either use solution 1 above (safe) or solution 2 (unsafe, but will automatically raise the numpy version if suitable builds become available).

@richardjgowers

This comment has been minimized.

Member

richardjgowers commented Sep 11, 2018

@kain88-de

This comment has been minimized.

Member

kain88-de commented Sep 11, 2018

Having a conda-forge numpy 1.15 build for python 3.4 will only help in conjunction with builds of scipy and biopython (and maybe further dependencies) for python 3.4 and numpy 1.15.

Only a numpy build for python 3.4 should be required in theory. The best-practice for packages is to build against numpy 1.8 which is ABI compatible with any later version. MDAnalysis, for example, builds it's conda-forge packages against numpy 1.10 and can be run in combination with any later version.
This can work because we only pin numpy and conda is free to choose corresponding versions for our other dependencies.

The simple solution is to drop 3.4 though. Keep in mind our original argument for python3.4 was that this is the current python version in LTS distributions like CentOS, OpenSuse and Ubuntu LTS (not sure for 2018.4!). These operating systems are common on HPC clusters.

@zemanj

This comment has been minimized.

Member

zemanj commented Sep 11, 2018

Dropping testing for Python 3.4 essentially implies dropping support for Python 3.4 entirely, since the functionality can no longer be guaranteed.

The simple solution is to drop 3.4 though. Keep in mind our original argument for python3.4 was that this is the current python version in LTS distributions like CentOS, OpenSuse and Ubuntu LTS (not sure for 2018.4!). These operating systems are common on HPC clusters.

So dropping Python 3.4 testing is not really an option.

@kain88-de

This comment has been minimized.

Member

kain88-de commented Sep 11, 2018

So dropping Python 3.4 testing is not really an option.

It is an option. Conda offers a good way to get newer python version on a cluster. I just wanted to mention why we set the oldest python 3 version to 3.4 when we added python 3 support last year. I would also be OK to say we only support the two most recent versions. We only have limited developer time anyway.

@zemanj

This comment has been minimized.

Member

zemanj commented Sep 11, 2018

Ok, true that. And having no tests for 3.4 with the latest stable numpy version is not desirable anyway.

@orbeckst

This comment has been minimized.

Member

orbeckst commented Sep 11, 2018

In general I agree with dropping support when we don't have the time to support it. In this case, it seems an easy fix, though, that does not really cost us much, now that @zemanj figured out what is happening.

I think a user with Python 3.4 would do what is equivalent to NUMPY_VERSION='': just let conda figure it out. I think that's the sensible thing to do. Or what are the downsides?

For downstream packagers it would probably also be easier if we are not too restrictive about Python versions.

(ci-helpers does not document that they pin numpy to a specific version by default. (EDIT: I suppose it is implied that it is "stable" abd that release is decided on by ci-helpers). This should be mentioned because of exactly the problem we are having here, which breaks the assumption that you can rely on conda to figure out the dependencies. I understand the rationale of working with a specific version of numpy for a scientific package, though. In any case, I'm happy to open an issue once we know what we're doing.)

@zemanj

This comment has been minimized.

Member

zemanj commented Sep 12, 2018

I think the only downside is that, at the moment, we won't have python 3.4 tests with the latest numpy version. Maybe still better than nothing, though. Shall I revive PR #2067 or open a new PR?

@kain88-de

This comment has been minimized.

Member

kain88-de commented Sep 12, 2018

We do set the standard numpy version in tests to be stable meaning the last stable release.

- NUMPY_VERSION=stable

I set this to ensure we always test against the latest numpy release and make sure everything works. This works independent of what conda thinks and gives us a better reproducible test environment (at least I thought so). Our other tests run against develop and 1.10 our minimum required version. This combination should ensure we are good from numpy 1.10 to the latest and greatest.

@zemanj

This comment has been minimized.

Member

zemanj commented Sep 12, 2018

@kain88-de That's exactly what I intended to say with my last comment, thanks for clarifying. If we set NUMPY_VERSION='' in the 3.4 tests, this overwrites the global setting, and thus, we lose tests with the latest stable numpy release for python 3.4.

If we do this, we need to remove it again once there is a python 3.4 build for numpy 1.15.
This could even be checked for in .travis.yml by adding something like

if [ "$PYTHON_VERSION" == "3.4" ] && [ -z "$NUMPY_VERSION" ] && [ -n "$(conda info numpy=$NUMPY_STABLE | grep py34)" ]; then
    echo "Latest stable numpy build available for Python 3.4, remove NUMPY_VERSION='' in .travis.yml." 1>&2
    exit 1
fi
@tylerjereddy

This comment has been minimized.

Member

tylerjereddy commented Sep 12, 2018

We've removed Python 3.4 from the NumPy CI testing matrices; I think from the upcoming 1.16 there's no "official" 3.4 support, though we're not trying to break compatibility right out of the gate.

@orbeckst

This comment has been minimized.

Member

orbeckst commented Sep 12, 2018

Well, if numpy is not supporting/testing 3.4 anymore then we can drop it.

(Personally, I dislike the rat race of having to update all the time – the timescale for science is slower than the time scale for much of the open source software development world; I understand that this is a function of available developer time. However, perhaps we should have archived requirements.txt files that exactly record a set of dependencies that is known to work, maybe just before we announce dropping support for something.)

@orbeckst

This comment has been minimized.

Member

orbeckst commented Sep 12, 2018

@MDAnalysis/coredevs the motion to drop 3.4 support is on the table (see details in #2066).

Summary

  • latest numpy does not officially test 3.4 anymore (EDIT: #2066 (comment) said "upcoming 1.16", I assume that means that the current 1.15 is still supporting/testing 3.4 but the fact remains that numpy, our major dependency, is dropping official support in the near future)
  • our tests for 3.4 cannot use the latest numpy because the package dependencies do not resolve properly (we are currently pinning it to an earlier version but that's a band-aid that's also not working properly, details in PR #2067); more importantly, the testing philosophy has been to always test against latest stable numpy and the our oldest support numpy (with an older Python version). We cannot follow through with this philosophy anymore for 3.4
  • developer time is precious, so just supporting Python 2.7 (until 2020) and last two Python releases (3.5, 3.6, ... well 3.7 lingers with PR #1963 ) would be ok

Opinions, simple 👍 or 👎 ?

@jbarnoud

This comment has been minimized.

Contributor

jbarnoud commented Sep 12, 2018

We are supporting older versions of numpy that still support python 3.4. Unless we encounter issues that are specific to python 3.4, I would be in favor of keeping the support and test with an older version of numpy. There is some cost at supporting language features that are absent or different in python 3.4; though it is unlikely that we encounter them as long as we support python 2.7.

I am in favor of dropping support for python 3.4 when we bump the minimal required version of python to a version that does not support python 3.4, when we drop the support for python 2.7, or for MDAnalysis 1.0.

@tylerjereddy

This comment has been minimized.

Member

tylerjereddy commented Sep 12, 2018

Maybe supporting 3.4 is indeed just too much developer resources -- ideally, we might wait until NumPy drops it officially on a released version, but depends if we can spare someone to deal with the mess described above.

NumPy 1.16 will drop Python 3.4 support. 1.15 has been released and was "all green" for Python 3.4 at release time. Since that release has happened, we no longer test on 3.4 as we're developing for the 1.16 release on master. 1.16 will be an LTS release with 2.7 and 3.5-3.7 supported (bug fixes only) through 2020.

@orbeckst

This comment has been minimized.

Member

orbeckst commented Sep 13, 2018

We seem to have a majority for dropping 3.4 support.

Could I make a compromise suggestion?

  • We officially drop support for 3.4 once numpy 1.16 is officially out and they announce that they drop support for 3.4.
  • Until then we test against 3.4, but pin numpy to our minimal version of numpy (1.10.4 I think) for the 3.4 tests. (This should allow fast conda-based installs and avoid pip).
  • We add something to the docs that outline what we are supporting and roughly what our policy is with respect to supporting Python 2.7 and latest Python 3.x. (We need this for 1.0 anyway.)
@richardjgowers

This comment has been minimized.

Member

richardjgowers commented Sep 13, 2018

Yeah pinning to a 3.4 compatible numpy works too, which is what @jbarnoud is saying, so I'm changing my vote to that I guess

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment