speed up CI tests (do we need all conda packages?)

[lukasheinrich]

By using Conda, unfortunately the setup phase of the CI jobs have become a bit slower than without conda, maybe we can look into speeding them up again by checking whether we need all the packages that we install during CI

[matthewfeickert]

@lukasheinrich I'm not sure what can be done here, as we're already using Miniconda (which just installs an empty conda environment, right?). The only package that we're installing beyond PyTorch with conda is scipy

conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION pytorch scipy -c pytorch

so I can remove that and just have pip install it, but I don't know what else can really be done if we still use conda. I can look at seeing if there is an alternative way to get PyTorch in without conda though, but I think that will probably be equally slow.

[lukasheinrich]

Hi @matthewfeickert yeah.. I'm not sure exactly what can be done (and if nothing can be done that's fine).. I was just looking at the travis logs and saw

PREFIX=/home/travis/miniconda
installing: python-3.6.3-h6c0c0dc_5 ...
Python 3.6.3 :: Anaconda, Inc.
installing: ca-certificates-2017.08.26-h1d4fec5_0 ...
installing: conda-env-2.6.0-h36134e3_1 ...
installing: libgcc-ng-7.2.0-h7cc24e2_2 ...
installing: libstdcxx-ng-7.2.0-h7a57d05_2 ...
installing: libffi-3.2.1-hd88cf55_4 ...
installing: ncurses-6.0-h9df7e31_2 ...
installing: openssl-1.0.2n-hb7f436b_0 ...
installing: tk-8.6.7-hc745277_3 ...
installing: xz-5.2.3-h55aa19d_2 ...
installing: yaml-0.1.7-had09818_2 ...
installing: zlib-1.2.11-ha838bed_2 ...
installing: libedit-3.1-heed3624_0 ...
installing: readline-7.0-ha6073c6_4 ...
installing: sqlite-3.20.1-hb898158_2 ...
installing: asn1crypto-0.23.0-py36h4639342_0 ...
installing: certifi-2017.11.5-py36hf29ccca_0 ...
installing: chardet-3.0.4-py36h0f667ec_1 ...
installing: idna-2.6-py36h82fb2a8_1 ...
installing: pycosat-0.6.3-py36h0a5515d_0 ...
installing: pycparser-2.18-py36hf9f622e_1 ...
installing: pysocks-1.6.7-py36hd97a5b1_1 ...
installing: ruamel_yaml-0.11.14-py36ha2fb22d_2 ...
installing: six-1.11.0-py36h372c433_1 ...
installing: cffi-1.11.2-py36h2825082_0 ...
installing: setuptools-36.5.0-py36he42e2e1_0 ...
installing: cryptography-2.1.4-py36hd09be54_0 ...
installing: wheel-0.30.0-py36hfd4bba0_1 ...
installing: pip-9.0.1-py36h6c6f9ce_4 ...
installing: pyopenssl-17.5.0-py36h20ba746_0 ...
installing: urllib3-1.22-py36hbe7ace6_0 ...
installing: requests-2.18.4-py36he2e5f8d_1 ...
installing: conda-4.3.31-py36_0 ...
installation finished.

which might indeed just be the irreducible minimum for conda

[matthewfeickert]

@lukasheinrich Yeah that looks like the normal list of things that come with a new conda environment. I'll still poke at this though before we decide to close it.

I have some other questions about the CI related to speed but we can go over that later (or in another Issue).

[matthewfeickert]

@lukasheinrich I stand corrected, we can still do it in pip, as long as we run different install instructions for different versions of Python (they package it as different wheels). Do you have any reference YAML files for how to do this?

[matthewfeickert]

PyTorch is now on PyPI. I’ll do some tests to see what needs to be edited in terms of .travis.yml and then make a PR to see if we get speedup.

[matthewfeickert]

@lukasheinrich There are currently issues in PR #101 with having conditional requirements in setup.py to deal with PyTorch having different wheels for different versions of Python in non-GPU environments.

$ pip install -U --process-dependency-links -e .[develop]
Obtaining file:///home/travis/build/diana-hep/pyhf
    Complete output from command python setup.py egg_info:
    error in pyhf setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.

From searching it seems like this is an issue with setuptools, but using

before_install:
- pip install --upgrade pip setuptools

in .travis.yml is still causing things to error.

I don't suppose that you've run across something like this before?

[matthewfeickert]

The above seems to be an issue with setup.py not being compatible with explicit declarations of wheel urls. This in principle shouldn't be an issues, as pip should now default to wheel versions of packages if they exist. So I've defaulted just to asking pip to get torch.

However, there are still errors and differing errors between Python 2.7 and the Python 3.5/3.6 builds in Travis. Python 2.7 fails to install PyTorch with pip. Python 3.5/3.6 successfully installs PyTorch, however we get a memory error after all the tests successfully pass

=================== 22 passed, 18 warnings in 99.87 seconds ====================
*** Error in `/home/travis/virtualenv/python3.6.3/bin/python': corrupted size vs. prev_size: 0x0000000002161a90 ***
/home/travis/.travis/job_stages: line 57:  4971 Aborted                 (core dumped) pytest
The command "pytest" exited with 134.

I'll continue to follow up on this later, as we are seeing a multiple minute speedup over the ~10 minutes it takes for the CI with Conda.

[lukasheinrich]

Hi @matthewfeickert, yeah I think it's fine to pre-install torch.. In the end setup.py just declares the dependencies. That pip can install them for you is great, but it's not absolutely required. Does it work locally for you (i.e. is it just an issue with travis?)

[matthewfeickert]

I am running Python 3.6, so I'd have to setup a Python 3.5 Conda environment to check. I will follow up on this in Issue #103.

[lukasheinrich]

@matthewfeickert what happened to this? does the fact that we don't do 3.5 anymore help?

[matthewfeickert]

I had let it slide down the list. Let me look at it again tonight, now that Issue #103 is closed as you point out.

[matthewfeickert]

There seems to be more strangeness going on with memory errors, so unless this is viewed as an important issue I'm going to leave it till later.