Use Miniconda in Travis for package installation #231

Merged
merged 1 commit into from Jun 14, 2015

Conversation

Projects
None yet
2 participants
@oliverlee
Contributor

oliverlee commented Jun 14, 2015

Travis will test using Miniconda for installing all required Python
packages in addition to testing with Ubuntu 12.04 site packages for
Numpy, Scipy, and Cython.

  • There are no merge conflicts.
  • If there is a related issue, a reference to that issue is in the
    commit message.
  • Unit tests have been added for the new feature.
  • The PR passes tests both locally (run nosetests) and on Travis CI.
  • All public methods and classes have docstrings. (We use the numpydoc
    format
    .)
  • An explanation has been added to the online documentation. (docs
    directory)
  • The code follows PEP8 guidelines. (use a linter, e.g.
    pylint, to check your code)
  • The new feature is documented in the Release
    Notes
    .
  • The code is backwards compatible. (All public methods/classes must
    follow deprecation cycles.)
  • All reviewer comments have been addressed.
- - python: 2.7
- env: SYMPY_VERSION="master"
-virtualenv:
- system_site_packages: true

This comment has been minimized.

@moorepants

moorepants Jun 14, 2015

Member

Doesn't this flag have to stay in the file to ensure that the system packages are used? Travis defaults to a virtualenv that doesn't have access to the system packages.

@moorepants

moorepants Jun 14, 2015

Member

Doesn't this flag have to stay in the file to ensure that the system packages are used? Travis defaults to a virtualenv that doesn't have access to the system packages.

This comment has been minimized.

@oliverlee

oliverlee Jun 14, 2015

Contributor

Versions of python that are not 2.7 and 3.2 fail in Travis as site packages are not available. This shows an example of what happens if you leave the flag: travis-ci/travis-ci#2219
which also happened with previous versions of this commit: https://travis-ci.org/pydy/pydy/pull_requests

@oliverlee

oliverlee Jun 14, 2015

Contributor

Versions of python that are not 2.7 and 3.2 fail in Travis as site packages are not available. This shows an example of what happens if you leave the flag: travis-ci/travis-ci#2219
which also happened with previous versions of this commit: https://travis-ci.org/pydy/pydy/pull_requests

This comment has been minimized.

@moorepants

moorepants Jun 14, 2015

Member

So travis creates a default virtualenv for Python that doesn't have access to the system python packages. And it also installs a few other packages into this venv (numpy, mock, nose, etc). The process I had here was to first remove the venv numpy because it was a newer version of numpy, and then the venv python will use the system numpy, scipy, etc. which are the versions that shipped with Ubuntu 12.04. If the system_site_packages flag isn't set, then I don't think that is what happens. It may be using conda's packages or something in this case.

@moorepants

moorepants Jun 14, 2015

Member

So travis creates a default virtualenv for Python that doesn't have access to the system python packages. And it also installs a few other packages into this venv (numpy, mock, nose, etc). The process I had here was to first remove the venv numpy because it was a newer version of numpy, and then the venv python will use the system numpy, scipy, etc. which are the versions that shipped with Ubuntu 12.04. If the system_site_packages flag isn't set, then I don't think that is what happens. It may be using conda's packages or something in this case.

.travis.yml
- - pip uninstall -y numpy
+ - pip uninstall -y numpy;
+ - if [[ $TRAVIS_PYTHON_VERSION != "2.7_with_system_site_packages" ]]; then
+ sudo apt-get install -qq libatlas-dev libatlas-base-dev liblapack-dev gfortran;

This comment has been minimized.

@moorepants

moorepants Jun 14, 2015

Member

I'm not sure any of these are actually needed. Why are they added?

@moorepants

moorepants Jun 14, 2015

Member

I'm not sure any of these are actually needed. Why are they added?

This comment has been minimized.

@oliverlee

oliverlee Jun 14, 2015

Contributor

I thought they were necessary to build scipy but I guess if we install it using conda? I was building it with pip at one point. I'll see if it succeeds in a new build.

@oliverlee

oliverlee Jun 14, 2015

Contributor

I thought they were necessary to build scipy but I guess if we install it using conda? I was building it with pip at one point. I'll see if it succeeds in a new build.

This comment has been minimized.

@moorepants

moorepants Jun 14, 2015

Member

There are not required for scipy install from apt or conda because those are binary installations. (This is a good reason to switch to conda, because compiling numpy and scipy get old real quick, especially if you use environments or need different versions.)

@moorepants

moorepants Jun 14, 2015

Member

There are not required for scipy install from apt or conda because those are binary installations. (This is a good reason to switch to conda, because compiling numpy and scipy get old real quick, especially if you use environments or need different versions.)

This comment has been minimized.

@oliverlee

oliverlee Jun 14, 2015

Contributor

Not necessary: https://travis-ci.org/pydy/pydy/builds/667468
Looks like I forgot to push the testing the commit I had.

@oliverlee

oliverlee Jun 14, 2015

Contributor

Not necessary: https://travis-ci.org/pydy/pydy/builds/667468
Looks like I forgot to push the testing the commit I had.

@moorepants

This comment has been minimized.

Show comment
Hide comment
@moorepants

moorepants Jun 14, 2015

Member

This looks fine except for my two comments.

Member

moorepants commented Jun 14, 2015

This looks fine except for my two comments.

.travis.yml
+ wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O miniconda.sh;
+ chmod +x miniconda.sh;
+ ./miniconda.sh -b;
+ export PATH=/home/travis/miniconda/bin:$PATH;

This comment has been minimized.

@moorepants

moorepants Jun 14, 2015

Member

You probably don't want to install conda at all if you want to use the apt packages, like the previous setup. Right here you skip over the virtualenv with this line.

@moorepants

moorepants Jun 14, 2015

Member

You probably don't want to install conda at all if you want to use the apt packages, like the previous setup. Right here you skip over the virtualenv with this line.

This comment has been minimized.

@oliverlee

oliverlee Jun 14, 2015

Contributor

Yeah it's skipped.

@oliverlee

oliverlee Jun 14, 2015

Contributor

Yeah it's skipped.

@moorepants

This comment has been minimized.

Show comment
Hide comment
@moorepants

moorepants Jun 14, 2015

Member

I have a suggestion here. How about we do not use any apt system packages at all anymore. We just use conda for all the testing. The only downside (the reason I hadn't done it yet) is that we will have to bump up our minimum dependencies because anaconda does not ship with all of the versions of numpy, scipy, etc that shipped with Ubuntu 12.04. So we will just bump up the minimum deps for (maybe only) scipy.

Member

moorepants commented Jun 14, 2015

I have a suggestion here. How about we do not use any apt system packages at all anymore. We just use conda for all the testing. The only downside (the reason I hadn't done it yet) is that we will have to bump up our minimum dependencies because anaconda does not ship with all of the versions of numpy, scipy, etc that shipped with Ubuntu 12.04. So we will just bump up the minimum deps for (maybe only) scipy.

@oliverlee

This comment has been minimized.

Show comment
Hide comment
@oliverlee

oliverlee Jun 14, 2015

Contributor

Can we install phantomjs with conda?

Contributor

oliverlee commented Jun 14, 2015

Can we install phantomjs with conda?

@moorepants

This comment has been minimized.

Show comment
Hide comment
@moorepants

moorepants Jun 14, 2015

Member

Ah, still install phantom with apt. All the JS testing stuff needs to be separate from conda as there may not be conda packages for that.

Member

moorepants commented Jun 14, 2015

Ah, still install phantom with apt. All the JS testing stuff needs to be separate from conda as there may not be conda packages for that.

Use Miniconda in Travis for package installation
Travis will test using Miniconda for installing all required Python
packages. The Ubuntu 12.04 site packages for Numpy, Scipy, and Cython
are no longer used and the corresponding required versions will need to
be changed as we no longer test them.
@oliverlee

This comment has been minimized.

Show comment
Hide comment
@oliverlee

oliverlee Jun 14, 2015

Contributor

Only difference from last patch set is removal of comments about package versions in Ubuntu 12.04, as we no longer use those.

Build passes, see here: https://travis-ci.org/pydy/pydy/builds/66777772

Contributor

oliverlee commented Jun 14, 2015

Only difference from last patch set is removal of comments about package versions in Ubuntu 12.04, as we no longer use those.

Build passes, see here: https://travis-ci.org/pydy/pydy/builds/66777772

@moorepants

This comment has been minimized.

Show comment
Hide comment
@moorepants

moorepants Jun 14, 2015

Member

+1, merging.

Member

moorepants commented Jun 14, 2015

+1, merging.

moorepants added a commit that referenced this pull request Jun 14, 2015

Merge pull request #231 from oliverlee/enh-use-conda-in-travis
Use Miniconda in Travis for package installation

@moorepants moorepants merged commit bfd31ff into pydy:master Jun 14, 2015

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

@oliverlee oliverlee deleted the oliverlee:enh-use-conda-in-travis branch Jun 14, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment