Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Travis integration, conda environment yml #2

Merged
merged 26 commits into from Jul 25, 2019

Conversation

JelleAalbers
Copy link
Contributor

@JelleAalbers JelleAalbers commented Jul 21, 2019

This does the following:

  • Add a .travis.yml file to test the container build on TravisCI. Besides testing, we could use Travis in the future to push new container releases automatically (https://docs.travis-ci.com/user/docker/#pushing-a-docker-image-to-a-registry).
  • Move the list of conda packages to a conda.yml file. For clarity I listed only the conda packages we need directly for the moment, though I understand we might want to list all of them for maximum reproducibility. Pyup can apparently keep these yml files up to date just as it does for requirements.txt. If this would work, we can pin our dependencies here and remove the functionality in strax (see setup.py should read from a minimal-requirements list by default AxFoundation/strax#197)
  • Add code to run the strax/straxen tests at the end of the container build. If the tests fail, the container exits.
  • Add installation of the tensorflow2 beta, used in flamedisx and the new straxen version. I only got this to work by first installing regular tensorflow via conda (which pulls in a lot of related packages), then installing tensorflow2 via pip. This should become easier once tf2 is officially released, but that could be a few months.
  • Replace anaconda by miniconda. This means the base environment is lighter. Maybe we can also move to miniconda3, so the base environment is also python 3 (although it doesn't matter much since the base env is not used).

Currently the build fails (see https://travis-ci.org/JelleAalbers/base_environment) due to an issue with the GFAL2 python bindings: they can't find the boost headers. I saw the same problem trying to build the container on my laptop, though I don't think I tried exactly the current master (and I don't have enough disk space at the moment to try again ;-). @rynge Is the current master working for you, and if so, any ideas on what I messed up?

The travis.yml file has a curious while command, borrowed from travis-ci/travis-ci#4190. We currently compile boost from source, which takes >= 15 minutes. This is longer than TravisCI allows commands to run without producing output. I tried adding boost to the conda package list instead, but then gfal complains about missing headers. I tried to remove run_quietly, but then ravis kills the job for producing too much output. I also tried travis_wait, but this apparently only works when wrapped around the entire build command, and would suppress its output.

(and sorry for the mess of commits, when we get this to work you probably want to squash-merge)

@rynge
Copy link
Collaborator

rynge commented Jul 22, 2019

@JelleAalbers Thanks! This looks great. Re boost - let me work on this and see if we can't pull boost from anaconda as well. Note that I already have deployments in DeployHQ, but switching to Travis makes sense if that is what the rest of the project is using. I will try to look at this later this week.

@rynge
Copy link
Collaborator

rynge commented Jul 25, 2019

I'm going to merge this, and then put my local changes on top

@rynge rynge merged commit 7f9f883 into XENONnT:master Jul 25, 2019
@JelleAalbers JelleAalbers deleted the conda_yml branch July 27, 2019 11:44
ershockley pushed a commit that referenced this pull request Mar 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants