New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GEOS-Chem CI Pipelines! #112
Conversation
This is great! I can try to set it up on the Github end soon. |
Great! Let me know if you run into any problems. |
One other question: should we also trigger a quick build for all "feature/*" requests? Sometimes we open feature branches before merging into dev? |
Sure, that's fine by me. I went back and forth on this. The reason ended up not adding the trigger on feature branches is because feature branches oftentime have transient changes that break the build (at least major features), and open pull requests and commits to open pull requests already do trigger quick build. Quick build only takes about 10 minutes though (~5 min overhead and ~5 min to compile), so there isn't really a downside to triggering it frequently |
Ah I see. Maybe it's OK if we only check the dev then. That's when it's really important. |
If someone does want quick build to check their feature branch, they can open a draft pull request which can't be merged but will trigger quick build. |
I've set up an Azure project under the name GEOSChem and have established a connection to dockerhub. So what is the next step? Also I think we should ask the GCST to create identities on Azure so I can add them as admins. |
I'll message you on Slack |
GEOS-Chem CI Pipelines!
Hi everyone,
This is a pull request for adding CI pipelines to GEOS-Chem Classic. I've set these CI pipelines up on my fork, and I've given you all the permissions to commit/push/tag/release on my fork so that you can try it out if you want. You can find the Azure pipelines for my GEOS-Chem fork here: https://dev.azure.com/lrbindle/geos-chem/_build?definitionId=10 (let me know if you can't access that link).
Sorry, this PR is long because there are a number of moving parts. Please don't think this means any cause for concern though, because the Azure pipeline is independent of the GitHub repo (besides a few YAML config files). I think we can start moving forward with this any time now.
I set up three pipelines. The first, "quick-build", is a fast and simple build pipeline that tries to compile GEOS-Chem in a "normal" build environment. This pipeline is triggered by commits to
dev/*
andbugfix/*
, as well as pull requests (and commits to open pull requests). The second, "build-matrix", tries to build GEOS-Chem in 11 different environments (I call them lines) which each focus on a specific GEOS-Chem dependency (e.g. there are lines for GCC 4-9, lines for old and new NetCDF's, etc.). This pipeline is triggered by pre-releases and commits to the master branch. The third pipeline, "release", builds a docker container with GEOS-Chem prebuilt in it (with separate executables for standard, tropchem, and soa_svpoa mechanisms) and automatically pushes it to Docker Hub. This pipeline is triggered by GitHub releases.Note: The build matrix pipeline already identified a problem with building with GCC 9! (see #113)
Please feel free to play around with pushing, making PRs, merging, and tagging on my GEOS-Chem fork to explore the behavior if you're interested (there's nothing important on my fork...I plan to re-fork soon anyways). Note that I've added some weird tags like 12.7.4 to my fork to test the CI behaviour...ignore these.
Moving forward
To merge this PR a few things need to happen. These things will need to be done by someone with admin privileges for the GEOS-Chem repo. The first of these is setting up the Azure pipelines. To do this, they need to:
Service connections
The "release" pipeline requires a service connection because it automatically pushes the built docker container to Docker Hub. To do this you should
To whoever sets this up: If you would like some help with this when you are setting this up, feel free to let me know and I can make sure I'm on Slack. We could message and/or video call with screen share if that would be helpful.
Release Docker container
The containers that the release pipeline builds have
geos-chem-standard
,geos-chem-tropchem
,geos-chem-soa_svpoa
executables in the PATH and their environments are configured. To run GEOS-Chem, the user shoulddocker run ...
cd
to their run directory (in the docker container)geos-chem-xxxxx
wherexxxxx
is the chemistry they wantBtw, these docker containers are going to be useful for us here at WashU because our new cluster runs docker images (when you log in you specify the docker container you want to load).
Penelope
All of this relies on my
penelope
Docker Hub registry which is a set of consistent containers for various build environments. By consistent I mean that if you runpenelope:0.1.0-ubuntu16.04-mvapich2.3.1-esmf8.0.0
it looks the same aspenelope:0.1.0-ubuntu16.04-openmpi3.1.4-esmf8.0.0
. This is important so that our CI pipelines can run the same commands in the various containers which makes the build matrix easy.These containers are built by an Azure pipeline that I set up for LiamBindle/penelope. I'm not convinced going with an Azure pipeline for building/deploying these was the best approach, but it was easy so that's why I did it. It's a bit finicky (it build containers for 35 different build environments every time). I'm not sure if this project should be moved to GEOS-Chem control or spun off as it's own standalone project. Any thoughts?
I think this can be done immediately
Keep in mind that none of this affects the GEOS-Chem repo. I've already added the pipeline YAML files to
feature/ci-pipelines
and that should be all we need to set up the DevOps project (i.e. this PR doesn't need to be merged before we start using the Azure pipelines). The only public-facing thing is the release containers that will automatically be pushed to Docker Hub upon tagged releases on GitHub.Looking forward to hearing your thoughts on moving forward.
Liam