CI for your environment

[betatim]

If you want a not-terribly-complex way to make sure your environment still builds and does something useful adding something like repo2docker --no-run . to your CI setup should get you most of the way there. There are several repos that do this as a way to get continuous integration for their environment.

repo2docker . verify where verify is an executable script in your repo that determines if things still "work" is a neat addition for when you want to check more than just "could we build the environment".

I think by now everyone who has tried doing this (for more than a few weeks) has realised that the only way to keep things working is to continuously exercise them 😀.

[dhimmel]

I think by now everyone who has tried doing this (for more than a few weeks) has realised that the only way to keep things working is to continuously exercise them

Hah! But what about if we tag a commit and link users to the tag? This way users should always arrive at a prebuilt image and we just have to make sure is that we correct any binder issues if we're going to force change the tag?

If you want a not-terribly-complex way to make sure your environment still builds and does something useful adding something like repo2docker --no-run . to your CI setup should get you most of the way there.

@agitter I am okay with this solution, but I think it underscores that we probably should just move the figure generation to its own repo with its own CI

[betatim]

Hah! But what about if we tag a commit and link users to the tag? This way users should always arrive at a prebuilt image and we just have to make sure is that we correct any binder issues if we're going to force change the tag?

mybinder.org assumes that rebuilding your image at a given SHA at different points in time will result in the same image and makes no promises about how long we will keep cached images, so you have to be ready at "any time" for your image to be rebuilt. When it does happen people make one step forwards on the above list :) In practice it seems to happen a few times a year that we empty the cache.

we just have to make sure

Old man Tim would remove the "just" because it turns out to be tricky and tedious job, yay. 😀

From my experience what breaks your build is (in order of time that has passed since "let's make this reproducible, how hard can this be."):

• a library that got forgotten
• a dataset that got forgotten
• some analysis code at the very end of the process that uses cached results
• a direct dependency that wasn't pinned
• a dependency of a dependency that wasn't pinned
• one of the above that wasn't pinned "well enough" (maybe a library author says they do semver but then included a breaking change in a patch release)
• the host that hosts one of your inputs has gone down/got renamed
• repo2docker gets changed in a way that contains a mistake
• repo2docker gets changed to correct a mistake made previously
• the version you pinned contains a security hole and no one will let you run it any more
• ... etc etc

It seems like I have yet to find the end of this list. Now resigned myself to "unless the code doesn't interact with the universe or the universe stops changing things will always keep breaking" :-/

So I think the least hassle is to rebuild every few weeks and fix up what has broken. That way the breakage stays small, but it does require "constant attention".

[agitter]

mybinder.org... makes no promises about how long we will keep cached images

Okay, that part is important for our intended use case. We have a few high-priority changes to make in the next few weeks but then will discuss setting up CI for this environment, possibly in a new analysis repo.