Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[MRG+1] Split julia support into pre julia 1.0 and post julia 1.0 #595
This is an alternative to #594, that I think is cleaner.
@betatim This doesn't follow your suggestion of composition, instead it just treats these two julia versions as entirely different stories. I think we probably eventually want to just remove support for version pre 1.0: they are not maintained, and I have a hard time imagining that anyone will use them for much longer. Removing support for this old legacy stuff would be super easy with the code structure in this PR: one wouldn't even have to touch the julia 1.0 support code at all. But we can also leave it in indefinitely, given that support for the two is more or less completely isolated, it probably doesn't do any harm.
The logic of which of the julia implementations should handle a given repo is super simple, and I don't think there is a risk that they would get in each others way.
There is also a more radical suggestion: we could say that a
The major thing that is still missing from this PR is the logic that selects the exact julia 1.x.x version subject to the version specifier in the
I'm stuck right now on a) already, because I don't know what I need to do to add a package dependency :) I'm not really a Python expert ;)
The support for julia 1.0 here has some other improvements: 1) I managed to install the Julia kernel into the right location right away, no manual file move needed anymore, 2) we don't need the julia code that resolves the repo dependencies, the package manager handles all of that now.
Random other question: why does the julia build pack inherit from the python build pack? Wouldn't it make more sense to inherit from something that doesn't install the python kernel? I.e. something that does install the jupyter notebook & lab, but not the python kernel itself?
Also CC @NHDaly.
@davidanthoff: Thanks so much for putting work into this!!
First off, I'm sorry i haven't been a part of this conversation; I'm on vacation this week and last, and haven't been looking at a computer much. This looks really great though, thanks for your work!
Second apology: I'm sorry I never posted a PR for it, but I actually started some work on this a while ago but never posted it! You can see my take on it here:
In particular I wrote a python library for (b), because I couldn't find a python semver library either that worked how we want it to for Julia. But i'm also not a super python expert and actually getting someone to check this over is one of the main reasons I never opened a PR with these changes. :)
Sorry i don't have time to look at this PR for a couple days (still on vacation), but i will! In the meantime, hopefully looking at my work will help some! I THINK it's a reasonable python implementation of Julia's semver.
I think keeping support "for longer than is reasonable" should be the strategy for repo2docker given we are in the business of reproducible science. Once we have had support for repositories specifying the version of repo2docker they want to use to build we can get more aggressive/less conservative.
I wouldn't add the
After reading this one more time and pondering it I think we should name the build packs after the files that trigger them instead of versions because you can install Julia 1.0 and 1.1 via
I think adding
semver: I'd have to do a bit of googling as well, don't have a go to library for that. The implementation by @NHDaly in the not-yet-opened-PR seems like a good starting point. Especially if we turn all the assert statements into unit tests.
In one of the other PRs you also asked about build vs assemble scripts or some such?
(edit: semver comment updated)
@NHDaly I copied some of the code from your branch over here. BUT, I didn't just copy the whole julia_semver.py file in, because I don't want git to credit me with writing this code when really you did all the work :) Do you want to open a PR against this branch here that adds just that one file? If the API of it stays the same, it should hopefully work with the rest of this branch here.
There is also a test failure here that I need to figure out and fix, and then all the house keeping things that @betatim pointed out.
Ha, I figured out that I can commit something but change the author info for that commit :) So, now I copied the code from @NHDaly over into this PR, and he is the author for that commit.
I removed all the asserts because they should probably become tests. We can try to sort this out once things are actually functional.
Alright everyone, thanks so much for all the feedback! I think I addressed most of them.
Tests now pass and I think the best path forward would be to merge this at this stage and then do any additional improvements in further PRs.
A couple of issues that are still open that could fall into this bucket:
Maybe someone else could try to tackle these once we have things merged?
(edit by Tim to add "old Julia versions")
The remaining failed test is because the total coverage of the project will decrease if we merge this.
https://codecov.io/gh/jupyter/repo2docker/compare/e6cbf3e3a3e399508d5628b4ff1d30afed8c944f...c3d49966403b1f2b785450f86e33e49d7989e1db/diff shows which lines of the diff are covered and which aren't. I think it is basically the semver code so I'd be ok making an exception.
Thanks everyone for guiding me through this process and all the help :) And the vibe here is great!!
I'm wondering whether we should tackle #610 and #609 before we deploy... I think both have a very, very small potential for breaking things if folks rely on corner case behavior, so if we were to sort that out ASAP, we could probably just squeeze this in before anyone even has a chance to rely on any behavior that relates to the