Too many old Julia versions, 1.5 is not packaged, and all versions fail in tests

[doronbehar]

Julia 1.5 is out, see: https://docs.julialang.org/en/v1/NEWS/ and we don't have it packaged. More over, see how all-packages.nix defines our other Julias:

nixpkgs/pkgs/top-level/all-packages.nix

Lines 8935 to 8956 in fdfa609

	julia_07 = callPackage ../development/compilers/julia/0.7.nix {
	gmp = gmp6;
	inherit (darwin.apple_sdk.frameworks) CoreServices ApplicationServices;
	};
	julia_10 = callPackage ../development/compilers/julia/1.0.nix {
	gmp = gmp6;
	inherit (darwin.apple_sdk.frameworks) CoreServices ApplicationServices;
	};
	julia_11 = callPackage ../development/compilers/julia/1.1.nix {
	gmp = gmp6;
	inherit (darwin.apple_sdk.frameworks) CoreServices ApplicationServices;
	};
	julia_13 = callPackage ../development/compilers/julia/1.3.nix {
	gmp = gmp6;
	inherit (darwin.apple_sdk.frameworks) CoreServices ApplicationServices;
	};
	julia_1 = julia_10;
	julia = julia_1;

Several issues: julia is pointing to a TLS version which is old IMO. Certainly there are breaking changes seen in their NEWS pages, but since Nixpkgs is a somewhat rolling release distro, are we going to keep these older versions and default to them for ever? I've once tried to reach for the community to hear if there's a strict policy regarding such scenarios, but I think the answer is no

Anyway, I'd like to propose a change that will keep only the LTS version and the latest - currently 1.4, and perhaps make the attribute julia, point to the latest.

Any thoughts from maintainers or users?

cc @raskin @rob @garrison

[doronbehar]

Also, now I noticed julia_13 all of our Julias fail to build, e.g: https://hydra.nixos.org/build/123240981/nixlog/1

[garrison]

Anyway, I'd like to propose a change that will keep only the LTS version and the latest - currently 1.4, and perhaps make the attribute julia, point to the latest.

I agree. And the 1.5 release candidates are out, so this will become the latest version in the near future.

Last time I looked, the package built the same deps for all versions, including 0.7. In versions 1.0 and later, there are actually fewer dependencies (e.g. arpack and fftw have been dropped), so the package should be cleaned up to not include these unnecessary libraries in the build.

I also think the default version should be the latest version (i.e., unless somebody explicitly installs julia-1-lts, they should get the latest).

[doronbehar]

I also think the default version should be the latest version (i.e., unless somebody explicitly installs julia-1-lts, they should get the latest).

OK I'm glad I have your support. I'll let this issue stay open for a while and hopefully I'll be able to take care of this a month or so from now.

[veprbl]

cc @7c6f434c

[7c6f434c]

Back when Julia tests were not so messed up, I think there was often 1 previous version.

I guess with 1.3 needing overriding to no tests, it is fine to just have julia-1-lts and Julia 1.4 as julia once Julia 1.4 gets built fine with some tests.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-can-we-build-a-julia-derivation-with-tests-disabled/8052/2

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-can-we-build-a-julia-derivation-with-tests-disabled/8052/3

[rbvermaa]

Haven't had a lot of time to work on julia in nixpkgs, so just posting this as a FYI.

We used multiple versions of julia at work, but we use a custom derivation, similar to this: https://gist.github.com/rbvermaa/08f916a63ba4f3f6123f97478309dfa0

We also do not run the tests, but in general it has been working great.

Previously we also used the binary builds patched with patchelf, as there were some differences in the structure of the results of the vanilla binary build provided by julia and what we got when building from source. We gave up on using 'system' libraries, and build the -full.tar.gz releases that julia provides, as julia is very specific in the versions for its dependencies (LLVM patches mostly).

[doronbehar]

@rbvermaa I tried to use that -full.tar.gz tarball in order to write a derivation from scratch. Here's my attempt:

https://github.com/doronbehar/nixpkgs/blob/julia-latest/pkgs/development/compilers/julia/default.nix

It fails with:

Could not find working curl, wget, or fetch.
You need to install one of these to download dependencies.
make[1]: *** [/build/julia-1.4.2/deps/suitesparse.mk:144: /build/julia-1.4.2/deps/srccache/SuiteSparse.v5.4.0-2.x86_64-linux-gnu.tar.gz] Error 1
make: *** [Makefile:60: julia-deps] Error 2

Naturally I didn't further attempt to add curl or wget to the inputs, as that would fail in a sandbox as well. I've opened an issue upstream: JuliaLang/julia#36604 complaining that that -full.tar.gz tarball doesn't really contain all dependencies.

[rbvermaa]

@doronbehar I ran with sandbox enabled, and the expression I linked with works for me. So I think it is fine to just add curl to the deps, as with the full tarball, it isn't actually used.