julia_15: failures in the official test suite

[samuela]

Describe the bug
The julia_15 derivation does not pass the official julia test suite, including a number of BLAS errors where numerical results are incorrect.

This bug report is a followup to #117881 and in particular #117881 (comment).

To Reproduce
This bug seems to only manifest on certain hardware configurations. All of the results I'm presenting were run on an AWS c5.2xlarge instance running NixOS 20.09.3301.42809feaa9f (Nightingale).

Steps to reproduce the behavior:

1. nix-build -A julia_15 in the nixpkgs repository.
2. result/bin/julia --check-bounds=yes --startup-file=no --depwarn=error `result/bin/julia -e 'println(joinpath(Sys.BINDIR, Base.DATAROOTDIR, "julia/test/runtests.jl"))'`
3. Observe test failures.

Expected behavior
The test suite to pass.

Screenshots
n/a

Additional context
We know that the patched official binaries from julia upstream do not have the same test failures, so this appears to be an issue with nixpkgs version of blas or that the julia-custom BLAS patches are relevant and that we should be using their vendored blas in the build process.

As I proposed in #117881 (comment), I think we should add the test suite to run in the julia_15 derivation and then either mark it as broken or find a fix. Of course we can mark as broken and then always find a fix later and update it.

Notify maintainers

@raskin @rob @garrison

other interested parties: @ninjin @7c6f434c

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

❯ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.100, NixOS, 20.09.3301.42809feaa9f (Nightingale)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.10`
 - channels(root): `"nixos-20.09.3301.42809feaa9f, nix-ld"`
 - channels(skainswo): `"home-manager-20.09, nixpkgs-unstable-21.05pre282548.4c87cb87a2d"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: julia_15
# a list of nixos modules affected by the problem
module:

[samuela]

See also #118000 for a sense of what these errors look like.

[samuela]

After poking around at this for a few hours I am truly stumped. I began with the assumption that blas was to blame. So I proceeded by letting julia vendor in its own version of BLAS, and dropping the dependency on the nixpkgs blas. That didn't work, so I proceeded to one-by-one vendor in each dependency that has anything to do with numerics: blas, lapack, openlibm, mpfr, fftw, fftwSinglePrec, openspecfun. That leaves only libunwind, readline, utf8proc, zlib, curl, libgit2, and pcre2 as non-vendored dependencies. And yet still it fails the tests... Here's what I ended up with for anyone who's curious: https://gist.github.com/samuela/b50a0515fd70934f137518f7fc5e608f.

So I think we just mark julia_15 as broken and move on with our lives. @ninjin has a sick 1.6 derivation that does pass all the tests anyhow so people ought to just use that instead.

If anyone has any other ideas, do let me know! I'd be really curious to get to the bottom of this.

[ghost]

On Wed 28 Apr 2021, Samuel Ainsworth wrote: After poking around at this for a few hours I am truly stumped. I began with the assumption that `blas` was to blame. So I proceeded by letting julia vendor in its own version of BLAS, and dropping the dependency on the nixpkgs `blas`. That didn't work, so I proceeded to one-by-one vendor in each dependency that has anything to do with numerics: blas, lapack, openlibm, mpfr, fftw, fftwSinglePrec, openspecfun. That leaves only libunwind, readline, utf8proc, zlib, curl, libgit2, and pcre2 as non-vendored dependencies. And yet still it fails the tests... Here's what I ended up with for anyone who's curious: https://gist.github.com/samuela/b50a0515fd70934f137518f7fc5e608f. So I think we just mark `julia_15` as broken and move on with our lives. @ninjin has a sick 1.6 derivation that does pass all the tests anyhow so people ought to just use that instead. If anyone has any other ideas, do let me know! I'd be really curious to get to the bottom of this.

Odd, not seeing anything obvious apart from encouraging you to use `doCheckInstall` rather than `doCheck` as the former usually gets rid of a few spurious failures. If you do have a log I could have a look, although just like you I am inclined to think that we are better off leaving `julia_15` marked as broken and moving to correct the state of things with a “clean” `julia_16` expression. If you are truly dedicated to get to the bottom of this, you could hypothetically grab my Julia fork [1] and apply my BinaryBuilder patch on `master` to `release-1.5` to see if you can build the whole thing from source. Just beware that if you run `make testall` you are likely to get at least some failures for `Sockets` and friends depending on your network setup, but I guess we are mainly interested in the linear algebra portions anyway. [1]: https://sr.ht/~ninjin/julia-nix