updating to v1.3.0: Unable to automatically install 'OpenSpecFun'

I having difficulties in updating packages to v1.3.0 and @fredrikekre suggested I file an issue here. Here's the beginning of the stack trace:

  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed Plots ─ v0.28.0
┌ Error: Tree Hash Mismatch!
│   Expected git-tree-sha1:   d322d52dbe634d659d67312eea8a78c16112127e
│   Calculated git-tree-sha1: 0aee4fdfd24ee5162c06e2507d82c11a8f6a9ad2
└ @ Pkg.Artifacts /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Artifacts.jl:724
ERROR: Unable to automatically install 'OpenSpecFun' from '/home/healyp/.julia/packages/OpenSpecFun_jll/XrUb6/Artifacts.toml'
Stacktrace:
 [1] error(::String) at ./error.jl:33

On suggestion of @giordano, versinoinfo() gives

julia> versioninfo()
Julia Version 1.3.0
Commit 46ce4d7933 (2019-11-26 06:09 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, sandybridge)
Environment:
  JULIAPATH = .:/home/healyp/src/julia-1.3.0/bin/
  JULIA_PKGDIR = /home/healyp/.julia/
  JULIA_NUM_THREADS = 8

All suggestions appreciated.

Many thanks.

[giordano]

What's the output of

using Pkg.BinaryPlatforms
platform_key_abi()

I think I have your same platform and the hash is correct for me. Maybe a corrupted download? But that would have had a mismatch in the hash of the tarball

[fredrikekre]

Might be possible that the hash is wrong in the Artifacts file.

[giordano]

Might be possible that the hash is wrong in the Artifacts file.

I assume I'd have had the same error installing SpecialFunctions.jl in an empty environment, no?

[fredrikekre]

Thats true, maybe we just compute it wrong? But I assume the same function is used when creating the artifacts file as when unpacking the tarball.

julia> platform_key_abi()
Linux(:x86_64, libc=:glibc, compiler_abi=CompilerABI(libgfortran_version=v"4.0.0", cxxstring_abi=:cxx11))

[giordano]

No, ok, your platform is x86_64-linux-gnu-libgfortran4, I have libgfortran5 instead of 4. There is then the possibility that the hash is wrong in Artifacts.toml

Might be possible that the hash is wrong in the Artifacts file.

I assume I'd have had the same error installing SpecialFunctions.jl in an empty environment, no?

I should say that this same difficulty comes up in both cases, both upgrading from v1.1.1 and doing a first-time install on another machine.

Thanks,
PH

[fredrikekre]

On my machine all the hashes look correct;

using Pkg
Pkg.PlatformEngines.probe_platform_engines!()

toml = read(`curl -sL https://raw.githubusercontent.com/JuliaBinaryWrappers/OpenSpecFun_jll.jl/master/Artifacts.toml`);
artifacts = Pkg.TOML.parse(IOBuffer(toml))
for a in artifacts["OpenSpecFun"]
    for dl in a["download"]
        dir = mktempdir()
        @assert Pkg.PlatformEngines.download_verify_unpack(dl["url"], dl["sha256"], dir; ignore_existence=true, verbose=true)
        treehash = bytes2hex(Pkg.GitTools.tree_hash(dir))
        treehash == a["git-tree-sha1"] || error("hash mismatch!")
    end
end

passes. Does that error for you?

[PhilipLoewen]

I see the same error message, with the same expected and calculated hash values, as the OP.

Running the code suggested by @fredrikekre above produces an interesting result:

           for dl in a["download"]
               dir = mktempdir()
               @assert Pkg.PlatformEngines.download_verify_unpack(dl["url"], dl["sha256"], dir; ignore_existence=true, verbose=true)
               treehash = bytes2hex(Pkg.GitTools.tree_hash(dir))
               treehash == a["git-tree-sha1"] || error("hash mismatch!")
           end
       end
[ Info: Downloading https://github.com/JuliaBinaryWrappers/OpenSpecFun_jll.jl/releases/download/OpenSpecFun-v0.5.3+1/OpenSpecFun.v0.5.3.powerpc64le-linux-gnu-libgfortran3.tar.gz to /tmp/jl_y2QFE6-download.gz...
######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash 964c133100e41f1b99c11d2177e78ebb72523ec154118c70ca25bbc51b81d3e1 for file /tmp/jl_y2QFE6-download.gz
[ Info: Unpacking /tmp/jl_y2QFE6-download.gz into /tmp/jl_UN4hrU...
ERROR: hash mismatch!
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] top-level scope at ./REPL[6]:6

At least, I think it's interesting because (I am unfamiliar with the basics, but) the downloaded item seems to have "powerpc64le" in its name, whereas my CPU comes from Intel. Details:

Julia Version 1.3.0
Commit 46ce4d7933 (2019-11-26 06:09 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)

I hope there is something relevant in here. Thanks.

On my machine all the hashes look correct;
:
passes. Does that error for you?

Yes. There's a hash mismatch. The end of the trace:

--2019-11-28 09:50:35--  https://github.com/JuliaBinaryWrappers/OpenSpecFun_jll.jl/releases/download/OpenSpecFun-v0.5.3+1/OpenSpecFun.v0.5.3.powerpc64le-linux-gnu-libgfortran3.tar.gz
Resolving github.com (github.com)... 140.82.118.4
Connecting to github.com (github.com)|140.82.118.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/205283016/20afff80-d9f9-11e9-8611-37a93729c4cf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191128%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191128T095035Z&X-Amz-Expires=300&X-Amz-Signature=e4f42b31dd8f0a51568cb309490f62169913a88deead0ab2643dbf63bd962596&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3DOpenSpecFun.v0.5.3.powerpc64le-linux-gnu-libgfortran3.tar.gz&response-content-type=application%2Foctet-stream [following]
--2019-11-28 09:50:35--  https://github-production-release-asset-2e65be.s3.amazonaws.com/205283016/20afff80-d9f9-11e9-8611-37a93729c4cf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191128%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191128T095035Z&X-Amz-Expires=300&X-Amz-Signature=e4f42b31dd8f0a51568cb309490f62169913a88deead0ab2643dbf63bd962596&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3DOpenSpecFun.v0.5.3.powerpc64le-linux-gnu-libgfortran3.tar.gz&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.36.20
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.36.20|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 205399 (201K) [application/octet-stream]
Saving to: ‘/tmp/jl_9bkc5j-download.gz’

/tmp/jl_9bkc5j-down 100%[===================>] 200.58K   143KB/s    in 1.4s    

2019-11-28 09:50:37 (143 KB/s) - ‘/tmp/jl_9bkc5j-download.gz’ saved [205399/205399]

[ Info: No hash cache found
[ Info: Calculated hash 964c133100e41f1b99c11d2177e78ebb72523ec154118c70ca25bbc51b81d3e1 for file /tmp/jl_9bkc5j-download.gz
[ Info: Unpacking /tmp/jl_9bkc5j-download.gz into /tmp/jl_glFMHm...
ERROR: hash mismatch!
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] top-level scope at ./REPL[15]:6

julia> 

[fredrikekre]

At least, I think it's interesting because (I am unfamiliar with the basics, but) the downloaded item seems to have "powerpc64le" in its name, whereas my CPU comes from Intel. Details:

The snippet there downloads all different versions.

So it looks like we compute the hash wrong then, alternatively that your unpacking engines do something weird.

[fredrikekre]

What unpack engine do you get from:

julia> Pkg.PlatformEngines.probe_platform_engines!()

julia> Pkg.PlatformEngines.gen_unpack_cmd("file.tar.gz", "/tmp/file")
`tar -xzf file.tar.gz -C/tmp/file`

Edit:

$ tar --version
tar (GNU tar) 1.28

[giordano]

For the record, I don't have any hash mismatch and get the same unpack command as Fredrik

julia> Pkg.PlatformEngines.gen_unpack_cmd("file.tar.gz", "/tmp/file")
`tar -xzf file.tar.gz -C/tmp/file`

and

$ tar --version
tar (GNU tar) 1.30

This is the version of tar that is running on /both/ machines where I'm encountering this difficulty.

[giordano]

% tar --version
tar (GNU tar) 1.32