memtest86-efi: 8.4 -> 9.3.1000

[TredwellGit]

Things done

• Built on platform(s)
  • x86_64-linux
• Tested basic functionality of all binary files (usually in ./result/bin/)
• Fits CONTRIBUTING.md.

[cdepillabout]

What's up with this?

I don't know anything about IPFS, but is this a reasonably reliable way to download this file? How does this memtest86-usb.zip file get on these IPFS servers?

I don't see any other uses of IPFS in Nixpkgs, so I'm worried about merging this. I do like that this appears to fix the problem with the company not providing versioned zip files.

[cdepillabout]

There does also appear to be a https://www.memtest86.com/downloads/memtest86-9.2.2000-usb.zip version that is available to download without needing IPFS.

I guessed this URL from the version numbers on https://www.memtest86.com/whats-new.html (thanks for this link btw).

[TredwellGit]

See https://docs.ipfs.io/concepts/what-is-ipfs/
Those are IPFS gateways, not servers: https://docs.ipfs.io/concepts/ipfs-gateway/
I am pinning the file so it stays available: https://docs.ipfs.io/how-to/pin-files/
It is more reliable than using the MemTest86 website because the file is hosted by anyone that has the file and we can use any public gateway. I picked all the modern gateways from https://ipfs.github.io/public-gateway-checker/ and ordered them by throughput.

[cdepillabout]

Wow, the the IPFS documentation is sure nice. A bunch of work must have went in to making that simple and easy-to-understand.

---

My main concerns with this are the following:

1. When it comes time to update to a new version, I won't be able to do it, because I don't use IPFS.

2. It is more reliable than using the MemTest86 website because the file is hosted by anyone that has the file and we can use any public gateway.

   From reading https://docs.ipfs.io/how-to/pin-files/, it appears that now the zip file is just hosted by you (and anyone that downloads it from you). It is entirely possible that you stop using IPFS/Nix and the file just vanishes from IPFS?

   It seems like downloading it from the company's website could be more future-proof.

Here's my suggestions of how we should proceed (in order of my preference):

1. Continue to just use one version behind the current version, since the company provides versioned zip files for old versions. For instance, the current latest version is 9.3.1000, but there is no https://www.memtest86.com/downloads/memtest86-9.3-1000-usb.zip. However, there is a zip file for the previous version available: https://www.memtest86.com/downloads/memtest86-9.2.2000-usb.zip
2. Switch to using some mirror from another Linux distribution. I haven't checked, but I wouldn't be surprised if another Linux distribution has a mirror of files like this.
3. Just upload the zip files to a repo on GitHub and use that as the source. The zip files are less than 10 mb, so hopefully this won't be a problem.

Just in case, I'll ask on Discourse to see how other people feel about using IPFS as a source here in Nixpkgs.

[ajs124]

Just as a note, nixpkgs often just uses web.archive.org for these kinds of files, if they're on there.

I don't really have an opinion on if that's better or worse than fetching from ipfs, just saying that that's what I've seen in nixpkgs before.

[L-as]

I think this is a much better solution.

[cdepillabout]

@TredwellGit

Okay, it seems like there isn't really anyone supporting using IPFS as a source in https://discourse.nixos.org/t/should-ipfs-be-used-as-a-source-for-fetchurl-in-nixpkgs/16312/1, and using IPFS means that I won't be able to update this in the future, so I won't merge this in as-is.

Although, if you wanted to change this PR to use one of the alternatives I listed in #147574 (comment), I'd be happy to merge this in.

---

Also, it seemed like a lot of people thought that #147574 (comment) would be really neat.

[TredwellGit]

I don't understand the objection, though, since it can always be changed the to use something like the Internet Archive or GitHub on the next release if convenient. I also don't understand why IPFS is not acceptable but the Internet Archive and GitHub are.

The idea that I had about IPFS fallback is something that we could do in theory, but is significantly more work and out of scope of this PR.

[cdepillabout]

I don't understand the objection, though, since it can always be changed the to use something like the Internet Archive or GitHub on the next release if convenient.

My main objections are:

1. If we will eventually revert this PR in a future release, then we should just go to using the alternative right now. We shouldn't knowingly merge in something that will create more work in the future to revert.
2. Unless it is documented well, moving to IPFS will discourage some people from updating the derivation, since it will seem like you have to use IPFS to do the update. I imagine not all Nixpkgs maintainers are using IPFS.
3. If it is documented well, it will make updating this derivation more complicated. I think this is an unnecessary complication to some extent, because there are easier alternatives I've listed in memtest86-efi: 8.4 -> 9.3.1000 #147574 (comment).

I also don't understand why IPFS is not acceptable but the Internet Archive and GitHub are.

My own thoughts and a summary of the discourse thread:

1. If you're a programmer, I'm sure you can appreciate that people hate when new tools are forced into their workflow. Moving ahead with using IPFS will mean that people need to use IPFS for updating this derivation. I imagine there are many Nixpkgs maintainers that don't know IPFS (myself included). I don't think it is surprising you would get push-back from people who don't want new tools introduced into an existing workflow. (You can of course make the argument that it is fine to go back to not using IPFS in a future update, but if that is the case, then we should do that now and not use IPFS in the first place.)
2. There is some worry about trust. GitHub is a big organization and already widely used and trusted in Nixpkgs. IPFS isn't already widely used and trusted in Nixpkgs. There is also a worry about trusting that people will keep this derivation pinned.

[TredwellGit]

I changed it to use the Internet Archive because it is more important for this to be updated, but I still think that IPFS is the better choice. Part of the motivation behind IPFS is to help avoid dependency on big organizations and to make it so that you don't need to trust file hosts.

I hope to promote better workflows than status quo and feel like this is not that that much of a complication, but so be it: curl -Lf https://www.memtest86.com/downloads/memtest86-usb.zip | ipfs add --cid-version 1 -w --stdin-name memtest86-usb.zip

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/should-ipfs-be-used-as-a-source-for-fetchurl-in-nixpkgs/16312/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/should-ipfs-be-used-as-a-source-for-fetchurl-in-nixpkgs/16312/3

[cdepillabout]

Could you add more of an explanation here?

In particular, exactly what a new user should do to go from a link to https://www.memtest86.com/downloads/memtest86-usb.zip to a web.archive.org link?

[aanderse]

FYI - It looks like memtest86-efi is broken on stable now. Maybe this should be backported.

[github-actions]

Successfully created backport PR #160490 for release-21.11.