common-updater-scripts: incorrect handling of latest nix's hash mismatch messages

[dtzWill]

Issue description

When used with latest Nix the "new" hash is extracted incorrectly, sha256 = "got: sha256:hashgoeshere or thereabouts. This seems to be due to Nix changing the message used to be more friendly to humans:

NixOS/nix#2574

Steps to reproduce

Use an update script invoking common-updater-scripts. Sorry, LMK if repro is needed.

Technical details

Please run nix-shell -p nix-info --run "nix-info -m" and paste the
results.

I don't expect this to be helpful but happy to provide anyway!

• system: "x86_64-linux"
• host os: Linux 4.20.5, NixOS, 19.03.git.06b6bb8 (Koi)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3dtz6739_232d967
• channels(will): "nur-nopkgs, testing, unstable-19.03pre166987.bc41317e243, nixos-1609-16.09.1943.25f4906da6"
• channels(root): "nixos-18.09.1922.97e0d53d669"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

[jtojnar]

--- a/pkgs/common-updater/scripts/update-source-version
+++ b/pkgs/common-updater/scripts/update-source-version
@@ -96,7 +96,7 @@ fi
 if [ -z "$newHash" ]; then
     nix-build --no-out-link -A "$attr.src" 2>"$attr.fetchlog" >/dev/null || true
     # FIXME: use nix-build --hash here once https://github.com/NixOS/nix/issues/1172 is fixed
-    newHash=$(egrep -v "killing process|dependencies couldn't be built" "$attr.fetchlog" | tail -n2 | sed "s~output path .* has .* hash ‘\(.*\)’ when .* was expected\|fixed-output derivation produced path '.*' with .* hash '\(.*\)' instead of the expected hash '.*'~\1\2~" | head -n1)
+    newHash=$(egrep -v "killing process|dependencies couldn't be built|wanted: " "$attr.fetchlog" | tail -n2 | sed "s~output path .* has .* hash ‘\(.*\)’ when .* was expected\|fixed-output derivation produced path '.*' with .* hash '\(.*\)' instead of the expected hash '.*'\|  got:    .*:\(.*\)~\1\2\3~" | head -n1)
 fi
 
 if [ -z "$newHash" ]; then

It is increasingly messy, perhaps we can drop the old regexes, most people probably set nix.package instead of changing nix in overlays so this would not be affected.

[typetetris]

What about adding a flag to nix, which turns on machine readable error messages with a guaranty that those messages will not change in a backward incompatible way?
Would need someone to add that.
But wouldn't that be better to parse error messages intended for human readers with shell tools?

[jtojnar]

Alternatively, if the build fails, I think we could do something like:

nix-hash --flat --base32 --type sha256 $(nix-instantiate -A "$attr.src" --json --eval --strict)

[dtzWill]

On Thu, 31 Jan 2019 09:21:25 -0800, Jan Tojnar ***@***.***> wrote: ``` --- a/pkgs/common-updater/scripts/update-source-version +++ b/pkgs/common-updater/scripts/update-source-version @@ -96,7 +96,7 @@ fi if [ -z "$newHash" ]; then nix-build --no-out-link -A "$attr.src" 2>"$attr.fetchlog" >/dev/null || true # FIXME: use nix-build --hash here once NixOS/nix#1172 is fixed - newHash=$(egrep -v "killing process|dependencies couldn't be built" "$attr.fetchlog" | tail -n2 | sed "s~output path .* has .* hash ‘\(.*\)’ when .* was expected\|fixed-output derivation produced path '.*' with .* hash '\(.*\)' instead of the expected hash '.*'~\1\2~" | head -n1) + newHash=$(egrep -v "killing process|dependencies couldn't be built|wanted: " "$attr.fetchlog" | tail -n2 | sed "s~output path .* has .* hash ‘\(.*\)’ when .* was expected\|fixed-output derivation produced path '.*' with .* hash '\(.*\)' instead of the expected hash '.*'\| got: .*:\(.*\)~\1\2\3~" | head -n1) fi if [ -z "$newHash" ]; then ``` It is increasingly messy, perhaps we can drop the old regexes, most people probably set `nix.package` instead of changing nix in overlays so this would not be affected.

I'm not sure I follow, but FWIW I ran into this and I set `nix.package` and don't set it via overlay. Looks like the new behavior is in 2.2, the current default for `nix`: ``` $ nix run -f channel:nixos-unstable nix -i -c nix eval '(builtins.fetchurl { url = https://nixos.org; sha256 = "1111111111111111111111111111111111111111111111111111"; })' error: hash mismatch in file downloaded from 'https://nixos.org': wanted: sha256:1111111111111111111111111111111111111111111111111111 got: sha256:1q7rksf170b1zhy54b5cwig77ycgdiaibcbh47aq8rrw3szra24w ``` (err formatting might not work for email responses, maybe? sorry :)) I think it's reasonable to drop support for nix older than what's in the tree-- especially since you'd have to overlay to replace it and as you say would likely do via `nix.package` anyway. Oh... that's probably what you meant! :D

…

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #54962 (comment) part: text/html

[dtzWill]

nix-hash --flat --base32 --type sha256 $(nix-instantiate -A "$attr.src" --json --eval --strict)

Can we do that on a path that isn't valid? I was going to try nix hash-path or so but basically they all have the same problem :). Maybe we should just add the extra sed 😇.

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[siraben]

Any updates on this?

[stale]

I marked this as stale due to inactivity. → More info

[FliegendeWurst]

Should have been fixed by #55605

[dtzWill]

Thanks!