New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: add fake hashes #53754

Merged
merged 1 commit into from Jan 10, 2019

Conversation

Projects
None yet
8 participants
@danbst
Copy link
Contributor

danbst commented Jan 10, 2019

Etalon fake hashes can be used as placeholders for all the places, where
Nix expression requires a hash, but we don't yet have one.

This should replace all current ways to do that:

  • echo|sha256sum, copy into clipboard, go to editor, paste into previously
    edited place
  • search nixpkgs for a random package, copy it's hash to cliboard, go to
    editor, paste into previously edited place

Nix can add support for etalon fake hashes. In that case printed error should contain
only 1 hash, so no more problem "which of two hashes from error should I use?"

Idea by irc:Synthetica

@danbst danbst requested review from edolstra and nbp as code owners Jan 10, 2019

# Generated with `echo|XXXSum`
fakeMd5 = "68b329da9893e34099c7d8ad5cb9c940";
fakeSha256 = "01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b";
fakeSha512 = "be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09";

This comment has been minimized.

@grahamc

grahamc Jan 10, 2019

Member

How about just 0s?

@Synthetica9

This comment has been minimized.

Copy link
Contributor

Synthetica9 commented Jan 10, 2019

Maybe make them more recognisable in output? Something like all 0's or all 1's, so that you know that it isn't the hash you're looking for. (Even better would be a specialised error message that doesn't mention the place-holder, but that would probably be non-trivial to implement)

@@ -294,4 +294,11 @@ rec {
*/
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));

# Etalon fake hashes. Can be used as hash placeholders, when computing hash

This comment has been minimized.

@grahamc

grahamc Jan 10, 2019

Member

What is "Etalon"?

@@ -295,8 +295,7 @@ rec {
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));

# Etalon fake hashes. Can be used as hash placeholders, when computing hash

This comment has been minimized.

@grahamc

grahamc Jan 10, 2019

Member

French words were fine, but ok :)

@danbst

This comment has been minimized.

Copy link
Contributor Author

danbst commented Jan 10, 2019

thanks @grahamc @Synthetica9 , removed french word and replaced with 0s. Much better!

@grahamc

This comment has been minimized.

Copy link
Member

grahamc commented Jan 10, 2019

This looks great. Can you add something to https://nixos.org/nixpkgs/manual/#chap-quick-start about this, maybe instead of nix-prefetch-* ?

@7c6f434c

This comment has been minimized.

Copy link
Member

7c6f434c commented Jan 10, 2019

Does this change our stance about certificate verification in fetchers?

(not sure if this is more discoverable than 64a0 or its non-Vim equivalent…)

@Synthetica9

This comment has been minimized.

Copy link
Contributor

Synthetica9 commented Jan 10, 2019

@7c6f434c for 64a0 to work, you need to remember the number of characters that each hash type takes.

@danbst

This comment has been minimized.

Copy link
Contributor Author

danbst commented Jan 10, 2019

@7c6f434c

Does this change our stance about certificate verification in fetchers?

can you explain a bit?

@edolstra

This comment has been minimized.

Copy link
Member

edolstra commented Jan 10, 2019

@7c6f434c That's not really affected by this PR, since it's already an issue whenever you do a nix-build with a fake hash.

@7c6f434c

This comment has been minimized.

Copy link
Member

7c6f434c commented Jan 10, 2019

@danbst Right now fetchurl source says «and work on SSL without a certificate (this isn't a security problem because we check the cryptographic hash of the output anyway)»; nix-prefetch-url does verify TLS certificate, and builtins.fetchurl does unless hash is specified.

(By the way, nix-prefetch-url supports '<nixpkgs> -A something.src which I got reminded about by checking Nix source of nix-prefetch-url — it is described in the Nix reference but not mentioned in the Nixpkgs manual)

@edolstra The idea of documenting fake hashes as the primary way of obtaining the correct hash is a bit of a last straw here.

@Synthetica9 Yes, but SHA-256 is 256 bits i.e. 64 4-bit nibbles, SHA-512 is 512 bits i.e. 128 nibbles, and MD5 is either something shorter (i.e. 32 nibbles, which is correct) or a reason to reconsider the idea of adding a new MD5-hashed entry…

@Synthetica9

This comment has been minimized.

@@ -294,4 +294,9 @@ rec {
*/
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));

# Fake hashes. Can be used as hash placeholders, when computing hash ahead isn't trivial
fakeMd5 = "00000000000000000000000000000000";

This comment has been minimized.

@grahamc

grahamc Jan 10, 2019

Member

Let's delete the md5 option. People shouldn't use it anymore.

@danbst

This comment has been minimized.

Copy link
Contributor Author

danbst commented Jan 10, 2019

I agree with @7c6f434c , this shouldn't be "official". It also doesn't belong to #chap-quick-start, newcomers shouldn't learn hacks from start. The advice to use nix-prefetch-url is perfectly fine there.

@grahamc we can spread word on wiki though. It's more targeted for "useful hacks".

@edolstra

This comment has been minimized.

Copy link
Member

edolstra commented Jan 10, 2019

I'm not sure whether lib is the right place for useful hacks though. (Maybe lib/deprecated.nix?)

@danbst danbst force-pushed the danbst:lib-fake-hashes branch from 04c8405 to 3642c3d Jan 10, 2019

@danbst

This comment has been minimized.

Copy link
Contributor Author

danbst commented Jan 10, 2019

@edolstra done

@7c6f434c @grahamc
I think what misses in manual is how various ways of getting a hash interfere with security concerns.
I know some people didn't like that Nix hash has to be different from upstream one in some cases. If we point out all security issues, then we can include lib.fakeSha256 as an example. I'll create a separate doc PR

@@ -294,4 +294,5 @@ rec {
*/
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));

This comment has been minimized.

@Synthetica9

Synthetica9 Jan 10, 2019

Contributor

Probably should not touch this file if it's going in misc

lib: add shortcuts for fake hashes (fakeSha256, fakeSha512)
Fake hashes can be used as placeholders for all the places, where
Nix expression requires a hash, but we don't yet have one.

This should be more convenient than following:
- echo|sha256sum, copy into clipboard, go to editor, paste into previously
  edited place
- search nixpkgs for a random package, copy it's hash to cliboard, go to
  editor, paste into previously edited place

Nix can add support for these fake hashes. In that case printed error should contain
only 1 hash, so no more problem "which of two hashes from error should I use?"

Idea by irc:Synthetica

@danbst danbst force-pushed the danbst:lib-fake-hashes branch from 3642c3d to 68a6b47 Jan 10, 2019

@Mic92 Mic92 changed the title lib: add étalon fake hashes lib: add fake hashes Jan 10, 2019

@Mic92 Mic92 merged commit b75aff7 into NixOS:master Jan 10, 2019

9 checks passed

grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details

@danbst danbst deleted the danbst:lib-fake-hashes branch Jan 10, 2019

@joepie91

This comment has been minimized.

Copy link
Contributor

joepie91 commented Jan 14, 2019

I agree with @7c6f434c , this shouldn't be "official". It also doesn't belong to #chap-quick-start, newcomers shouldn't learn hacks from start. The advice to use nix-prefetch-url is perfectly fine there.

So what is the correct way to do this, and where is it documented? And is it actually more ergonomic than this "hack"?

@danbst

This comment has been minimized.

Copy link
Contributor Author

danbst commented Jan 14, 2019

@joepie91

This comment has been minimized.

Copy link
Contributor

joepie91 commented Jan 14, 2019

Thanks. I feel like there should probably be a more streamlined utility for this purpose, though... one that can deal with all source variants automatically in a single command. Right now, using an all-zeroes hash is still the easiest "just works" method :/

@7c6f434c

This comment has been minimized.

Copy link
Member

7c6f434c commented Jan 14, 2019

I think I am generally for fetch* doing certificate validation by default with an option to switch off, and maybe a global config option to switch that off.

Any drawbacks?

@danbst

This comment has been minimized.

Copy link
Contributor Author

danbst commented Jan 14, 2019

Don't see any. I imagine only a situation, when system hadn't been updated for ages, and all certificates invalidated. Trying to upgrade stuff in this scenario may not work (but should work with current solution).

Even if it is done, not every url is https, so fake-hash method usage still is insecure when applied blindly.

@7c6f434c

This comment has been minimized.

Copy link
Member

7c6f434c commented Jan 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment