Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: add fake hashes #53754

Merged
merged 1 commit into from
Jan 10, 2019
Merged

lib: add fake hashes #53754

merged 1 commit into from
Jan 10, 2019

Conversation

danbst
Copy link
Contributor

@danbst danbst commented Jan 10, 2019

Etalon fake hashes can be used as placeholders for all the places, where
Nix expression requires a hash, but we don't yet have one.

This should replace all current ways to do that:

  • echo|sha256sum, copy into clipboard, go to editor, paste into previously
    edited place
  • search nixpkgs for a random package, copy it's hash to cliboard, go to
    editor, paste into previously edited place

Nix can add support for etalon fake hashes. In that case printed error should contain
only 1 hash, so no more problem "which of two hashes from error should I use?"

Idea by irc:Synthetica

lib/trivial.nix Outdated
# Generated with `echo|XXXSum`
fakeMd5 = "68b329da9893e34099c7d8ad5cb9c940";
fakeSha256 = "01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b";
fakeSha512 = "be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just 0s?

@Synthetica9
Copy link
Member

Synthetica9 commented Jan 10, 2019

Maybe make them more recognisable in output? Something like all 0's or all 1's, so that you know that it isn't the hash you're looking for. (Even better would be a specialised error message that doesn't mention the place-holder, but that would probably be non-trivial to implement)

lib/trivial.nix Outdated
@@ -294,4 +294,11 @@ rec {
*/
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));

# Etalon fake hashes. Can be used as hash placeholders, when computing hash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is "Etalon"?

lib/trivial.nix Outdated
@@ -295,8 +295,7 @@ rec {
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));

# Etalon fake hashes. Can be used as hash placeholders, when computing hash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

French words were fine, but ok :)

@danbst
Copy link
Contributor Author

danbst commented Jan 10, 2019

thanks @grahamc @Synthetica9 , removed french word and replaced with 0s. Much better!

@grahamc
Copy link
Member

grahamc commented Jan 10, 2019

This looks great. Can you add something to https://nixos.org/nixpkgs/manual/#chap-quick-start about this, maybe instead of nix-prefetch-* ?

@7c6f434c
Copy link
Member

Does this change our stance about certificate verification in fetchers?

(not sure if this is more discoverable than 64a0 or its non-Vim equivalent…)

@Synthetica9
Copy link
Member

@7c6f434c for 64a0 to work, you need to remember the number of characters that each hash type takes.

@danbst
Copy link
Contributor Author

danbst commented Jan 10, 2019

@7c6f434c

Does this change our stance about certificate verification in fetchers?

can you explain a bit?

@edolstra
Copy link
Member

@7c6f434c That's not really affected by this PR, since it's already an issue whenever you do a nix-build with a fake hash.

@7c6f434c
Copy link
Member

@danbst Right now fetchurl source says «and work on SSL without a certificate (this isn't a security problem because we check the cryptographic hash of the output anyway)»; nix-prefetch-url does verify TLS certificate, and builtins.fetchurl does unless hash is specified.

(By the way, nix-prefetch-url supports '<nixpkgs> -A something.src which I got reminded about by checking Nix source of nix-prefetch-url — it is described in the Nix reference but not mentioned in the Nixpkgs manual)

@edolstra The idea of documenting fake hashes as the primary way of obtaining the correct hash is a bit of a last straw here.

@Synthetica9 Yes, but SHA-256 is 256 bits i.e. 64 4-bit nibbles, SHA-512 is 512 bits i.e. 128 nibbles, and MD5 is either something shorter (i.e. 32 nibbles, which is correct) or a reason to reconsider the idea of adding a new MD5-hashed entry…

@Synthetica9
Copy link
Member

lib/trivial.nix Outdated
@@ -294,4 +294,9 @@ rec {
*/
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));

# Fake hashes. Can be used as hash placeholders, when computing hash ahead isn't trivial
fakeMd5 = "00000000000000000000000000000000";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's delete the md5 option. People shouldn't use it anymore.

@danbst
Copy link
Contributor Author

danbst commented Jan 10, 2019

I agree with @7c6f434c , this shouldn't be "official". It also doesn't belong to #chap-quick-start, newcomers shouldn't learn hacks from start. The advice to use nix-prefetch-url is perfectly fine there.

@grahamc we can spread word on wiki though. It's more targeted for "useful hacks".

@edolstra
Copy link
Member

I'm not sure whether lib is the right place for useful hacks though. (Maybe lib/deprecated.nix?)

@danbst
Copy link
Contributor Author

danbst commented Jan 10, 2019

@edolstra done

@7c6f434c @grahamc
I think what misses in manual is how various ways of getting a hash interfere with security concerns.
I know some people didn't like that Nix hash has to be different from upstream one in some cases. If we point out all security issues, then we can include lib.fakeSha256 as an example. I'll create a separate doc PR

lib/trivial.nix Outdated
@@ -294,4 +294,5 @@ rec {
*/
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should not touch this file if it's going in misc

Fake hashes can be used as placeholders for all the places, where
Nix expression requires a hash, but we don't yet have one.

This should be more convenient than following:
- echo|sha256sum, copy into clipboard, go to editor, paste into previously
  edited place
- search nixpkgs for a random package, copy it's hash to cliboard, go to
  editor, paste into previously edited place

Nix can add support for these fake hashes. In that case printed error should contain
only 1 hash, so no more problem "which of two hashes from error should I use?"

Idea by irc:Synthetica
@Mic92 Mic92 changed the title lib: add étalon fake hashes lib: add fake hashes Jan 10, 2019
@Mic92 Mic92 merged commit b75aff7 into NixOS:master Jan 10, 2019
@danbst danbst deleted the lib-fake-hashes branch January 10, 2019 18:02
@joepie91
Copy link
Contributor

I agree with @7c6f434c , this shouldn't be "official". It also doesn't belong to #chap-quick-start, newcomers shouldn't learn hacks from start. The advice to use nix-prefetch-url is perfectly fine there.

So what is the correct way to do this, and where is it documented? And is it actually more ergonomic than this "hack"?

@danbst
Copy link
Contributor Author

danbst commented Jan 14, 2019

@joepie91 #53785

@joepie91
Copy link
Contributor

Thanks. I feel like there should probably be a more streamlined utility for this purpose, though... one that can deal with all source variants automatically in a single command. Right now, using an all-zeroes hash is still the easiest "just works" method :/

@7c6f434c
Copy link
Member

I think I am generally for fetch* doing certificate validation by default with an option to switch off, and maybe a global config option to switch that off.

Any drawbacks?

@danbst
Copy link
Contributor Author

danbst commented Jan 14, 2019

Don't see any. I imagine only a situation, when system hadn't been updated for ages, and all certificates invalidated. Trying to upgrade stuff in this scenario may not work (but should work with current solution).

Even if it is done, not every url is https, so fake-hash method usage still is insecure when applied blindly.

@7c6f434c
Copy link
Member

7c6f434c commented Jan 14, 2019 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants