New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Signed and certified packages #29

Open
Ginden opened this Issue Jan 16, 2015 · 47 comments

Comments

Projects
None yet
@Ginden

Ginden commented Jan 16, 2015

It should be possible for require function to validate required packages for their security.

Assume I'm running company that uses io.js for something important. We can't allow our programmists to use potentially dangerous (because of author pushing malicious package to npm or something) packages. These packages can be hidden deep in package dependencies.

I think it should be allowed to certify ("We, Code Review Company, guarantees it's valid and secure package") and sign packages using PGP keys. Moreover - it should be possible to have parameter (flag?) that allows requiring only safe packages (we have theirs public keys) and throws error when something requires potentially dangerous package.

It can result in increased adoption of io.js as trustworthy platfom.

@KenanY

This comment has been minimized.

Show comment
Hide comment
@KenanY

KenanY Jan 16, 2015

Member

I don't think require should be in charge of this. This seems more like an issue for npm, and in fact there are a couple of entries on the issue tracker regarding this (the integrity part of your request, at least): npm/npm#3252, npm/npm#4016.

Member

KenanY commented Jan 16, 2015

I don't think require should be in charge of this. This seems more like an issue for npm, and in fact there are a couple of entries on the issue tracker regarding this (the integrity part of your request, at least): npm/npm#3252, npm/npm#4016.

@ruimarinho

This comment has been minimized.

Show comment
Hide comment
@ruimarinho

ruimarinho Jan 18, 2015

Member

Agree with @KenanY. I think this should be handled at the package manager layer.

Member

ruimarinho commented Jan 18, 2015

Agree with @KenanY. I think this should be handled at the package manager layer.

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Jan 19, 2015

In fact, if you use npmE (and I'm sure there are other private registries and registry proxies that support this), there are already whitelist and blacklist features that give teams control over which packages can be used in their projects, set up in such a way that no packages that aren't on the approved list can even be installed within a protected intranet.

There have indeed been requests and proposals to come up with a general mechanism for package signing, and the main barrier there is the usual problem of how to authenticate signed packages without creating a usability and security nightmare. I think doing it at the package level is appropriate, though, because anything that's cryptographically secure is going to impose CPU overhead that's ridiculously high in the context of bringing up a web service in production.

othiym23 commented Jan 19, 2015

In fact, if you use npmE (and I'm sure there are other private registries and registry proxies that support this), there are already whitelist and blacklist features that give teams control over which packages can be used in their projects, set up in such a way that no packages that aren't on the approved list can even be installed within a protected intranet.

There have indeed been requests and proposals to come up with a general mechanism for package signing, and the main barrier there is the usual problem of how to authenticate signed packages without creating a usability and security nightmare. I think doing it at the package level is appropriate, though, because anything that's cryptographically secure is going to impose CPU overhead that's ridiculously high in the context of bringing up a web service in production.

@yurivict

This comment has been minimized.

Show comment
Hide comment
@yurivict

yurivict Mar 16, 2015

Why can't npm distribute the public key as part of its package, and sign all npm packages with the private key?

yurivict commented Mar 16, 2015

Why can't npm distribute the public key as part of its package, and sign all npm packages with the private key?

@indutny

This comment has been minimized.

Show comment
Hide comment
@indutny

indutny Mar 16, 2015

@yurivict In fact I thought about it too, but then I got distracted and never finished the: npm/npm-registry-client#43

indutny commented Mar 16, 2015

@yurivict In fact I thought about it too, but then I got distracted and never finished the: npm/npm-registry-client#43

@mbonaci

This comment has been minimized.

Show comment
Hide comment
@mbonaci

mbonaci Mar 16, 2015

Of course you can, but it's not in his employer's interest to do so, so he'll keep slipping you enterprise version and packaging the potential implementation in nightmares.
What about NPM foundation?

mbonaci commented Mar 16, 2015

Of course you can, but it's not in his employer's interest to do so, so he'll keep slipping you enterprise version and packaging the potential implementation in nightmares.
What about NPM foundation?

@yurivict

This comment has been minimized.

Show comment
Hide comment
@yurivict

yurivict Mar 16, 2015

I asked about package authentication here https://www.npmjs.com/support and here npm@npm.org, but can only hear this as an answer: https://www.youtube.com/watch?v=Re72di5phM0

yurivict commented Mar 16, 2015

I asked about package authentication here https://www.npmjs.com/support and here npm@npm.org, but can only hear this as an answer: https://www.youtube.com/watch?v=Re72di5phM0

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Mar 16, 2015

@isaacs, @seldo, and I have discussed adding package signing to npm, and have some tentative ideas about how it should be designed (probably more likely built on ssh keys than OpenPGP / GnuPG, for many, many reasons), but getting to that is a ways out, and is probably a ways behind getting more robust authentication to the registry.

othiym23 commented Mar 16, 2015

@isaacs, @seldo, and I have discussed adding package signing to npm, and have some tentative ideas about how it should be designed (probably more likely built on ssh keys than OpenPGP / GnuPG, for many, many reasons), but getting to that is a ways out, and is probably a ways behind getting more robust authentication to the registry.

@yurivict

This comment has been minimized.

Show comment
Hide comment
@yurivict

yurivict Mar 16, 2015

pgp is good for e-mail. But in respect of package signing, it depends what is meant here. If you are going to download pgp key from the public pgp server, this is also a security threat.

yurivict commented Mar 16, 2015

pgp is good for e-mail. But in respect of package signing, it depends what is meant here. If you are going to download pgp key from the public pgp server, this is also a security threat.

@timkuijsten

This comment has been minimized.

Show comment
Hide comment
@timkuijsten

timkuijsten Mar 17, 2015

@othiym23 maybe signify(1) is something usable. I know it's used with pkg_sign(1) and is licensed under ISC. See it's announcement from last year: http://www.undeadly.org/cgi?action=article&sid=20140121114349&mode=expanded&count=0

timkuijsten commented Mar 17, 2015

@othiym23 maybe signify(1) is something usable. I know it's used with pkg_sign(1) and is licensed under ISC. See it's announcement from last year: http://www.undeadly.org/cgi?action=article&sid=20140121114349&mode=expanded&count=0

@rvagg

This comment has been minimized.

Show comment
Hide comment
@rvagg

rvagg Mar 18, 2015

This is something we're tinkering with at NodeSource, so pulling @bmeck in here at least so he knows this discussion exists.

rvagg commented Mar 18, 2015

This is something we're tinkering with at NodeSource, so pulling @bmeck in here at least so he knows this discussion exists.

@bmeck

This comment has been minimized.

Show comment
Hide comment
@bmeck

bmeck Mar 18, 2015

Yarrrr! Brain dump as follows (too tired to go into depth):

  1. Developers hate signing things, we need to use keys that they already have. I have spoken to several people about this and I think _ssh keys_ are winning. Many different kinds of keys can even be used by reused as ssh keys which is a win.
  2. Having require perform code signing on a per require basis is a bad idea, you want to perform the analysis on the whole of what you require, thus we would want signatures to be associated with the whole module tree (difficult without moving to archive files ala noda-loader. I am currently speccing out getting the loader to do signing, but have not gotten around to making anything
  3. There are multiple points where signing is important as noted in this doc. We can associate module signing with some things, but signing an application in the same manner would most likely prove difficult since we generally want to associate with a certificate chain from the people I have talked to.
    1. There are also multiple definitions of deployable targets; we can ship a full archive of the compiled source, and there are several tactics to create single binary files. Our signing process should stay limited to the archives and leave binary signing to the tool of choice for the target os/arch.
    2. There is also a question of how to approve keys for usage, revocation lists might need to become prevalent and that is something that no one enjoys.
    3. There is a question about signing via groups, for example if a audit group signs modules as having been audited. To do this we would need to define our CA setup more in depth than just via keys.
  4. IO.js and Node.js dist use pgp keys.

Feel free to mention me about various points that are brought up.

PS: Just say no to PGP unless you have to.

bmeck commented Mar 18, 2015

Yarrrr! Brain dump as follows (too tired to go into depth):

  1. Developers hate signing things, we need to use keys that they already have. I have spoken to several people about this and I think _ssh keys_ are winning. Many different kinds of keys can even be used by reused as ssh keys which is a win.
  2. Having require perform code signing on a per require basis is a bad idea, you want to perform the analysis on the whole of what you require, thus we would want signatures to be associated with the whole module tree (difficult without moving to archive files ala noda-loader. I am currently speccing out getting the loader to do signing, but have not gotten around to making anything
  3. There are multiple points where signing is important as noted in this doc. We can associate module signing with some things, but signing an application in the same manner would most likely prove difficult since we generally want to associate with a certificate chain from the people I have talked to.
    1. There are also multiple definitions of deployable targets; we can ship a full archive of the compiled source, and there are several tactics to create single binary files. Our signing process should stay limited to the archives and leave binary signing to the tool of choice for the target os/arch.
    2. There is also a question of how to approve keys for usage, revocation lists might need to become prevalent and that is something that no one enjoys.
    3. There is a question about signing via groups, for example if a audit group signs modules as having been audited. To do this we would need to define our CA setup more in depth than just via keys.
  4. IO.js and Node.js dist use pgp keys.

Feel free to mention me about various points that are brought up.

PS: Just say no to PGP unless you have to.

@indolering

This comment has been minimized.

Show comment
Hide comment
@indolering

indolering Sep 7, 2015

While there are plenty of valid criticisms of PGP, it would fit in nicely with Git/Hub's support for PGP.

indolering commented Sep 7, 2015

While there are plenty of valid criticisms of PGP, it would fit in nicely with Git/Hub's support for PGP.

@krotscheck

This comment has been minimized.

Show comment
Hide comment
@krotscheck

krotscheck Sep 30, 2015

A couple of things re: PGP vs. SSH.

The difference between the two is that they use a different container format - inside of that container, the key is going to be something like RSA, ED25519, etc. Which container is used is mostly irrelevant at that point, though you'll get into a bit of a religious war if npm forces one over the other.

The much harder part is determining what the trust model is going to be. Signing and keys are easy, figuring out whether those signatures can be trusted is hard. There's an excellent blog post here ->
https://caremad.io/2013/07/packaging-signing-not-holy-grail/ that talks about different approaches. Also, there's an effort being spearheaded by Google called Binary Transparency that tries a somewhat different approach altogether - involvement in that starts here: https://groups.google.com/forum/#!forum/binary-transparency

(note, all of this commentary provided courtesy of @dstufft, who runs pypi. I am just a messenger who really cares about this topic).

krotscheck commented Sep 30, 2015

A couple of things re: PGP vs. SSH.

The difference between the two is that they use a different container format - inside of that container, the key is going to be something like RSA, ED25519, etc. Which container is used is mostly irrelevant at that point, though you'll get into a bit of a religious war if npm forces one over the other.

The much harder part is determining what the trust model is going to be. Signing and keys are easy, figuring out whether those signatures can be trusted is hard. There's an excellent blog post here ->
https://caremad.io/2013/07/packaging-signing-not-holy-grail/ that talks about different approaches. Also, there's an effort being spearheaded by Google called Binary Transparency that tries a somewhat different approach altogether - involvement in that starts here: https://groups.google.com/forum/#!forum/binary-transparency

(note, all of this commentary provided courtesy of @dstufft, who runs pypi. I am just a messenger who really cares about this topic).

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Sep 30, 2015

@indolering I am not swayed by Git's support for PGP. While git does an admirable job of making it Just Work, as long as you have your keys set up, etc., the dearth of signed commits and tags in GitHub (and signed packages in rubygems) should indicate that it's not easy or usable enough to be considered a valid approach.

@krotscheck Thanks for the links.

We don't need a holy grail, and I agree that we should not try to claim that package signing is one. We do need an approach that will be (a) usable enough to enable it by default without disrupting our users, and (b) given relatively high adoption rates, increase the security level enough to justify the cost of implementation. Ideal security is less valuable, in aggregate, than ideal usability, and less attainable.

There's a lot to admire about PGP:

  1. It doesn't pretend at perfect security. It's "pretty good protection", which can be then used to bootstrap other techniques to strengthen the trust in the key set. (Webs of trust, signing individual files, etc.)
  2. It's got a nice vocabulary of "stuff you can do with keys". Encryption for a specific recipient, signature by a known source, etc.
  3. Just removing a key from the set of trusted keys has the side-effect of invalidating everything signed with that key. So it's a nice kill-switch.

The downsides of PGP, which I think are critical:

  1. No one has PGP set up. Ok, a few people do, probably a lot of them on this discussion thread, but seriously, it's a rounding error away from no one. The difficulty of getting it set up and working is too high for the vast majority of publishers, and even higher for the vast majority of installers, so what's the point?
  2. PGP tools don't exist for Windows. About half of all Node users are doing their development on a Windows machine.

I was once hopeful Keybase.io might move the needle a bit on this. It hasn't. Unless it's trivial for 99% of publishers, and at least 80% of installers, there's no point. So PGP is out, at least for signing npm packages.

SSH keys are a bit less convenient to work with from the installer's pov, since there isn't as clear a vocabulary of actions and use cases. However:

  1. Everyone using GitHub already has SSH keys on their system.
  2. GitHub is already a public repository of public SSH keys. https://github.com/isaacs.keys So we can pretty trivially import them. (And if those keys get pwned, you're dead anyway, so who cares?)
  3. SSH keys work on Windows.
  4. We have in Node.js all the same crypto APIs to do what we need, and it's Just Typing to get to all the same benefits we would have with PGP.

I would really like to separate the problems of (a) "this artifact was definitely published by a user presenting this specific key" from the problems of (b) "which keys belong to this user" and (c) "which users should I trust". These problems are all orthogonal, and "package signing" can really only solve (a), and the challenge is to do so in such a way that doesn't make (b) and (c) impossible.

Before we (ie, npm, Inc.) invest resources in this, we need to finish shipping the products that will keep our lights on so that we continue to have a registry worth securing ;) I expect that this will be a revenue blocker soon enough, however, because it's a thing that big enterprise shops care a lot about, and then I can justify putting resources behind it. My best WAG is probably some time in 2016 it'll get started in earnest, and then a year or so after deployment it'll be in widespread usage enough to start getting obnoxious about packages that are unsigned.

Back to the point of the OP here: This will probably never be something worth doing at the require() level. We're all really talking about signing at npm publish time, and verifying at npm install time, and there are a lot of usability barriers that might crop up once we start actually doing it.

isaacs commented Sep 30, 2015

@indolering I am not swayed by Git's support for PGP. While git does an admirable job of making it Just Work, as long as you have your keys set up, etc., the dearth of signed commits and tags in GitHub (and signed packages in rubygems) should indicate that it's not easy or usable enough to be considered a valid approach.

@krotscheck Thanks for the links.

We don't need a holy grail, and I agree that we should not try to claim that package signing is one. We do need an approach that will be (a) usable enough to enable it by default without disrupting our users, and (b) given relatively high adoption rates, increase the security level enough to justify the cost of implementation. Ideal security is less valuable, in aggregate, than ideal usability, and less attainable.

There's a lot to admire about PGP:

  1. It doesn't pretend at perfect security. It's "pretty good protection", which can be then used to bootstrap other techniques to strengthen the trust in the key set. (Webs of trust, signing individual files, etc.)
  2. It's got a nice vocabulary of "stuff you can do with keys". Encryption for a specific recipient, signature by a known source, etc.
  3. Just removing a key from the set of trusted keys has the side-effect of invalidating everything signed with that key. So it's a nice kill-switch.

The downsides of PGP, which I think are critical:

  1. No one has PGP set up. Ok, a few people do, probably a lot of them on this discussion thread, but seriously, it's a rounding error away from no one. The difficulty of getting it set up and working is too high for the vast majority of publishers, and even higher for the vast majority of installers, so what's the point?
  2. PGP tools don't exist for Windows. About half of all Node users are doing their development on a Windows machine.

I was once hopeful Keybase.io might move the needle a bit on this. It hasn't. Unless it's trivial for 99% of publishers, and at least 80% of installers, there's no point. So PGP is out, at least for signing npm packages.

SSH keys are a bit less convenient to work with from the installer's pov, since there isn't as clear a vocabulary of actions and use cases. However:

  1. Everyone using GitHub already has SSH keys on their system.
  2. GitHub is already a public repository of public SSH keys. https://github.com/isaacs.keys So we can pretty trivially import them. (And if those keys get pwned, you're dead anyway, so who cares?)
  3. SSH keys work on Windows.
  4. We have in Node.js all the same crypto APIs to do what we need, and it's Just Typing to get to all the same benefits we would have with PGP.

I would really like to separate the problems of (a) "this artifact was definitely published by a user presenting this specific key" from the problems of (b) "which keys belong to this user" and (c) "which users should I trust". These problems are all orthogonal, and "package signing" can really only solve (a), and the challenge is to do so in such a way that doesn't make (b) and (c) impossible.

Before we (ie, npm, Inc.) invest resources in this, we need to finish shipping the products that will keep our lights on so that we continue to have a registry worth securing ;) I expect that this will be a revenue blocker soon enough, however, because it's a thing that big enterprise shops care a lot about, and then I can justify putting resources behind it. My best WAG is probably some time in 2016 it'll get started in earnest, and then a year or so after deployment it'll be in widespread usage enough to start getting obnoxious about packages that are unsigned.

Back to the point of the OP here: This will probably never be something worth doing at the require() level. We're all really talking about signing at npm publish time, and verifying at npm install time, and there are a lot of usability barriers that might crop up once we start actually doing it.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Sep 30, 2015

@isaacs I'm afraid you're a bit over-optimistic about SSH keys, at least for your points 1 and 2. GitHub encourages HTTPS use these days, and as such SSH is not part of the workflow for most new users. I did set up SSH one upon a time, and I guess there are some things at https://github.com/domenic.keys, but I don't have those corresponding private keys anymore, since I just use HTTPS.

domenic commented Sep 30, 2015

@isaacs I'm afraid you're a bit over-optimistic about SSH keys, at least for your points 1 and 2. GitHub encourages HTTPS use these days, and as such SSH is not part of the workflow for most new users. I did set up SSH one upon a time, and I guess there are some things at https://github.com/domenic.keys, but I don't have those corresponding private keys anymore, since I just use HTTPS.

@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Sep 30, 2015

Honestly, the mechanism by which a package is signed (GPG, PGP, NaCL, x509, whatever) is probably the least important part of secure signing toolchain. Lots of language repositories have implemented (a) and punted on (b) and (c) and essentially gained nothing. It's my belief that if npm does (a) without a solution for (b) and (c) they'll have gained nothing as well. I don't use Node or npm, so I don't have a horse in this race, I'm just one of @krotscheck's coworkers but I've spent a fair amount of time working on and thinking about this particular problem in general. I'm more than happy to offer advice if it's wanted, but If y'all want to do your thing without it that's fine too.

Like I said, I don't have a horse in this race, but am willing to help make a more secure internet.

dstufft commented Sep 30, 2015

Honestly, the mechanism by which a package is signed (GPG, PGP, NaCL, x509, whatever) is probably the least important part of secure signing toolchain. Lots of language repositories have implemented (a) and punted on (b) and (c) and essentially gained nothing. It's my belief that if npm does (a) without a solution for (b) and (c) they'll have gained nothing as well. I don't use Node or npm, so I don't have a horse in this race, I'm just one of @krotscheck's coworkers but I've spent a fair amount of time working on and thinking about this particular problem in general. I'm more than happy to offer advice if it's wanted, but If y'all want to do your thing without it that's fine too.

Like I said, I don't have a horse in this race, but am willing to help make a more secure internet.

@krotscheck

This comment has been minimized.

Show comment
Hide comment
@krotscheck

krotscheck Sep 30, 2015

The wonderful thing about open source is that you don't necessarily have to provide your own resources, @isaacs. There definitely seem to be enough individuals interested in this feature to solicit the community for help.

krotscheck commented Sep 30, 2015

The wonderful thing about open source is that you don't necessarily have to provide your own resources, @isaacs. There definitely seem to be enough individuals interested in this feature to solicit the community for help.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Sep 30, 2015

@dstufft Most implementations of (a) that I've seen have never risen to the level of usage required to be relevant. Without a vast majority of people using it, any implementation of (a) doesn't matter much.

(b) is solvable to within a pretty reasonable degree of safety relatively easily, though it is some feature work (email the user every time their keys change, require login to add/remove keys).

(c) is essentially an unsolved problem in the history of human existence in any context, but somehow we still manage to trust one another enough for society to function, so I'm less worried about it. If we get data about (a) and make (b) reasonable trustworthy, then folks can decide for themselves who they want to trust, and it's not too hard to have a system for defining some kind of policy (with the default being "trust everyone", since that's the only default that can make sense in general for newcomers; if we want to make everyone stop trusting someone, we'd just remove their keys or ban them or whatever). We could get into some kind of npm trust substack kind of thing where I sign the keys of another user, and other people can decide to trust anyone within isaacs + 2 degrees or something. But probably that's overthinking it, and like most WOT systems, the math is more fun than it is useful in practice. We're talking about a hundred thousand strangers, not a billion.

Thankfully, there is a for-profit entity that is both aware of these issues and stands to benefit financially from solving them adequately.

@domenic Do you have to type your password in to push code to your repos?

isaacs commented Sep 30, 2015

@dstufft Most implementations of (a) that I've seen have never risen to the level of usage required to be relevant. Without a vast majority of people using it, any implementation of (a) doesn't matter much.

(b) is solvable to within a pretty reasonable degree of safety relatively easily, though it is some feature work (email the user every time their keys change, require login to add/remove keys).

(c) is essentially an unsolved problem in the history of human existence in any context, but somehow we still manage to trust one another enough for society to function, so I'm less worried about it. If we get data about (a) and make (b) reasonable trustworthy, then folks can decide for themselves who they want to trust, and it's not too hard to have a system for defining some kind of policy (with the default being "trust everyone", since that's the only default that can make sense in general for newcomers; if we want to make everyone stop trusting someone, we'd just remove their keys or ban them or whatever). We could get into some kind of npm trust substack kind of thing where I sign the keys of another user, and other people can decide to trust anyone within isaacs + 2 degrees or something. But probably that's overthinking it, and like most WOT systems, the math is more fun than it is useful in practice. We're talking about a hundred thousand strangers, not a billion.

Thankfully, there is a for-profit entity that is both aware of these issues and stands to benefit financially from solving them adequately.

@domenic Do you have to type your password in to push code to your repos?

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Sep 30, 2015

@krotscheck In my experience, relying on that for stuff like this is less effective than you might imagine. Running a keyserver system and managing signatures is a non-trivial task that is fiddly enough to require full-time attention for a while. The bits that are strictly coding are trivial; what's needed is a service run reliably at scale by a widely trusted authority on npm packages, and integration with the npm accounts and website systems.

isaacs commented Sep 30, 2015

@krotscheck In my experience, relying on that for stuff like this is less effective than you might imagine. Running a keyserver system and managing signatures is a non-trivial task that is fiddly enough to require full-time attention for a while. The bits that are strictly coding are trivial; what's needed is a service run reliably at scale by a widely trusted authority on npm packages, and integration with the npm accounts and website systems.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Sep 30, 2015

Do you have to type your password in to push code to your repos?

No; one uses the git credential helper.

domenic commented Sep 30, 2015

Do you have to type your password in to push code to your repos?

No; one uses the git credential helper.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Sep 30, 2015

Fascinating!

Another thing worth mentioning here is that, if the user doesn't have an SSH key in the expected location (or available via SSH_AUTH_SOCK or whatever) then we can also create one for them on demand relatively easily and use that. More steps but as long as we're selling vaporware, may as well make it vaporware worth using!

isaacs commented Sep 30, 2015

Fascinating!

Another thing worth mentioning here is that, if the user doesn't have an SSH key in the expected location (or available via SSH_AUTH_SOCK or whatever) then we can also create one for them on demand relatively easily and use that. More steps but as long as we're selling vaporware, may as well make it vaporware worth using!

@krotscheck

This comment has been minimized.

Show comment
Hide comment
@krotscheck

krotscheck Oct 2, 2015

@isaacs I agree - that's why I find the binary transparency effort so interesting, because it comes with two major orgs that are already willing to build the distributed service at a scale that will meet both FreeBSD and Google's demands. The only reason HP hasn't thrown its weight behind it is that I've been busy trying to get our debian boxes to have a recent version of npm :-P

krotscheck commented Oct 2, 2015

@isaacs I agree - that's why I find the binary transparency effort so interesting, because it comes with two major orgs that are already willing to build the distributed service at a scale that will meet both FreeBSD and Google's demands. The only reason HP hasn't thrown its weight behind it is that I've been busy trying to get our debian boxes to have a recent version of npm :-P

@carlos8f

This comment has been minimized.

Show comment
Hide comment
@carlos8f

carlos8f May 24, 2016

As a package author, my ideal API would be:

command description auth level
npm signkey init initialize a new keypair, encrypt with passphrase new passphrase
npm signkey upload upload my public key to registry user
npm signkey ls list my uploaded keys and ids user
npm signkey revoke <id> revoke key by id user, perhaps passphrase
npm publish if a keypair is found, create a signature during the publish process user, passphrase
npm sign <package>@version sign a package version, adding your signature to a list user, passphrase
npm trust <username> or <@namespace> trust packages signed by username/group user
npm trust --make-public [--make-private] make your trusted users list public/private user
npm install --trusted [--only-trusted] warning/error if any packages aren't signed by trusted users user
npm ls --trusted display signers of installed packages and color-code trusted users user
npm signkey export <outpem> output encrypted PEM of keypair old passphrase, new passphrase
npm signkey import <inpem> import keypair from an encrypted PEM passphrase

I recommend separating existing npm auth from the new signing private key, so that if one is compromised, the other isn't. I also question using SSH keys for something they weren't intended for. Ideally the signing key is encrypted on disk using a passphrase, with something like AES-256.

carlos8f commented May 24, 2016

As a package author, my ideal API would be:

command description auth level
npm signkey init initialize a new keypair, encrypt with passphrase new passphrase
npm signkey upload upload my public key to registry user
npm signkey ls list my uploaded keys and ids user
npm signkey revoke <id> revoke key by id user, perhaps passphrase
npm publish if a keypair is found, create a signature during the publish process user, passphrase
npm sign <package>@version sign a package version, adding your signature to a list user, passphrase
npm trust <username> or <@namespace> trust packages signed by username/group user
npm trust --make-public [--make-private] make your trusted users list public/private user
npm install --trusted [--only-trusted] warning/error if any packages aren't signed by trusted users user
npm ls --trusted display signers of installed packages and color-code trusted users user
npm signkey export <outpem> output encrypted PEM of keypair old passphrase, new passphrase
npm signkey import <inpem> import keypair from an encrypted PEM passphrase

I recommend separating existing npm auth from the new signing private key, so that if one is compromised, the other isn't. I also question using SSH keys for something they weren't intended for. Ideally the signing key is encrypted on disk using a passphrase, with something like AES-256.

@taoeffect

This comment has been minimized.

Show comment
Hide comment
@taoeffect

taoeffect May 24, 2016

Maybe a simpler API? Complexity is the enemy of security. I'm also a package author, and I would prefer something like:

npm publish

  • If a private key exists, asks for its password and signs an update
  • Otherwise, creates a new key pair and asks for password to encrypt it with. Warns dev to back up the encrypted key.

npm install <package>

Does TOFU on the public key received from NPM.

npm install <package> --fingerprint <fingerprint>

Most secure method, as neither NPM nor anyone else can perform a MITM attack due OOB (out-of-band) knowledge of fingerprint.

Public key is always stored PEM encoded. Private key is always stored in some encrypted format. The two could even be stored in the same file, perhaps .npmkeys that NPM never publishes.

This way everyone will be migrated to doing E2E for their packages and no new APIs are needed. It will be non-optional.

taoeffect commented May 24, 2016

Maybe a simpler API? Complexity is the enemy of security. I'm also a package author, and I would prefer something like:

npm publish

  • If a private key exists, asks for its password and signs an update
  • Otherwise, creates a new key pair and asks for password to encrypt it with. Warns dev to back up the encrypted key.

npm install <package>

Does TOFU on the public key received from NPM.

npm install <package> --fingerprint <fingerprint>

Most secure method, as neither NPM nor anyone else can perform a MITM attack due OOB (out-of-band) knowledge of fingerprint.

Public key is always stored PEM encoded. Private key is always stored in some encrypted format. The two could even be stored in the same file, perhaps .npmkeys that NPM never publishes.

This way everyone will be migrated to doing E2E for their packages and no new APIs are needed. It will be non-optional.

@carlos8f

This comment has been minimized.

Show comment
Hide comment
@carlos8f

carlos8f May 24, 2016

Complexity is the enemy of security.

Oh how I wish this were true :]

npm install Does TOFU on the public key received from NPM.

Not sure if we're on the same page (what is TOFU lol). If the publisher's password (or session token) is validated by registry during npm publish, and the shasum is verified, what purpose would signatures serve? A previous attempt at merely validating signatures didn't fly. Why should anyone trust somebody's (dependency's) key? Without a trust model, it's only a more complicated version of what is already in place.

This way everyone will be migrated to doing E2E for their packages and no new APIs are needed. It will be non-optional.

Hmm, again we're on a different page. End-to-end is for encryption, we're only talking about signing. And all this is optional on top of the auth already in place. We shouldn't require people to create and shuffle around secret data keys in order to use npm for simple stuff.

carlos8f commented May 24, 2016

Complexity is the enemy of security.

Oh how I wish this were true :]

npm install Does TOFU on the public key received from NPM.

Not sure if we're on the same page (what is TOFU lol). If the publisher's password (or session token) is validated by registry during npm publish, and the shasum is verified, what purpose would signatures serve? A previous attempt at merely validating signatures didn't fly. Why should anyone trust somebody's (dependency's) key? Without a trust model, it's only a more complicated version of what is already in place.

This way everyone will be migrated to doing E2E for their packages and no new APIs are needed. It will be non-optional.

Hmm, again we're on a different page. End-to-end is for encryption, we're only talking about signing. And all this is optional on top of the auth already in place. We shouldn't require people to create and shuffle around secret data keys in order to use npm for simple stuff.

@taoeffect

This comment has been minimized.

Show comment
Hide comment
@taoeffect

taoeffect May 24, 2016

Oh how I wish this were true :]

Good thing it is! 😄

If the publisher's password (or session token) is validated by registry during npm publish, and the shasum is verified, what purpose would signatures serve?

Because that's not at all secure.

End-to-end is for encryption, we're only talking about signing.

Sounds like confusion. E2E just means ... exactly what it says: end-to-end. Signing can be done end-to-end, and signing is encryption. How do you think the signature is generated? It's done using the same asymmetric primitives that are used to encrypt messages.

EDIT: yes, signing and encrypting are typically considered different things, but underneath they are effectively the same math being used in different ways, and yes E2E signature validation is a thing.

taoeffect commented May 24, 2016

Oh how I wish this were true :]

Good thing it is! 😄

If the publisher's password (or session token) is validated by registry during npm publish, and the shasum is verified, what purpose would signatures serve?

Because that's not at all secure.

End-to-end is for encryption, we're only talking about signing.

Sounds like confusion. E2E just means ... exactly what it says: end-to-end. Signing can be done end-to-end, and signing is encryption. How do you think the signature is generated? It's done using the same asymmetric primitives that are used to encrypt messages.

EDIT: yes, signing and encrypting are typically considered different things, but underneath they are effectively the same math being used in different ways, and yes E2E signature validation is a thing.

@carlos8f

This comment has been minimized.

Show comment
Hide comment
@carlos8f

carlos8f May 24, 2016

sure, signing can be done on a shared secret. but then the trust is not transferable and the signature is only valid for 2 parties. We need signatures that are publicly verifiable or else there is no web of trust possible.

carlos8f commented May 24, 2016

sure, signing can be done on a shared secret. but then the trust is not transferable and the signature is only valid for 2 parties. We need signatures that are publicly verifiable or else there is no web of trust possible.

@taoeffect

This comment has been minimized.

Show comment
Hide comment
@taoeffect

taoeffect May 24, 2016

sure, signing can be done on a shared secret. but then the trust is not transferable and the signature is only valid for 2 parties. We need signatures that are publicly verifiable or else there is no web of trust possible.

Signing is not done using a shared secret. It is publicly verifiable.

taoeffect commented May 24, 2016

sure, signing can be done on a shared secret. but then the trust is not transferable and the signature is only valid for 2 parties. We need signatures that are publicly verifiable or else there is no web of trust possible.

Signing is not done using a shared secret. It is publicly verifiable.

@carlos8f

This comment has been minimized.

Show comment
Hide comment
@carlos8f

carlos8f May 24, 2016

it really depends on the protocol's goals but yeah. i'm done here.

carlos8f commented May 24, 2016

it really depends on the protocol's goals but yeah. i'm done here.

@tobytteh

This comment has been minimized.

Show comment
Hide comment
@tobytteh

tobytteh commented Aug 1, 2017

🤢

@davidk01

This comment has been minimized.

Show comment
Hide comment
@davidk01

davidk01 Aug 4, 2017

So it is a little disheartening that this has been open for 2 years now. Recent typo squatting of packages with malware that steal secret tokens from environment variables is one good example of why this problem should be solved. If npm supported package signing then it would have been much harder to deploy/install malware infected packages.

davidk01 commented Aug 4, 2017

So it is a little disheartening that this has been open for 2 years now. Recent typo squatting of packages with malware that steal secret tokens from environment variables is one good example of why this problem should be solved. If npm supported package signing then it would have been much harder to deploy/install malware infected packages.

@seldo

This comment has been minimized.

Show comment
Hide comment
@seldo

seldo Aug 4, 2017

How would package signing prevent people from requesting the wrong package? The malware author could also sign their package.

seldo commented Aug 4, 2017

How would package signing prevent people from requesting the wrong package? The malware author could also sign their package.

@bmeck

This comment has been minimized.

Show comment
Hide comment
@bmeck

bmeck Aug 4, 2017

@seldo depends on verification setup. If it is done at install time you can have a more aggressive check of expiration/revocation lists/OCSP. If it is done at startup time you might still want to do this.

As per prompts of accepting packages; just like SSH, if something is signed with an unknown key you can get a prompt. Most people unfortunately will just accept the key, even as a developer who might have some security knowledge. However, for production scenarios you could completely disable this to a list of known keys. This allows a whitelist on the client rather than just on the registry.

bmeck commented Aug 4, 2017

@seldo depends on verification setup. If it is done at install time you can have a more aggressive check of expiration/revocation lists/OCSP. If it is done at startup time you might still want to do this.

As per prompts of accepting packages; just like SSH, if something is signed with an unknown key you can get a prompt. Most people unfortunately will just accept the key, even as a developer who might have some security knowledge. However, for production scenarios you could completely disable this to a list of known keys. This allows a whitelist on the client rather than just on the registry.

@bmeck

This comment has been minimized.

Show comment
Hide comment
@bmeck

bmeck Aug 4, 2017

Plus side of known cert approval is that if a new transitive dependency appears from an unknown source it would be picked up very quickly.

bmeck commented Aug 4, 2017

Plus side of known cert approval is that if a new transitive dependency appears from an unknown source it would be picked up very quickly.

@davidk01

This comment has been minimized.

Show comment
Hide comment
@davidk01

davidk01 Aug 4, 2017

@bmeck covered a lot of it so much thanks but I'll also repeat and elaborate.

@seldo the scenario would be the user runs npm install xyz and xyz is signed lets say with my key at https://keybase.io/davidk01. During installation user is prompted about the set of keys and the corresponding packages and whether they want to trust those keys and install those packages or not trust those keys (for whatever reason). If they don't trust but still want to proceed then they can but the burden of proof is now on them to make sure they are not installing malware. If they do trust the keys then my public key is added to the local NPM trust chain and they have a few more assurances that whenever I ship a package it actually comes from me. In this case if I'm trusted then the only way they get malware from me is if I go rogue or I get hacked and my private keys are stolen.

It doesn't solve the problem of malware authors pushing signed packages but it does at least allow the end user to be more cognizant and aware where their packages come from and how trusted those users and packages are based on how the packages are signed and how trusted the signing key is in the user's local trust chain and on keybase. Right now it doesn't look like such provisions exist. If I want to verify the authenticity of a package I have to go out of my way by asking the package author to post signatures somewhere where I can verify and then I have to write wrapper scripts for npm to do all of it on my end.

davidk01 commented Aug 4, 2017

@bmeck covered a lot of it so much thanks but I'll also repeat and elaborate.

@seldo the scenario would be the user runs npm install xyz and xyz is signed lets say with my key at https://keybase.io/davidk01. During installation user is prompted about the set of keys and the corresponding packages and whether they want to trust those keys and install those packages or not trust those keys (for whatever reason). If they don't trust but still want to proceed then they can but the burden of proof is now on them to make sure they are not installing malware. If they do trust the keys then my public key is added to the local NPM trust chain and they have a few more assurances that whenever I ship a package it actually comes from me. In this case if I'm trusted then the only way they get malware from me is if I go rogue or I get hacked and my private keys are stolen.

It doesn't solve the problem of malware authors pushing signed packages but it does at least allow the end user to be more cognizant and aware where their packages come from and how trusted those users and packages are based on how the packages are signed and how trusted the signing key is in the user's local trust chain and on keybase. Right now it doesn't look like such provisions exist. If I want to verify the authenticity of a package I have to go out of my way by asking the package author to post signatures somewhere where I can verify and then I have to write wrapper scripts for npm to do all of it on my end.

@seldo

This comment has been minimized.

Show comment
Hide comment
@seldo

seldo Aug 4, 2017

Thanks both for elaborating.

npm is doing design work around signed packages, but our near term security work is around 2FA to prevent unauthorized publishes with stolen credentials, as well as a lot of different new strategies for detecting spam and malware and taking them down faster.

seldo commented Aug 4, 2017

Thanks both for elaborating.

npm is doing design work around signed packages, but our near term security work is around 2FA to prevent unauthorized publishes with stolen credentials, as well as a lot of different new strategies for detecting spam and malware and taking them down faster.

@bmeck

This comment has been minimized.

Show comment
Hide comment
@bmeck

bmeck Aug 4, 2017

@seldo I'd be happy to discuss if https://github.com/WICG/webpackage might be suitable in their repository issues.

bmeck commented Aug 4, 2017

@seldo I'd be happy to discuss if https://github.com/WICG/webpackage might be suitable in their repository issues.

@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Aug 4, 2017

Please note, I am not a Node.js user, but I do work on similiar packaging tooling so I've thought about lot of these issues!

TOFU like with SSH doesn't work great for package managers, because with SSH you generally connect to a small set of stable servers that you know ahead of time. So your list of ~/.know_hosts rarely changes and if you add new administrators the host key for the server stays the same.

This is unlike packages, where you're often times installing packages you've never installed before, and your packages are likely going to depend on things that you don't directly know ahead of time that it's going to pull in. Further more this list is constantly changing which makes it even harder to manage a TOFU trustdb. Finally since you're attempting to validate authors not a per-package key, if you have two authors both able to release software, suddenly the person through a normal course of installing software has to deal with invalid signatures. A TOFU database tends to normalize end users needing to just go ahead and add whatever key they need to to the trust store, without really digging into or thinking about what they're doing (because it is a normal day to day operation for them at that point).

dstufft commented Aug 4, 2017

Please note, I am not a Node.js user, but I do work on similiar packaging tooling so I've thought about lot of these issues!

TOFU like with SSH doesn't work great for package managers, because with SSH you generally connect to a small set of stable servers that you know ahead of time. So your list of ~/.know_hosts rarely changes and if you add new administrators the host key for the server stays the same.

This is unlike packages, where you're often times installing packages you've never installed before, and your packages are likely going to depend on things that you don't directly know ahead of time that it's going to pull in. Further more this list is constantly changing which makes it even harder to manage a TOFU trustdb. Finally since you're attempting to validate authors not a per-package key, if you have two authors both able to release software, suddenly the person through a normal course of installing software has to deal with invalid signatures. A TOFU database tends to normalize end users needing to just go ahead and add whatever key they need to to the trust store, without really digging into or thinking about what they're doing (because it is a normal day to day operation for them at that point).

@bmeck

This comment has been minimized.

Show comment
Hide comment
@bmeck

bmeck Aug 4, 2017

@dstufft I agree on developer machines to a limited extent if the TOFU db never gets checked against mechanisms for revocation. Installing on a build server can be much more limited and have safety on that side. We can discuss trust models if desired but I am personally for X.509 style trees rather than web of trust when discussing revocation.

bmeck commented Aug 4, 2017

@dstufft I agree on developer machines to a limited extent if the TOFU db never gets checked against mechanisms for revocation. Installing on a build server can be much more limited and have safety on that side. We can discuss trust models if desired but I am personally for X.509 style trees rather than web of trust when discussing revocation.

@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Aug 4, 2017

Revocation doesn't really work very well TBH, most browsers don't even bother to support certificate revocation anymore (well they've started shipping revocations built into the browser itself, but only for important certificates). The issue comes down to the fact an attacker can just block the revocation check and then you have to decide to hard fail or soft fail. If you hard fail then your revocation becomes a SPOF and if you soft fail then revocation is trivially defeatable. You can see more information about that here: https://www.imperialviolet.org/2012/02/05/crlsets.html

I do agree that x509 trees are better than web of trust. In Python we're planning on implementing TUF and I recommend using that. You still need to decide how specifically to manage trust, but it's pretty solid tech.

dstufft commented Aug 4, 2017

Revocation doesn't really work very well TBH, most browsers don't even bother to support certificate revocation anymore (well they've started shipping revocations built into the browser itself, but only for important certificates). The issue comes down to the fact an attacker can just block the revocation check and then you have to decide to hard fail or soft fail. If you hard fail then your revocation becomes a SPOF and if you soft fail then revocation is trivially defeatable. You can see more information about that here: https://www.imperialviolet.org/2012/02/05/crlsets.html

I do agree that x509 trees are better than web of trust. In Python we're planning on implementing TUF and I recommend using that. You still need to decide how specifically to manage trust, but it's pretty solid tech.

@frio

This comment has been minimized.

Show comment
Hide comment
@frio

frio Feb 13, 2018

conventional-changelog/conventional-changelog#282 has had an impact today. In this instance, the damage was limited to an easily expunged crypto-miner; across an organisation (especially one with an NPM cache), this is difficult to track down but not devastating. If this had been a cryptolocker alternative, this could have been devastating for a lot of people. Where is the work to implement 2fa or signing tracked :)?

frio commented Feb 13, 2018

conventional-changelog/conventional-changelog#282 has had an impact today. In this instance, the damage was limited to an easily expunged crypto-miner; across an organisation (especially one with an NPM cache), this is difficult to track down but not devastating. If this had been a cryptolocker alternative, this could have been devastating for a lot of people. Where is the work to implement 2fa or signing tracked :)?

@bmeck

This comment has been minimized.

Show comment
Hide comment
@bmeck

bmeck Feb 13, 2018

@frio I'd watch https://github.com/WICG/webpackage (it has a long outstanding name change request) which should be applicable across Web, Electron, and Node. There exists a separate efforts at a purely package manager level and not for loader checks in package-community/discussions#5

bmeck commented Feb 13, 2018

@frio I'd watch https://github.com/WICG/webpackage (it has a long outstanding name change request) which should be applicable across Web, Electron, and Node. There exists a separate efforts at a purely package manager level and not for loader checks in package-community/discussions#5

@sashahilton00

This comment has been minimized.

Show comment
Hide comment
@sashahilton00

sashahilton00 Jun 26, 2018

@isaacs is there any update on this? I note that you mentioned in September 2015 that you guessed we might see it rolled out in 1-2 years, and whilst it was only a guess, we're nearing the 3 year mark for this issue with no apparent end in sight. Is it even on the roadmap anymore?

sashahilton00 commented Jun 26, 2018

@isaacs is there any update on this? I note that you mentioned in September 2015 that you guessed we might see it rolled out in 1-2 years, and whilst it was only a guess, we're nearing the 3 year mark for this issue with no apparent end in sight. Is it even on the roadmap anymore?

@kmoe

This comment has been minimized.

Show comment
Hide comment

kmoe commented Jul 12, 2018

nevar 4get eslint/eslint-scope#39

@carlos8f

This comment has been minimized.

Show comment
Hide comment
@carlos8f

carlos8f Jul 12, 2018

@sashahilton00 @kmoe @isaacs @bmeck i can do a PR that incorporates Salty(https://github.com/carlos8f/salty) using Daniel J Bernstein's libsodium. it's the most bulletproof future-proof crypto that is public domain in existence, and its a shame that no one is using it. it's my life mission to spread Daniel's work around and make encrypted, signed, and verified data the standard, instead of the exception. Daniel understands that most crypto is backdoored, including all NIST-published curves, and this has been proven in writing and with equations by Schneider et al. The only usable curves are those that Satoshi chose and that Daniel chose, the others are phonies and are being spied on and NSA'd.

Let's incorporate Salty as the way to easily sign an npm package. i can do the PR in a day. all i need is ppl to agree with me and start using Salty. I am working with Satoshi right now on Bitcoin next gen, which will have full encryption and privacy via Salty. So take it or leave it :) it was over 2 years ago that i wrote Salty and so far no one is hip enough to apply it. Im here to change that.

https://s8f.org/1463990203
https://s8f.org/1465262642
https://s8f.org/1465282150
https://s8f.org/1466743328

carlos8f commented Jul 12, 2018

@sashahilton00 @kmoe @isaacs @bmeck i can do a PR that incorporates Salty(https://github.com/carlos8f/salty) using Daniel J Bernstein's libsodium. it's the most bulletproof future-proof crypto that is public domain in existence, and its a shame that no one is using it. it's my life mission to spread Daniel's work around and make encrypted, signed, and verified data the standard, instead of the exception. Daniel understands that most crypto is backdoored, including all NIST-published curves, and this has been proven in writing and with equations by Schneider et al. The only usable curves are those that Satoshi chose and that Daniel chose, the others are phonies and are being spied on and NSA'd.

Let's incorporate Salty as the way to easily sign an npm package. i can do the PR in a day. all i need is ppl to agree with me and start using Salty. I am working with Satoshi right now on Bitcoin next gen, which will have full encryption and privacy via Salty. So take it or leave it :) it was over 2 years ago that i wrote Salty and so far no one is hip enough to apply it. Im here to change that.

https://s8f.org/1463990203
https://s8f.org/1465262642
https://s8f.org/1465282150
https://s8f.org/1466743328

@strugee

This comment has been minimized.

Show comment
Hide comment
@strugee

strugee Jul 12, 2018

@carlos8f the crypto parts of this are the least hard to do. It's everything else that's the problem. You should read this issue all the way through.

strugee commented Jul 12, 2018

@carlos8f the crypto parts of this are the least hard to do. It's everything else that's the problem. You should read this issue all the way through.

@claui claui referenced this issue Jul 22, 2018

Open

Enable GPG verification for Casks. #4120

4 of 6 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment