[meta issue] unpublish policy #44

Closed
ashleygwilliams opened this Issue Mar 29, 2016 · 115 comments

Comments

Projects
None yet
@ashleygwilliams
Contributor

ashleygwilliams commented Mar 29, 2016

today we published a blog post on our first steps towards a new unpublish policy.
this issue is a place for readers to comment an ask questions about the unpublish policy.
comments and questions that are not specific to the unpublish policy will be moderated.

link to the blog post: http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy

remember: npm has a Code of Conduct

@groundwater

This comment has been minimized.

Show comment
Hide comment
@groundwater

groundwater Mar 29, 2016

In the event a package has an open-source license like MIT that allows redistribution, I think maintaining a package in the registry makes sense.

There are however packages listed with non-open licenses, and likely in some cases licenses that can be revoked. In that case you effectively have unusable software published to the registry. I recognize one solution might be "only open licenses allowed" but that's probably not reality.

I suppose one could deprecate the module, but I think most people ignore warnings during install.

In the event a package has an open-source license like MIT that allows redistribution, I think maintaining a package in the registry makes sense.

There are however packages listed with non-open licenses, and likely in some cases licenses that can be revoked. In that case you effectively have unusable software published to the registry. I recognize one solution might be "only open licenses allowed" but that's probably not reality.

I suppose one could deprecate the module, but I think most people ignore warnings during install.

@searls

This comment has been minimized.

Show comment
Hide comment
@searls

searls Mar 29, 2016

First, I think this is a terrific policy change in favor of dependency stability, so kudos! I'm all for dependency registries treating their artifacts as permanently etched in stone for the sake of keeping end-users' software working. It's in that spirit that I'm asking this question:

The policy change affects how npm users who own depended-on packages can un-publish their own content from the registry, but I'm curious if it affects how npm will respond to DMCA takedown notices of depended-on modules. If npm voluntarily acquiesces to a takedown complaint (to retain its DMCA safe harbor protection), it might find itself putting npm install users in a very similar situation. Obviously, there's only so much that could be done if npm were ultimately compelled to take down a module by court order, but other types of content hosting services have shown there is a pretty broad range of potential responses to DMCA takedown requests.

Ideally, I'd like to see npm take a stance that defaulted to defending end-users of packages, potentially providing tools for addressing infringement with a less-blunt instrument than removing a module (or a version of a module) altogether.

I realize this is a hypothetical, but the situation that brought this policy change about is only one logical leap away; if npm had been dealt with a DMCA takedown or a cease & desist for the leftpad module as opposed to kik, it would have been put in a similar situation, even without the rogue package maintainer that this policy seeks to mitigate the impact of.

searls commented Mar 29, 2016

First, I think this is a terrific policy change in favor of dependency stability, so kudos! I'm all for dependency registries treating their artifacts as permanently etched in stone for the sake of keeping end-users' software working. It's in that spirit that I'm asking this question:

The policy change affects how npm users who own depended-on packages can un-publish their own content from the registry, but I'm curious if it affects how npm will respond to DMCA takedown notices of depended-on modules. If npm voluntarily acquiesces to a takedown complaint (to retain its DMCA safe harbor protection), it might find itself putting npm install users in a very similar situation. Obviously, there's only so much that could be done if npm were ultimately compelled to take down a module by court order, but other types of content hosting services have shown there is a pretty broad range of potential responses to DMCA takedown requests.

Ideally, I'd like to see npm take a stance that defaulted to defending end-users of packages, potentially providing tools for addressing infringement with a less-blunt instrument than removing a module (or a version of a module) altogether.

I realize this is a hypothetical, but the situation that brought this policy change about is only one logical leap away; if npm had been dealt with a DMCA takedown or a cease & desist for the leftpad module as opposed to kik, it would have been put in a similar situation, even without the rogue package maintainer that this policy seeks to mitigate the impact of.

@mattdesl

This comment has been minimized.

Show comment
Hide comment
@mattdesl

mattdesl Mar 29, 2016

Nice post!

Question about this:

if removing that version of your package would break any other installs

  1. How can you detect this reliably? What about private modules, unpublished GitHub repos, devDependents, etc?
  2. Does this extend to all versions of a module? foo@1.2.2 might not have any dependents, but foo@1.2.1 may have a single dependent. Because of semver, pinning, magic zero and other complexities, foo@1.2.1 can still cause widespread breakages across the ecosystem.

The second point already happened when esprima-six was unpublished – it's latest version had zero direct dependents but its removal still caused widespread issues for many npm users.

Nice post!

Question about this:

if removing that version of your package would break any other installs

  1. How can you detect this reliably? What about private modules, unpublished GitHub repos, devDependents, etc?
  2. Does this extend to all versions of a module? foo@1.2.2 might not have any dependents, but foo@1.2.1 may have a single dependent. Because of semver, pinning, magic zero and other complexities, foo@1.2.1 can still cause widespread breakages across the ecosystem.

The second point already happened when esprima-six was unpublished – it's latest version had zero direct dependents but its removal still caused widespread issues for many npm users.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Mar 29, 2016

Member

@mattdesl For dependencies, we can query for all the dependents of a package, and then check what their dependency is. If it's pinned to a specific version, then we'll see that. Removing version 1.2.1 will still fail if 1.2.1 is more than 24 hours old, and then support will be able to deny the request if we see that case.

For the very rare cases where a package has zero dependents, on any version, we can still look at downloads. It's a hard thing to judge programmatically, because of the statistical noise introduced by bots and downstream replicas that download every package. But it's not completely impossible to determine if something is hugely popular (hundreds or thousands of downloads) vs more or less unknown (7 downloads in the last month).

In any event, we'll be erring hard on the side of "No, do not unpublish, use deprecation or transfer ownership" in any case that is even slightly questionable.

Member

isaacs commented Mar 29, 2016

@mattdesl For dependencies, we can query for all the dependents of a package, and then check what their dependency is. If it's pinned to a specific version, then we'll see that. Removing version 1.2.1 will still fail if 1.2.1 is more than 24 hours old, and then support will be able to deny the request if we see that case.

For the very rare cases where a package has zero dependents, on any version, we can still look at downloads. It's a hard thing to judge programmatically, because of the statistical noise introduced by bots and downstream replicas that download every package. But it's not completely impossible to determine if something is hugely popular (hundreds or thousands of downloads) vs more or less unknown (7 downloads in the last month).

In any event, we'll be erring hard on the side of "No, do not unpublish, use deprecation or transfer ownership" in any case that is even slightly questionable.

@tlrobinson

This comment has been minimized.

Show comment
Hide comment
@tlrobinson

tlrobinson Mar 29, 2016

(cross-posted at https://news.ycombinator.com/item?id=11383113)

Because all dependents are satisfied by 1.0.1, support agrees to grant Supreet’s request to delete 1.0.0.

There are some good changes here, unfortunately this policy will still break projects that use npm shrinkwrap to lock down specific versions, which IMHO, anyone who wishes to retain their sanity will do.

When this happens AFAIK there's no foolproof way to "patch" your npm-shrinkwrap.json with the new version without deleting it and re-running shrinkwrap, and thus possibly bumping versions of every other package as well. The usual recommendation seems to be "find and replace" the version in npm-shrinkwrap.json with a regex, which is fine if the package's dependencies didn't change but will break if a dependency is added.

I'd love to know if there's a better way.

FWIW, if you think this is a very rare occurrence, it's now happened to me 3 times in last few months, the latest being less than a week after the left-pad incident.

(cross-posted at https://news.ycombinator.com/item?id=11383113)

Because all dependents are satisfied by 1.0.1, support agrees to grant Supreet’s request to delete 1.0.0.

There are some good changes here, unfortunately this policy will still break projects that use npm shrinkwrap to lock down specific versions, which IMHO, anyone who wishes to retain their sanity will do.

When this happens AFAIK there's no foolproof way to "patch" your npm-shrinkwrap.json with the new version without deleting it and re-running shrinkwrap, and thus possibly bumping versions of every other package as well. The usual recommendation seems to be "find and replace" the version in npm-shrinkwrap.json with a regex, which is fine if the package's dependencies didn't change but will break if a dependency is added.

I'd love to know if there's a better way.

FWIW, if you think this is a very rare occurrence, it's now happened to me 3 times in last few months, the latest being less than a week after the left-pad incident.

@thekemkid

This comment has been minimized.

Show comment
Hide comment
@thekemkid

thekemkid Mar 29, 2016

I'm wondering about private modules, can you straight up unpublish them or must you go through support for it, as with public modules? You would more than likely own or collaborate on the private modules that depend upon it, so you'll only break your own things, which at the end of the day, is your problem, not npm's.

I'm wondering about private modules, can you straight up unpublish them or must you go through support for it, as with public modules? You would more than likely own or collaborate on the private modules that depend upon it, so you'll only break your own things, which at the end of the day, is your problem, not npm's.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Mar 29, 2016

Member

@groundwater

There are however packages listed with non-open licenses, and likely in some cases licenses that can be revoked. In that case you effectively have unusable software published to the registry. I recognize one solution might be "only open licenses allowed" but that's probably not reality.

By publishing it on the npm registry, you agree to let us distribute it. Of course, users might not have any rights to use the package, but it's out there, and up to real lawyers to fight that one out. Ultimately, it's not our responsibility to enforce your license. You publish it, it's published.

@searls One rare scenario where we are legally required to take down a module is DMCA. If the proper forms are followed, we have to follow procedure. Our policy is here: http://npm.im/policies/dmca

Member

isaacs commented Mar 29, 2016

@groundwater

There are however packages listed with non-open licenses, and likely in some cases licenses that can be revoked. In that case you effectively have unusable software published to the registry. I recognize one solution might be "only open licenses allowed" but that's probably not reality.

By publishing it on the npm registry, you agree to let us distribute it. Of course, users might not have any rights to use the package, but it's out there, and up to real lawyers to fight that one out. Ultimately, it's not our responsibility to enforce your license. You publish it, it's published.

@searls One rare scenario where we are legally required to take down a module is DMCA. If the proper forms are followed, we have to follow procedure. Our policy is here: http://npm.im/policies/dmca

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 29, 2016

Contributor

to reiterate what @isaacs, @groundwater by publishing your package to the npm registry you agree to npm's terms which involve a special license that allows us to distribute it, regardless of whatever license you have (or have not) assigned it. you can read more about that here: https://github.com/npm/policies/blob/master/open-source-terms.md

the latest PR to that policy is from our lawyer specifically trying to make this as clear as possibe: 140ed66

Contributor

ashleygwilliams commented Mar 29, 2016

to reiterate what @isaacs, @groundwater by publishing your package to the npm registry you agree to npm's terms which involve a special license that allows us to distribute it, regardless of whatever license you have (or have not) assigned it. you can read more about that here: https://github.com/npm/policies/blob/master/open-source-terms.md

the latest PR to that policy is from our lawyer specifically trying to make this as clear as possibe: 140ed66

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Mar 29, 2016

Member

@tlrobinson There are two things here: the first is "how to more easily update shrinkwrap", and the second is "how to prevent pinned versions from breaking users".

For the first, the latest and greatest npm@3 supports updating your shrinkwrap when you npm install foo@latest --save. That's a smaller problem. (Or a regex replace, or text editor, or delete and re-run shrinkwrap, etc.)

For the second, I think the same logic in #44 (comment) would apply. We'll look at that case, and probably be able to get it right most of the time. When in doubt, we're going to err on the side of not allowing deletions.

I don't think this (or any!) policy is going to be 100% airtight in every conceivable scenario. But the perfect is the enemy of the good in situations like these. We're closing a lot of problematic issues with this change, but we aren't under the illusion that this sort of work is ever completely "done".

Member

isaacs commented Mar 29, 2016

@tlrobinson There are two things here: the first is "how to more easily update shrinkwrap", and the second is "how to prevent pinned versions from breaking users".

For the first, the latest and greatest npm@3 supports updating your shrinkwrap when you npm install foo@latest --save. That's a smaller problem. (Or a regex replace, or text editor, or delete and re-run shrinkwrap, etc.)

For the second, I think the same logic in #44 (comment) would apply. We'll look at that case, and probably be able to get it right most of the time. When in doubt, we're going to err on the side of not allowing deletions.

I don't think this (or any!) policy is going to be 100% airtight in every conceivable scenario. But the perfect is the enemy of the good in situations like these. We're closing a lot of problematic issues with this change, but we aren't under the illusion that this sort of work is ever completely "done".

@mattdesl

This comment has been minimized.

Show comment
Hide comment
@mattdesl

mattdesl Mar 29, 2016

Another thing that might be good to clarify is how the unpublish command or policy is handled by contributors to a module, rather than the original authors. Azer was able to unpublish some modules that he was just a contributor to, rather than the original creator.

@isaacs Thanks. I'm not sure download counts will tell you much. Some of our private (client) work may sit idle for months before we need to "revive" the project by cloning and re-installing dependencies.

Another thing that might be good to clarify is how the unpublish command or policy is handled by contributors to a module, rather than the original authors. Azer was able to unpublish some modules that he was just a contributor to, rather than the original creator.

@isaacs Thanks. I'm not sure download counts will tell you much. Some of our private (client) work may sit idle for months before we need to "revive" the project by cloning and re-installing dependencies.

@shazron

This comment has been minimized.

Show comment
Hide comment
@shazron

shazron Mar 29, 2016

Because all dependents are satisfied by 1.0.1, support agrees to grant Supreet’s request to delete 1.0.0.

Getting "all" dependents to agree would be problematic, this is the Internet (plus lag in responses). Perhaps more of a consensus or majority instead (especially if you need to remove a vulnerability). I assume that will be at the discretion of support anyway and not a hard and fast rule.

shazron commented Mar 29, 2016

Because all dependents are satisfied by 1.0.1, support agrees to grant Supreet’s request to delete 1.0.0.

Getting "all" dependents to agree would be problematic, this is the Internet (plus lag in responses). Perhaps more of a consensus or majority instead (especially if you need to remove a vulnerability). I assume that will be at the discretion of support anyway and not a hard and fast rule.

@tlrobinson

This comment has been minimized.

Show comment
Hide comment
@tlrobinson

tlrobinson Mar 29, 2016

@isaacs I don't think npm@3's install --save support for updating npm-shrinkwrap.json will help with updating sub-dependencies that I don't depend on directly. I suppose I could do npm install foo@latest --save then delete "foo" from my package.json. As long as the version I wanted to update was in the top level node_modules that would work, but if there were more than one version and the broken one was in a nested node_modules directory this wouldn't help, would it?

Separately, this is probably a lot more work for you guys, but how about just putting unpublished packages in a pseudo-unpublished state where they don't show up in npm show etc and warn users installing them, either permanently or (more likely) for a grace period. Something like left-pad@0.0.2 has been deprecated and will be removed on 2016-04-15. It's possible this is due to a security vulnerability, please investigate and remove or upgrade this dependency before that date. I'm sure this copy could be worded more elegantly, but it has the benefit of not breaking people's builds and warning them of a possible security issue.

In the case of chalk/ansi-styles#15 version 2.2.0 was unpublished because it was "broken" from the developers' perspective, but from my perspective it was working just fine (they admit chalk's use of ansi-styles was not broken), and the act of unpublishing actually broke things for me. According to your new policy as long as they published 2.2.1 before asking to unpublish 2.2.0 the request would still have been granted?

@isaacs I don't think npm@3's install --save support for updating npm-shrinkwrap.json will help with updating sub-dependencies that I don't depend on directly. I suppose I could do npm install foo@latest --save then delete "foo" from my package.json. As long as the version I wanted to update was in the top level node_modules that would work, but if there were more than one version and the broken one was in a nested node_modules directory this wouldn't help, would it?

Separately, this is probably a lot more work for you guys, but how about just putting unpublished packages in a pseudo-unpublished state where they don't show up in npm show etc and warn users installing them, either permanently or (more likely) for a grace period. Something like left-pad@0.0.2 has been deprecated and will be removed on 2016-04-15. It's possible this is due to a security vulnerability, please investigate and remove or upgrade this dependency before that date. I'm sure this copy could be worded more elegantly, but it has the benefit of not breaking people's builds and warning them of a possible security issue.

In the case of chalk/ansi-styles#15 version 2.2.0 was unpublished because it was "broken" from the developers' perspective, but from my perspective it was working just fine (they admit chalk's use of ansi-styles was not broken), and the act of unpublishing actually broke things for me. According to your new policy as long as they published 2.2.1 before asking to unpublish 2.2.0 the request would still have been granted?

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 29, 2016

Contributor

@shazron

I assume that will be at the discretion of support anyway and not a hard and fast rule.

exactly that. hard and fast rules are also often easy to exploit, for now we are relying on the discretion of humans as we see how this current policy plays out. i imagine some patterns will emerge that will help us further this policy.

Contributor

ashleygwilliams commented Mar 29, 2016

@shazron

I assume that will be at the discretion of support anyway and not a hard and fast rule.

exactly that. hard and fast rules are also often easy to exploit, for now we are relying on the discretion of humans as we see how this current policy plays out. i imagine some patterns will emerge that will help us further this policy.

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 29, 2016

Contributor

@mattdesl it turns out that "ownership" is a significantly complicated idea in the npm public registry, though it may seem simple at face-value. we know that we need to make "ownership" clearer, and to do so we'll need to make some technical decisions and changes. top of mind, for sure.

Contributor

ashleygwilliams commented Mar 29, 2016

@mattdesl it turns out that "ownership" is a significantly complicated idea in the npm public registry, though it may seem simple at face-value. we know that we need to make "ownership" clearer, and to do so we'll need to make some technical decisions and changes. top of mind, for sure.

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Mar 29, 2016

One thing that is not addressed by this policy change is the potential for a legitimate npm package owner's credentials to be compromised, allowing an attacker to publish a new version of a trusted package containing malicious code.

Because npm automatically installs the most recent version of a package, for packages with many consumers, this would result in a very rapid spread of a virus or trojan horse in a short period of time. That is, if I get access to publishing credentials for foo, and publish foo@1.4.5, any consumers depending on semver ^1.4.4 (or indeed any prior version in the 1.x range) will automatically consume it with npm install.. Since this is a very common pattern - npm install foo --save will automatically create a dependency using this type of semver - the vast majority of consumers simply doing a clean install of something would probably consume 1.4.5 automatically.

This risk could be greatly mitigated by changing the semver matching policy to consume the oldest matching version instead of the newest matching version. This would mean new versions of packages are never consumed automatically, except as explicitly required by a consumer. This would greatly slow the speed with which a new version of a package would be consumed.

This is a saner policy for software developers for other reasons, anyway. We make a habit of using shrinkwrap all the time for production code, since even with semver, people make mistakes, and a new version could introduce a problem you didn't have with the current version you were using. We don't want to consume new versions of anything automatically, unless we have to or need to for some known reason. Generally, I would think most people would only want to consume a new version of a package to solve a problem, or make use of a new feature. Either of these reasons would result in a deliberate decison to upgrade, rather than essentially having new versions of code continually pushed.

Have you given consideration to changing this behavior for npm install?

One thing that is not addressed by this policy change is the potential for a legitimate npm package owner's credentials to be compromised, allowing an attacker to publish a new version of a trusted package containing malicious code.

Because npm automatically installs the most recent version of a package, for packages with many consumers, this would result in a very rapid spread of a virus or trojan horse in a short period of time. That is, if I get access to publishing credentials for foo, and publish foo@1.4.5, any consumers depending on semver ^1.4.4 (or indeed any prior version in the 1.x range) will automatically consume it with npm install.. Since this is a very common pattern - npm install foo --save will automatically create a dependency using this type of semver - the vast majority of consumers simply doing a clean install of something would probably consume 1.4.5 automatically.

This risk could be greatly mitigated by changing the semver matching policy to consume the oldest matching version instead of the newest matching version. This would mean new versions of packages are never consumed automatically, except as explicitly required by a consumer. This would greatly slow the speed with which a new version of a package would be consumed.

This is a saner policy for software developers for other reasons, anyway. We make a habit of using shrinkwrap all the time for production code, since even with semver, people make mistakes, and a new version could introduce a problem you didn't have with the current version you were using. We don't want to consume new versions of anything automatically, unless we have to or need to for some known reason. Generally, I would think most people would only want to consume a new version of a package to solve a problem, or make use of a new feature. Either of these reasons would result in a deliberate decison to upgrade, rather than essentially having new versions of code continually pushed.

Have you given consideration to changing this behavior for npm install?

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 29, 2016

Contributor

hi @jamietre - a couple of things:

One thing that is not addressed by this policy change is the potential for a legitimate npm package owner's credentials to be compromised, allowing an attacker to publish a new version of a trusted package containing malicious code.

this policy was never meant to address that problem.

This risk could be greatly mitigated by changing the semver matching policy to consume the oldest matching version instead of the newest matching version.

this is the opposite of npm's stance on semver. automatically including patch versions with bugfixes is exactly what we want to happen. if you, personally, don't want that, you are free to use other semver expressions or specify a single specific version.

also: semver is not something invented at npm and we do not plan on changing it. for more info, see http://semver.org/

Contributor

ashleygwilliams commented Mar 29, 2016

hi @jamietre - a couple of things:

One thing that is not addressed by this policy change is the potential for a legitimate npm package owner's credentials to be compromised, allowing an attacker to publish a new version of a trusted package containing malicious code.

this policy was never meant to address that problem.

This risk could be greatly mitigated by changing the semver matching policy to consume the oldest matching version instead of the newest matching version.

this is the opposite of npm's stance on semver. automatically including patch versions with bugfixes is exactly what we want to happen. if you, personally, don't want that, you are free to use other semver expressions or specify a single specific version.

also: semver is not something invented at npm and we do not plan on changing it. for more info, see http://semver.org/

@evanlucas

This comment has been minimized.

Show comment
Hide comment
@evanlucas

evanlucas Mar 29, 2016

Are private packages hosted by npm considered to be dependents when a request to be removed comes in?

Are private packages hosted by npm considered to be dependents when a request to be removed comes in?

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Mar 29, 2016

@ashleygwilliams I understand and believe in semver. But semver describes a way to describe and match compatible versions. The policy on which version you install given a particular semver range is a decision of npm's, not a mandate of semver.

I am simply pointing out a risk as a result of the policy of automatically installing the latest version matching the semver and proposing an alternative. This is consistent with the default behavior of other package managers that I've used (Maven and Nuget). I understand what the current stance is, the point of this comment is to ask that it be reconsidered, just as you reconsidered the policy on unpublishing.

I realize this policy isn't meant to address this problem - but in the context of everything that happened it seems a reasonable time to discuss the security issues that the events of last week brought to light.

@ashleygwilliams I understand and believe in semver. But semver describes a way to describe and match compatible versions. The policy on which version you install given a particular semver range is a decision of npm's, not a mandate of semver.

I am simply pointing out a risk as a result of the policy of automatically installing the latest version matching the semver and proposing an alternative. This is consistent with the default behavior of other package managers that I've used (Maven and Nuget). I understand what the current stance is, the point of this comment is to ask that it be reconsidered, just as you reconsidered the policy on unpublishing.

I realize this policy isn't meant to address this problem - but in the context of everything that happened it seems a reasonable time to discuss the security issues that the events of last week brought to light.

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 29, 2016

Contributor

@jamietre at this moment, we aren't. i'd be interested in reading Maven's and/or Nuget's rationale if you happen to have any links at hand, though.

Contributor

ashleygwilliams commented Mar 29, 2016

@jamietre at this moment, we aren't. i'd be interested in reading Maven's and/or Nuget's rationale if you happen to have any links at hand, though.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Mar 29, 2016

Member

@jamietre If the reason for a later-than-24-hours unpublish is "this thing has a security vuln", then I'm sure that support will act appropriately. (In fact, in that case, it should be sent to security@npmjs.com and/or abuse@npmjs.com instead!)

@tlrobinson Ah, ok, if it's a nested dep, then updating the shrinkwrap is indeed trickier. Sorry, I didn't catch that on the first pass.

In general, yes, we'll almost always use a deprecation warning for a period of time before deleting the package. It's a good practice anyway.

Member

isaacs commented Mar 29, 2016

@jamietre If the reason for a later-than-24-hours unpublish is "this thing has a security vuln", then I'm sure that support will act appropriately. (In fact, in that case, it should be sent to security@npmjs.com and/or abuse@npmjs.com instead!)

@tlrobinson Ah, ok, if it's a nested dep, then updating the shrinkwrap is indeed trickier. Sorry, I didn't catch that on the first pass.

In general, yes, we'll almost always use a deprecation warning for a period of time before deleting the package. It's a good practice anyway.

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Mar 29, 2016

@ashleygwilliams Here's a couple refs about nuget:

Release notes for 2.8, where the strategy was first introduced

Blog that refers to it

"NuGet by default loads the lowest possible version it can match. In this case that’s 2.50, even though 2.55 is available. There’s a bit of discussion on why this somewhat unintuitive behavior occurs. Summarized, it amounts to this: lower versions are safer and avoid pulling in newer versions of components that might break existing applications/components that depend on the same component. It avoids pulling in newer versions in unconditionally."

I am actually having a really hard time finding anything specific about Maven. The discussion and documentation is very fragmented. I always hated it anyway and am glad I don't use it anymore so maybe not the best example ;) but I will continue to look around.

@isaacs in the scenario I'm imagining, the damage would be done long before 24 hours had passed! The combination of relatively weak security for npm authentication (username/password) and automatic-latest-version-consuming is a juicy attack vector for a mischief maker.

@ashleygwilliams Here's a couple refs about nuget:

Release notes for 2.8, where the strategy was first introduced

Blog that refers to it

"NuGet by default loads the lowest possible version it can match. In this case that’s 2.50, even though 2.55 is available. There’s a bit of discussion on why this somewhat unintuitive behavior occurs. Summarized, it amounts to this: lower versions are safer and avoid pulling in newer versions of components that might break existing applications/components that depend on the same component. It avoids pulling in newer versions in unconditionally."

I am actually having a really hard time finding anything specific about Maven. The discussion and documentation is very fragmented. I always hated it anyway and am glad I don't use it anymore so maybe not the best example ;) but I will continue to look around.

@isaacs in the scenario I'm imagining, the damage would be done long before 24 hours had passed! The combination of relatively weak security for npm authentication (username/password) and automatic-latest-version-consuming is a juicy attack vector for a mischief maker.

@mcollina

This comment has been minimized.

Show comment
Hide comment
@mcollina

mcollina Mar 29, 2016

@isaacs @ashleygwilliams this is an awesome news. Congrats and well done!

@isaacs @ashleygwilliams this is an awesome news. Congrats and well done!

@Krettis

This comment has been minimized.

Show comment
Hide comment
@Krettis

Krettis Mar 29, 2016

What about this: Why not use a uuid instead of a namespace in your package depency list? The namespace ( plus semver ) could be something what is public provided by npm. But the uuid's are only known by the packages who use the package.

For example
You are using package 'supertool'
Supertool has a depency of package 'blacksheep@4.2.2' with the uuid 94653e3f-50ec-47e5-9666-b8e7d0d60bcc in your package list
Package 'sunbutshine' also has the same blacksheep@4.2.2 but with the uuid b946abb7-3d50-4ad6-8ade-55d029a8bb7d
This way the focus is not on the package blacksheep itself but on what links between two packages. Doesn't matter anymore if the packages namespace is public or not. This package with its version will never be broken, because the link-uuid is what counts and will never be deleted. Even though the package could be unpublished 50 more times, it will never be broken for no one.

Suddenly...
Corp. OzzyOzz claims package Blacksheep. Namespaces need to be unpublish. No worries. The links supertool and sunbutshine will always be available.

A new challenger appears
If a new package called 'WolfNeedBreakfast' would use the uuid of supertool or sunbutshine npm could easily block cause they aren't the authorized package to use it. They would need to create a new link by referencing to the namespace plus version.

Unpublish within 24 hours
You would only unpublish the public namespace+version. So for example 'Blacksheep@1.0.0' would have a lot of problems and goes back to 'Blacksheep@0.6.3'. In between this time it is up to the other folks who upgraded to the first version if they still use this version or not. It needs to be static imo, because it is already out in the open. The next version Blacksheep when they have a fix is according to semver 'blacksheep@1.0.1'. And blacksheep@1.0.0 will never be distributed as a namespace by npm itself.

Krettis commented Mar 29, 2016

What about this: Why not use a uuid instead of a namespace in your package depency list? The namespace ( plus semver ) could be something what is public provided by npm. But the uuid's are only known by the packages who use the package.

For example
You are using package 'supertool'
Supertool has a depency of package 'blacksheep@4.2.2' with the uuid 94653e3f-50ec-47e5-9666-b8e7d0d60bcc in your package list
Package 'sunbutshine' also has the same blacksheep@4.2.2 but with the uuid b946abb7-3d50-4ad6-8ade-55d029a8bb7d
This way the focus is not on the package blacksheep itself but on what links between two packages. Doesn't matter anymore if the packages namespace is public or not. This package with its version will never be broken, because the link-uuid is what counts and will never be deleted. Even though the package could be unpublished 50 more times, it will never be broken for no one.

Suddenly...
Corp. OzzyOzz claims package Blacksheep. Namespaces need to be unpublish. No worries. The links supertool and sunbutshine will always be available.

A new challenger appears
If a new package called 'WolfNeedBreakfast' would use the uuid of supertool or sunbutshine npm could easily block cause they aren't the authorized package to use it. They would need to create a new link by referencing to the namespace plus version.

Unpublish within 24 hours
You would only unpublish the public namespace+version. So for example 'Blacksheep@1.0.0' would have a lot of problems and goes back to 'Blacksheep@0.6.3'. In between this time it is up to the other folks who upgraded to the first version if they still use this version or not. It needs to be static imo, because it is already out in the open. The next version Blacksheep when they have a fix is according to semver 'blacksheep@1.0.1'. And blacksheep@1.0.0 will never be distributed as a namespace by npm itself.

@eddieajau

This comment has been minimized.

Show comment
Hide comment
@eddieajau

eddieajau Mar 29, 2016

I think the changes are good overall, but here are a few comments/thoughts:

  1. I think the policy should only apply to public packages, not private packages (that have always been private).
  2. However, a public package that changes to private access should be considered equivalent to unpublishing.
  3. I think you may need some flexibility with the 24-hour window as that may not give corporate users enough time to mitigate a problem on the next "business day". I can actually seeing this as a hard sell to my manager and will probably scare them off at best contributing to Open Source, or at worse using npm at all even for private packages. Just something to think about.
  4. I think you might need to give an indication of what the SLA for responding to post-24-hour unpublishing requests.
  5. Does npm need to consider the use-case where a significant security vulnerability is found in a package and the maintainers are not contactable, requiring npm to intervene in some fashion (which may lead to unpublishing the vulnerable package)?

I think the changes are good overall, but here are a few comments/thoughts:

  1. I think the policy should only apply to public packages, not private packages (that have always been private).
  2. However, a public package that changes to private access should be considered equivalent to unpublishing.
  3. I think you may need some flexibility with the 24-hour window as that may not give corporate users enough time to mitigate a problem on the next "business day". I can actually seeing this as a hard sell to my manager and will probably scare them off at best contributing to Open Source, or at worse using npm at all even for private packages. Just something to think about.
  4. I think you might need to give an indication of what the SLA for responding to post-24-hour unpublishing requests.
  5. Does npm need to consider the use-case where a significant security vulnerability is found in a package and the maintainers are not contactable, requiring npm to intervene in some fashion (which may lead to unpublishing the vulnerable package)?
@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 29, 2016

Contributor

hello @machineghost. this issue is specifically to discuss the unpublish policy. it is not appropriate AND derailing to attempt to discuss name dispute policy here. i have moderated your comment.

Contributor

ashleygwilliams commented Mar 29, 2016

hello @machineghost. this issue is specifically to discuss the unpublish policy. it is not appropriate AND derailing to attempt to discuss name dispute policy here. i have moderated your comment.

@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Mar 29, 2016

I appreciate that you guys have clarified the policy. However, this line is fairly concerning:

Your Content belongs to you. You decide whether and how to license it. But at a minimum, you license npm to provide Your Content to users of npm Services when you share Your Content.

This assumes a lot about the license involved. Consider the case where the author of a piece of code does not necessarily have the legal authority to grant you this license. You might argue they do have the authority and are warranting that by publishing the code to NPM, but I suspect there are a bunch of rightsholders that may beg to differ.

If you contact support, they will check to see if removing that version of your package would break any other installs. If so, we will not remove it. You’ll either have to transfer ownership of the package or reach out to the owners of dependent packages to change their dependency.

This is also really concerning. It places the entire onus on the author of the package to contact authors that may or may not reply in a timely fashion. There is no published time limit for these replies from absentee package authors. Any author of an npm package can block any other package's removal, indefinitely, and that is terrible design.

I don't think it makes sense, in the slightest, for npm as an organization to take on the additional liability to insure some sort of sane dependency model resolution. That really needs to be on the author of the code. The "right answer" here is to fail intelligently in the face of a missing library with a responsible error message and suitable behavior that doesn't try to build something that won't work, not to run roughshod over authors of code, rightsholders, and people who want nothing more to do with your model of "we own everything, kthxbye, and if we don't like the fact that you've unpublished well then go pound sand, we're not going to respect your decision as the author of code".

I appreciate that you guys have clarified the policy. However, this line is fairly concerning:

Your Content belongs to you. You decide whether and how to license it. But at a minimum, you license npm to provide Your Content to users of npm Services when you share Your Content.

This assumes a lot about the license involved. Consider the case where the author of a piece of code does not necessarily have the legal authority to grant you this license. You might argue they do have the authority and are warranting that by publishing the code to NPM, but I suspect there are a bunch of rightsholders that may beg to differ.

If you contact support, they will check to see if removing that version of your package would break any other installs. If so, we will not remove it. You’ll either have to transfer ownership of the package or reach out to the owners of dependent packages to change their dependency.

This is also really concerning. It places the entire onus on the author of the package to contact authors that may or may not reply in a timely fashion. There is no published time limit for these replies from absentee package authors. Any author of an npm package can block any other package's removal, indefinitely, and that is terrible design.

I don't think it makes sense, in the slightest, for npm as an organization to take on the additional liability to insure some sort of sane dependency model resolution. That really needs to be on the author of the code. The "right answer" here is to fail intelligently in the face of a missing library with a responsible error message and suitable behavior that doesn't try to build something that won't work, not to run roughshod over authors of code, rightsholders, and people who want nothing more to do with your model of "we own everything, kthxbye, and if we don't like the fact that you've unpublished well then go pound sand, we're not going to respect your decision as the author of code".

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 29, 2016

Contributor

hi @arimathea. we appreciate your feedback and have considered your concerns but are ultimately going with the direction we have published. remember that contacting support re: unpublishing is the safety valve. if bad actors (i.e. dependent packages that specifically refuse to change deps) are encountered, that is something our support humans will notice and take care of.

is there a specific question you have?

Contributor

ashleygwilliams commented Mar 29, 2016

hi @arimathea. we appreciate your feedback and have considered your concerns but are ultimately going with the direction we have published. remember that contacting support re: unpublishing is the safety valve. if bad actors (i.e. dependent packages that specifically refuse to change deps) are encountered, that is something our support humans will notice and take care of.

is there a specific question you have?

@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Mar 30, 2016

  1. What is the expected round trip time of a support human?
  2. What percentage of NPM packages is "stale" e.g. not updated frequently? Are you tracking any statistics on the number of package authors whose linked github repositories are unresponsive?
  3. Under what circumstances does a currently published package fall under these new rules? Did you send a note to all npm package authors asking them to consent?
  4. In the case of a legal challenge by the author(s) or rightsholders of the code in question, do you intend to defend your ability to continue publishing the package?
  5. What are the thresholds used? Using humans implies judgment. If the package has one listed dependency, is that an issue? Ten? Twenty?
  6. Are all packages treated the same way? What about a package that has never been downloaded?
  7. If your support humans get overloaded, at what point do you anticipate migrating to a more scalable system?
  8. How long does your sensible support human expect to take resolving a package that desires an unpublish that other packages depend on and are unresponsive? Modulecounts suggests 250k packages currently in NPM. If we assume 1% of those packages might have these issues, that's 2500 total, with an additional 4 (1% of 446 average) a day, for a total of 1,460 over the course of a year. Are these statistics accurate?
  9. What do you think the proposed contact mechanism is or should be for a developer who wishes to unpublish their package? "Any means necessary"? If a developer wishes to unpublish a package, do you accept their warranty that they've made all attempts to communicate to the people that depend on their package?
  10. If a developer thinks there is someone, or multiple someones, who are infringing their license for a given package, how does NPM propose to deal with that dispute? NPM's terms of service seem to indicate there is no avenue for a developer to stop such a package from being available to the infringer short of a lengthy court battle.
  11. Does NPM intend to treat larger companies different than smaller companies when it comes to this policy?
  1. What is the expected round trip time of a support human?
  2. What percentage of NPM packages is "stale" e.g. not updated frequently? Are you tracking any statistics on the number of package authors whose linked github repositories are unresponsive?
  3. Under what circumstances does a currently published package fall under these new rules? Did you send a note to all npm package authors asking them to consent?
  4. In the case of a legal challenge by the author(s) or rightsholders of the code in question, do you intend to defend your ability to continue publishing the package?
  5. What are the thresholds used? Using humans implies judgment. If the package has one listed dependency, is that an issue? Ten? Twenty?
  6. Are all packages treated the same way? What about a package that has never been downloaded?
  7. If your support humans get overloaded, at what point do you anticipate migrating to a more scalable system?
  8. How long does your sensible support human expect to take resolving a package that desires an unpublish that other packages depend on and are unresponsive? Modulecounts suggests 250k packages currently in NPM. If we assume 1% of those packages might have these issues, that's 2500 total, with an additional 4 (1% of 446 average) a day, for a total of 1,460 over the course of a year. Are these statistics accurate?
  9. What do you think the proposed contact mechanism is or should be for a developer who wishes to unpublish their package? "Any means necessary"? If a developer wishes to unpublish a package, do you accept their warranty that they've made all attempts to communicate to the people that depend on their package?
  10. If a developer thinks there is someone, or multiple someones, who are infringing their license for a given package, how does NPM propose to deal with that dispute? NPM's terms of service seem to indicate there is no avenue for a developer to stop such a package from being available to the infringer short of a lengthy court battle.
  11. Does NPM intend to treat larger companies different than smaller companies when it comes to this policy?
@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 30, 2016

Contributor

those are great questions @arimathea, thanks for asking them. it's gonna take a second for me to get all the answers, and some of them don't exist yet, but know that i see this post and intend to answer it, though it will likely be midday tomorrow, PST.

Contributor

ashleygwilliams commented Mar 30, 2016

those are great questions @arimathea, thanks for asking them. it's gonna take a second for me to get all the answers, and some of them don't exist yet, but know that i see this post and intend to answer it, though it will likely be midday tomorrow, PST.

@arodland

This comment has been minimized.

Show comment
Hide comment
@arodland

arodland Mar 30, 2016

If you had announced this in advance of implementing it, I bet you would have seen thousands of unpublishes in the time in between (which, I'm sure, is why you didn't), because I'm sure that a substantial fraction of your existing developer population has no wish to have their work distributed through such a system. As it is, I wouldn't be surprised (or displeased) if your support queue got completely snowed under.

If you had announced this in advance of implementing it, I bet you would have seen thousands of unpublishes in the time in between (which, I'm sure, is why you didn't), because I'm sure that a substantial fraction of your existing developer population has no wish to have their work distributed through such a system. As it is, I wouldn't be surprised (or displeased) if your support queue got completely snowed under.

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 30, 2016

Contributor

we are announcing this before we've implemented it @arodland. we are announcing this because we believe developers deserve to know. if they wish to unpublish, they currently can.

we developed this policy by listening to the concerns and asks of our community. we are a platform for the community and care deeply about their interests. this is the outcome of that care. we can't please everyone but we are trying to be receptive. the overwhelming response to this policy has been extremely positive.

we know that we rely on heavily on our support people, we always have. it will be a new challenge and we are excited to see how this goes. we will probably make mistakes, but we will be open and iterate.

Contributor

ashleygwilliams commented Mar 30, 2016

we are announcing this before we've implemented it @arodland. we are announcing this because we believe developers deserve to know. if they wish to unpublish, they currently can.

we developed this policy by listening to the concerns and asks of our community. we are a platform for the community and care deeply about their interests. this is the outcome of that care. we can't please everyone but we are trying to be receptive. the overwhelming response to this policy has been extremely positive.

we know that we rely on heavily on our support people, we always have. it will be a new challenge and we are excited to see how this goes. we will probably make mistakes, but we will be open and iterate.

@arodland

This comment has been minimized.

Show comment
Hide comment
@arodland

arodland Mar 30, 2016

@ashleygwilliams I don't see any mention of a timeframe in the blog post, only "going forward" which implies that it's true from the time of the post onwards. It might be helpful to clarify that :)

@ashleygwilliams I don't see any mention of a timeframe in the blog post, only "going forward" which implies that it's true from the time of the post onwards. It might be helpful to clarify that :)

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 30, 2016

Contributor

@arodland that's good feedback- thanks!

Contributor

ashleygwilliams commented Mar 30, 2016

@arodland that's good feedback- thanks!

@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Mar 30, 2016

if they wish to unpublish, they currently can.

If a developer unpublishes today (or before your policy takes effect) a package that is depended on highly by other packages, you won't intervene? Did I hear that correctly?

if they wish to unpublish, they currently can.

If a developer unpublishes today (or before your policy takes effect) a package that is depended on highly by other packages, you won't intervene? Did I hear that correctly?

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 30, 2016

Contributor

i didn't say we wouldn't intervene, @arimathea. i imagine our feels/actions would be similar to the incident with left-pad. software takes time to write tho! my point is we have now written this policy and we wanted everyone to know ASAP. we are also implementing it ASAP. there is a gap period. it is what it is. we believe in this policy and will be using it as a guide for our future actions.

Contributor

ashleygwilliams commented Mar 30, 2016

i didn't say we wouldn't intervene, @arimathea. i imagine our feels/actions would be similar to the incident with left-pad. software takes time to write tho! my point is we have now written this policy and we wanted everyone to know ASAP. we are also implementing it ASAP. there is a gap period. it is what it is. we believe in this policy and will be using it as a guide for our future actions.

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 30, 2016

Contributor

hello @phillip-haydon @lucentminds @simov. this issue is specifically to discuss the unpublish policy. it is not appropriate AND derailing to attempt to discuss name dispute policy here, or any other topic. i have moderated your comments.

Contributor

ashleygwilliams commented Mar 30, 2016

hello @phillip-haydon @lucentminds @simov. this issue is specifically to discuss the unpublish policy. it is not appropriate AND derailing to attempt to discuss name dispute policy here, or any other topic. i have moderated your comments.

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 30, 2016

Contributor

@phillip-haydon if you continue to derail this issue and use abusive language i will ban you from the npm Github org. please note our Code of Conduct, linked above. i have moderated your comment.

Contributor

ashleygwilliams commented Mar 30, 2016

@phillip-haydon if you continue to derail this issue and use abusive language i will ban you from the npm Github org. please note our Code of Conduct, linked above. i have moderated your comment.

@dmethvin

This comment has been minimized.

Show comment
Hide comment
@dmethvin

dmethvin Mar 30, 2016

Per threads on Twitter including here we have a suggestion to implement some way to abandon/disown a published module. That circumvents the problem with owning a module you don't want to be associated with but still leaves it published with some "placeholder" new owner.

Per threads on Twitter including here we have a suggestion to implement some way to abandon/disown a published module. That circumvents the problem with owning a module you don't want to be associated with but still leaves it published with some "placeholder" new owner.

@dmethvin

This comment has been minimized.

Show comment
Hide comment
@dmethvin

dmethvin Mar 30, 2016

Also I agree with @groundwater's comment above that as long as the license permits it npm should prevent all the dependents from being broken by unpublishing. It might be good to have npm explicitly state that by publishing on npm the owner grants npm this right (if it doesn't already).

Also I agree with @groundwater's comment above that as long as the license permits it npm should prevent all the dependents from being broken by unpublishing. It might be good to have npm explicitly state that by publishing on npm the owner grants npm this right (if it doesn't already).

@Twipped

This comment has been minimized.

Show comment
Hide comment
@Twipped

Twipped Mar 31, 2016

Sorry if this question has been answered, but how does npm determine that a package/version is not being depended on in the context of private code that isn't hosted by the registry, such as the hundreds of thousands of websites that are built from NPM. Is there some threshold of download counts for a version?

Twipped commented Mar 31, 2016

Sorry if this question has been answered, but how does npm determine that a package/version is not being depended on in the context of private code that isn't hosted by the registry, such as the hundreds of thousands of websites that are built from NPM. Is there some threshold of download counts for a version?

@jmelfi

This comment has been minimized.

Show comment
Hide comment
@jmelfi

jmelfi Mar 31, 2016

@jamietre Let me propose this situation.

I wish to unpublish a library from NPM. I do so. NPM under their license is allowed to distribute, publish and copy the code. Their new terms also allow them to analyze the code (e.g. see if there's malware or such in it).

NPM is granted not a full license but a very limited one. The license is also bounded by the ending clause.

Original Terms prior to the Policy change (NOT DISCUSSED BEFORE CHANGED)

The license lasts, for each piece of Your Content, until the last copy disappears from npm's backups, caches, and other systems, after you delete it from the Website or the Public Registry.

New Terms

When Your Content is removed from the Website or the Public Registry, whether by you or npm, npm's special license ends when the last copy disappears from npm's backups, caches, and other systems. Other licenses, such as open source licenses, may continue after Your Content is removed. Those licenses may give others, or npm itself, the right to share Your Content with npm Services again.

from here

The new terms gives NPM, Inc the freedom to use copy-left type licenses to republish them from their backups, etc. The original terms did not provide this. This was not discussed with the community prior to the change. The new terms state that you can remove your software, but NPM's license now allows it to restore content basically forever (if in this case their backups are forever). I'm against NPM having this because NPM didn't provide the creative work for the code/package/library. They don't have IP rights to it. Authors have ownership rights to it while NPM was given a use-license to distribute the content until the author removed it (at least prior).

Did we just grandfather in all prior versions and packages before that day or is it as it appears to be, fully retroactive? I find your image of Vadar stating "I've altered the deal, pray I don't alter it further" as a good way to frame this. NPM is a curator of NodeJS packages in a registry. They hold ownership over the idea and framework of the registry but have zero ownership of the content. They are permitted a use-license while it is published to their registry and until it leaves their backups.

This does lead to an interesting question. Perhaps a 12th @ashleygwilliams?

  1. How long does NPM hold backups for Content of contributors? In other words, after unpublishing a library, how long until NPM, Inc. would not have the ability to republish it?

jmelfi commented Mar 31, 2016

@jamietre Let me propose this situation.

I wish to unpublish a library from NPM. I do so. NPM under their license is allowed to distribute, publish and copy the code. Their new terms also allow them to analyze the code (e.g. see if there's malware or such in it).

NPM is granted not a full license but a very limited one. The license is also bounded by the ending clause.

Original Terms prior to the Policy change (NOT DISCUSSED BEFORE CHANGED)

The license lasts, for each piece of Your Content, until the last copy disappears from npm's backups, caches, and other systems, after you delete it from the Website or the Public Registry.

New Terms

When Your Content is removed from the Website or the Public Registry, whether by you or npm, npm's special license ends when the last copy disappears from npm's backups, caches, and other systems. Other licenses, such as open source licenses, may continue after Your Content is removed. Those licenses may give others, or npm itself, the right to share Your Content with npm Services again.

from here

The new terms gives NPM, Inc the freedom to use copy-left type licenses to republish them from their backups, etc. The original terms did not provide this. This was not discussed with the community prior to the change. The new terms state that you can remove your software, but NPM's license now allows it to restore content basically forever (if in this case their backups are forever). I'm against NPM having this because NPM didn't provide the creative work for the code/package/library. They don't have IP rights to it. Authors have ownership rights to it while NPM was given a use-license to distribute the content until the author removed it (at least prior).

Did we just grandfather in all prior versions and packages before that day or is it as it appears to be, fully retroactive? I find your image of Vadar stating "I've altered the deal, pray I don't alter it further" as a good way to frame this. NPM is a curator of NodeJS packages in a registry. They hold ownership over the idea and framework of the registry but have zero ownership of the content. They are permitted a use-license while it is published to their registry and until it leaves their backups.

This does lead to an interesting question. Perhaps a 12th @ashleygwilliams?

  1. How long does NPM hold backups for Content of contributors? In other words, after unpublishing a library, how long until NPM, Inc. would not have the ability to republish it?
@tlrobinson

This comment has been minimized.

Show comment
Hide comment
@tlrobinson

tlrobinson Mar 31, 2016

@jmelfi That change seems to just clarify the existing state of things, even if the original terms didn't explicitly state it. If you publish a piece of code under any suitable open source license, even if it's not on npm, then npm could republish it, period. That's how open source works. npm wasn't magically excluded.

I really don't understand how so many of you can be in favor of open source but at the same time want to retain absolute control over how your code is distributed.

@jmelfi That change seems to just clarify the existing state of things, even if the original terms didn't explicitly state it. If you publish a piece of code under any suitable open source license, even if it's not on npm, then npm could republish it, period. That's how open source works. npm wasn't magically excluded.

I really don't understand how so many of you can be in favor of open source but at the same time want to retain absolute control over how your code is distributed.

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Mar 31, 2016

Contributor

That change seems to just clarify the existing state of things, even if the original terms didn't explicitly state it. If you publish a piece of code, even if it's not on npm, under any suitable open source license, then npm could republish it, period. That's how open source works. npm wasn't magically excluded.

💯 correct.

Contributor

ashleygwilliams commented Mar 31, 2016

That change seems to just clarify the existing state of things, even if the original terms didn't explicitly state it. If you publish a piece of code, even if it's not on npm, under any suitable open source license, then npm could republish it, period. That's how open source works. npm wasn't magically excluded.

💯 correct.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Mar 31, 2016

Member

@jmelfi The change is a clarification. If you read it closely, I think you'll find that it doesn't grant us any additional rights we didn't already reserve. (If you disagree, then that's a bug for our lawyer to fix, because it wasn't the intent to do anything but further clarify what was already the case.)

What this meant (and still means) in practice is that, though we reserve the right to distribute any code that you publish to npm as long as we have it, what we're not going to do is track that code down from other places, and put it back on the registry if it ever goes away. So, you're only giving us a right to distribute the specific artifacts that you publish; not every copy of the content ever published anywhere. When and if those artifacts are no longer in our possession, we no longer have anything that you gave us the right to distribute.

The duration of our backups is an implementation detail. We don't disclose details of our infrastructure, because doing so is a foolish move that puts the security of our community at risk. But, the hand-wavey non-answer is we store it for "a while", long enough that we can come back from a catastrophe, but not so long that the catastrophe would no longer matter. Most of the cases we're talking about either are very relevant almost immediately, or never relevant at all.

Nothing in our previous or current license guaranteed that packages could always be deleted, or would necessarily be deleted, and that was by design. It said, if you delete it, then sometime after it's deleted, once all the backups are gone, that's when our license to those artifacts ends.


There are a lot of theoretical ethical issues being brought up in this thread, but I like to focus on more pragmatic ethics: what is our moral obligation – as a service and as individual authors – to the consumers of the code that we license freely as open source? How do we do the most good, and the least harm, for the greatest number of people? How do we ensure that our power is used responsibly? I'm not just talking about "npm, Inc." here, but all of us, as developers, as part of the social fabric of OSS. Think about this.

Publishing something as open source, under a liberal license, is a gift to a community. As an author, I retain "ownership", but give up nearly every aspect of that "ownership" that might be used to extort or injure downstream users.

Once distributed, what does an owner have the right to expect of users of their ISC/BSD-2-Clause/MIT/WTFPL code?

Can I force them to delete all copies of it if I don't like them? No, I cannot.
Can I prevent them from distributing it? No, I cannot.
Can I require that they pay me if they want to make derivative works? No.

Basically all that I can do is insist that (a) my copyright notice is not changed, and (b) the license remains intact with any distribution. (And with WTFPL, not even that!) These agreements bind us to a sort of mutual non-interference; you can't sue me if this code doesn't work, and I can't sue you for using it how you please. It's peace by way of elbow-room. The overwhelming majority of open source npm packages are published under a license like these.

This is very personal for me, bordering on spiritual devotion. So much so that in 2009, I decided that I didn't want to work on anything but open source for the rest of my career. I let my boss at the time know that they could either transfer me to the YUI team, or I'd leave Yahoo. I've written some proprietary code since then, but the overwhelming majority of it has been under a liberal non-copyleft license. Because this social bond of community is important.

Yes, there are other licenses that don't grant these rights to all users. That's why use of npm requires that you do grant some limited rights to npm. We don't, for example, get the right to use or modify your code; only to distribute it. It would be prohibitively difficult to rely only on the rights granted in a publisher's LICENSE.md file to determine whether or to whom we could distribute packages, and we have tried to carve out the bare minimum required to do our job. The existence of an npm registry is a good that we all benefit from, so making its existence practical is kind of important.

(Note: for private packages, we only reserve the right to distribute packages to the people who you've designated as having access to it. But, otherwise, the points are the same; we just follow the ACL, we don't read the license to determine distributability.)

The arguments presented against ever changing our policies ("pray that I don't alter it further") would require that we have the correct policies in advance of building a service that is relevant to a community of users, or fork the community any time our policies change. But we are humans living in the real world, where such things are not possible or practical, and a package ecosystem is somewhat unlike any other kind of thing; it's not just another GitHub or a blogging platform or anything else. It changes over time, and policies much adapt as well.

I absolutely understand that it is alarming when the terms of service change, especially when it seems like the company behind the change might be trying to trick you or take advantage of you. That's why we are so open about policy and ToS changes. I encourage you to find a qualified legal professional that you can trust (ie, who you hire to work professionally on your behalf) and have them poke at our terms of service. If they find something alarming, we can discuss it. They can open an issue or file a PR to update the language on anything that seems troubling, and we'll have that conversation in the open.

There is a lot of talk here about "rights". But I think "responsibilities" are more illuminating.

In hopes of bringing this somewhat back to the OP topic, is it ever reasonable to delete code that has been distributed as open source? Maybe. But, at this stage in npm's maturity, it's a tiny minority of the time, compared with the cases where deleting something from a distribution platform causes cascading harm.

It's our responsibility as developers to understand and appreciate this gift we give to the world. And as a distribution platform, it's npm's responsibility to protect our users from having the rug torn out from under them.

npm isn't GitHub. (And, even in GitHub's case, deleting a repository can cause quite a catastrophe!) But inherent in using npm to consume and deploy code, is the assumption that the OSS code provided by others is reliable, and won't suddenly disappear.

Let's say you construct a building. It's your building; you own it. Do you have the right to burn it down?

If it's in the middle of nowhere, maybe. But npm is a sprawling metropolis at this point. So, no, you can't do that, because no matter what your "rights" are, you have a responsibility to not burn your neighbors.

Maybe you really want to be able to burn down buildings, and maybe that means you can't live in this city. But it's a nice city, and I have a responsibility to protect it.

Member

isaacs commented Mar 31, 2016

@jmelfi The change is a clarification. If you read it closely, I think you'll find that it doesn't grant us any additional rights we didn't already reserve. (If you disagree, then that's a bug for our lawyer to fix, because it wasn't the intent to do anything but further clarify what was already the case.)

What this meant (and still means) in practice is that, though we reserve the right to distribute any code that you publish to npm as long as we have it, what we're not going to do is track that code down from other places, and put it back on the registry if it ever goes away. So, you're only giving us a right to distribute the specific artifacts that you publish; not every copy of the content ever published anywhere. When and if those artifacts are no longer in our possession, we no longer have anything that you gave us the right to distribute.

The duration of our backups is an implementation detail. We don't disclose details of our infrastructure, because doing so is a foolish move that puts the security of our community at risk. But, the hand-wavey non-answer is we store it for "a while", long enough that we can come back from a catastrophe, but not so long that the catastrophe would no longer matter. Most of the cases we're talking about either are very relevant almost immediately, or never relevant at all.

Nothing in our previous or current license guaranteed that packages could always be deleted, or would necessarily be deleted, and that was by design. It said, if you delete it, then sometime after it's deleted, once all the backups are gone, that's when our license to those artifacts ends.


There are a lot of theoretical ethical issues being brought up in this thread, but I like to focus on more pragmatic ethics: what is our moral obligation – as a service and as individual authors – to the consumers of the code that we license freely as open source? How do we do the most good, and the least harm, for the greatest number of people? How do we ensure that our power is used responsibly? I'm not just talking about "npm, Inc." here, but all of us, as developers, as part of the social fabric of OSS. Think about this.

Publishing something as open source, under a liberal license, is a gift to a community. As an author, I retain "ownership", but give up nearly every aspect of that "ownership" that might be used to extort or injure downstream users.

Once distributed, what does an owner have the right to expect of users of their ISC/BSD-2-Clause/MIT/WTFPL code?

Can I force them to delete all copies of it if I don't like them? No, I cannot.
Can I prevent them from distributing it? No, I cannot.
Can I require that they pay me if they want to make derivative works? No.

Basically all that I can do is insist that (a) my copyright notice is not changed, and (b) the license remains intact with any distribution. (And with WTFPL, not even that!) These agreements bind us to a sort of mutual non-interference; you can't sue me if this code doesn't work, and I can't sue you for using it how you please. It's peace by way of elbow-room. The overwhelming majority of open source npm packages are published under a license like these.

This is very personal for me, bordering on spiritual devotion. So much so that in 2009, I decided that I didn't want to work on anything but open source for the rest of my career. I let my boss at the time know that they could either transfer me to the YUI team, or I'd leave Yahoo. I've written some proprietary code since then, but the overwhelming majority of it has been under a liberal non-copyleft license. Because this social bond of community is important.

Yes, there are other licenses that don't grant these rights to all users. That's why use of npm requires that you do grant some limited rights to npm. We don't, for example, get the right to use or modify your code; only to distribute it. It would be prohibitively difficult to rely only on the rights granted in a publisher's LICENSE.md file to determine whether or to whom we could distribute packages, and we have tried to carve out the bare minimum required to do our job. The existence of an npm registry is a good that we all benefit from, so making its existence practical is kind of important.

(Note: for private packages, we only reserve the right to distribute packages to the people who you've designated as having access to it. But, otherwise, the points are the same; we just follow the ACL, we don't read the license to determine distributability.)

The arguments presented against ever changing our policies ("pray that I don't alter it further") would require that we have the correct policies in advance of building a service that is relevant to a community of users, or fork the community any time our policies change. But we are humans living in the real world, where such things are not possible or practical, and a package ecosystem is somewhat unlike any other kind of thing; it's not just another GitHub or a blogging platform or anything else. It changes over time, and policies much adapt as well.

I absolutely understand that it is alarming when the terms of service change, especially when it seems like the company behind the change might be trying to trick you or take advantage of you. That's why we are so open about policy and ToS changes. I encourage you to find a qualified legal professional that you can trust (ie, who you hire to work professionally on your behalf) and have them poke at our terms of service. If they find something alarming, we can discuss it. They can open an issue or file a PR to update the language on anything that seems troubling, and we'll have that conversation in the open.

There is a lot of talk here about "rights". But I think "responsibilities" are more illuminating.

In hopes of bringing this somewhat back to the OP topic, is it ever reasonable to delete code that has been distributed as open source? Maybe. But, at this stage in npm's maturity, it's a tiny minority of the time, compared with the cases where deleting something from a distribution platform causes cascading harm.

It's our responsibility as developers to understand and appreciate this gift we give to the world. And as a distribution platform, it's npm's responsibility to protect our users from having the rug torn out from under them.

npm isn't GitHub. (And, even in GitHub's case, deleting a repository can cause quite a catastrophe!) But inherent in using npm to consume and deploy code, is the assumption that the OSS code provided by others is reliable, and won't suddenly disappear.

Let's say you construct a building. It's your building; you own it. Do you have the right to burn it down?

If it's in the middle of nowhere, maybe. But npm is a sprawling metropolis at this point. So, no, you can't do that, because no matter what your "rights" are, you have a responsibility to not burn your neighbors.

Maybe you really want to be able to burn down buildings, and maybe that means you can't live in this city. But it's a nice city, and I have a responsibility to protect it.

@getify

This comment has been minimized.

Show comment
Hide comment
@getify

getify Mar 31, 2016

I think "responsibilities" are more illuminating

OK, let's turn around the conversation from rights to responsibilities... but that cuts both ways, responsibilities for us and responsibilities for npm.

Clearly, npm has a responsibility to do its best to deliver packages that authors want delivered (by publishing). Does it conversely have a responsibility to not deliver packages that authors no longer want delivered? Certainly users who depend on a package that npm currently provides will think npm has a responsibility to keep delivering that package.

Giving unrestricted and total access to deletion may or may not be morally correct, and may or may not (probably not) be technically feasible, but the greater harm it does to other users overrides and must restrain, even partially. If npm has a responsibility to stop delivering packages if an author expresses that desire (for whatever reason), but stopping delivering those packages harms other users that npm also has responsibility to, we will have to compromise.

We could swing that compromise more closely to the intent of the unpublishing author, or more closely the desires of the dependents. I would hope that npm would more feel a responsibility to the former than the latter. I propose that npm's responsibility is to maximally respect as much of the author's "unpublish" intent as possible, and therefore minimally but sufficiently respect the dependent's need.

The how of doing that is some combination of what I've already proposed.

getify commented Mar 31, 2016

I think "responsibilities" are more illuminating

OK, let's turn around the conversation from rights to responsibilities... but that cuts both ways, responsibilities for us and responsibilities for npm.

Clearly, npm has a responsibility to do its best to deliver packages that authors want delivered (by publishing). Does it conversely have a responsibility to not deliver packages that authors no longer want delivered? Certainly users who depend on a package that npm currently provides will think npm has a responsibility to keep delivering that package.

Giving unrestricted and total access to deletion may or may not be morally correct, and may or may not (probably not) be technically feasible, but the greater harm it does to other users overrides and must restrain, even partially. If npm has a responsibility to stop delivering packages if an author expresses that desire (for whatever reason), but stopping delivering those packages harms other users that npm also has responsibility to, we will have to compromise.

We could swing that compromise more closely to the intent of the unpublishing author, or more closely the desires of the dependents. I would hope that npm would more feel a responsibility to the former than the latter. I propose that npm's responsibility is to maximally respect as much of the author's "unpublish" intent as possible, and therefore minimally but sufficiently respect the dependent's need.

The how of doing that is some combination of what I've already proposed.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Mar 31, 2016

Member

@getify You've proposed an automated technical solution that is already pretty close to what our support folks will do for users who contact us to remove their packages. When we see a few of these, and it becomes clear what approach works, we'll consider some kind of technical solution. That might be a new cli command, or it might be some actions we take on the registry when we get a DELETE request. (EDIT: clicked save before finishing this paragraph.)

We could swing that compromise more closely to the intent of the unpublishing author, or more closely the desires of the dependents. I would hope that npm would more feel a responsibility to the former than the latter.

Can you explain why you hope that?

What is the ethical framework that says the desires of a publisher to delete their work from a distribution (especially if they have explicitly granted all holders of the artifact with the right to distribute it!) are more important than the desires of a consumer who depends on that code to not break their build?

It seems to me that balancing power and responsibility of the two actors means that the author's wishes take a back seat. They have forethought and control; they are visible, and involved, entering into the situation knowingly. They have power. The consumer will be shocked with a sudden disaster. They have less power, any way you cut it.

And, assuming that the package is one of the ~99% or so published under liberal non-copyleft licenses, the author has already given it to the world. No take-backs in OSS.

I'm genuinely curious about how you reason out the ethical responsibilities here, I'm not asking to be difficult. It may be that we are operating in different frameworks, and oh well, I'm in charge of npm and you're not, so willy nilly npm follows my framework and not yours, but I'd really like to at least identify this as an impasse before shrugging it away.

Member

isaacs commented Mar 31, 2016

@getify You've proposed an automated technical solution that is already pretty close to what our support folks will do for users who contact us to remove their packages. When we see a few of these, and it becomes clear what approach works, we'll consider some kind of technical solution. That might be a new cli command, or it might be some actions we take on the registry when we get a DELETE request. (EDIT: clicked save before finishing this paragraph.)

We could swing that compromise more closely to the intent of the unpublishing author, or more closely the desires of the dependents. I would hope that npm would more feel a responsibility to the former than the latter.

Can you explain why you hope that?

What is the ethical framework that says the desires of a publisher to delete their work from a distribution (especially if they have explicitly granted all holders of the artifact with the right to distribute it!) are more important than the desires of a consumer who depends on that code to not break their build?

It seems to me that balancing power and responsibility of the two actors means that the author's wishes take a back seat. They have forethought and control; they are visible, and involved, entering into the situation knowingly. They have power. The consumer will be shocked with a sudden disaster. They have less power, any way you cut it.

And, assuming that the package is one of the ~99% or so published under liberal non-copyleft licenses, the author has already given it to the world. No take-backs in OSS.

I'm genuinely curious about how you reason out the ethical responsibilities here, I'm not asking to be difficult. It may be that we are operating in different frameworks, and oh well, I'm in charge of npm and you're not, so willy nilly npm follows my framework and not yours, but I'd really like to at least identify this as an impasse before shrugging it away.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Mar 31, 2016

Member

I'm genuinely curious about how you reason out the ethical responsibilities here, I'm not asking to be difficult. It may be that we are operating in different frameworks, and oh well, I'm in charge of npm and you're not, so willy nilly npm follows my framework and not yours, but I'd really like to at least identify this as an impasse before shrugging it away.

@getify Sorry, on re-reading, I realize that sounds very flip/dismissive.

What I mean is, this is a case where reasonable people can disagree, but even if no agreement is possible, I'd still like to understand where you're coming from. I've learned in the past that you're a smart and thoughtful person, and though we do sometimes fundamentally just disagree on things, it's usually for interesting reasons.

Member

isaacs commented Mar 31, 2016

I'm genuinely curious about how you reason out the ethical responsibilities here, I'm not asking to be difficult. It may be that we are operating in different frameworks, and oh well, I'm in charge of npm and you're not, so willy nilly npm follows my framework and not yours, but I'd really like to at least identify this as an impasse before shrugging it away.

@getify Sorry, on re-reading, I realize that sounds very flip/dismissive.

What I mean is, this is a case where reasonable people can disagree, but even if no agreement is possible, I'd still like to understand where you're coming from. I've learned in the past that you're a smart and thoughtful person, and though we do sometimes fundamentally just disagree on things, it's usually for interesting reasons.

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Mar 31, 2016

I can't agree about your definition of responsibility in this context. The only responsibility of nails is to hold a house up. If you can't depend on your nails to do that, then you need to find new nails. This incident quite literally made node and npm the butt of every joke for two weeks. It made the people I work for literally ask me, as the person primarily responsible for choosing node for an important project, can we depend on this technology? Was this the right choice?

The point of npm is to help people make software, not feel good. I make software for a living. I also contribute to the community both as a byproduct of doing that, and sometimes deliberately, in my spare time, expecting nothing in return. But at the end of all that, I rely on tools to be successful. If npm is not dependable, then it will not be taken seriously by software professionals. If all node and npm ever hope to be is a toy for hobbyists, then fine. But that is not all they hope to be, and I certainly think that is not all that enthusiasts and boosters of the technology, community and ecosystem hope for it to be.

I someone doesn't like the weight that comes with that necessity, then respectfully, I suggest that they simply opt out of participating in the system. But for the system to work there need to be ground rules. And one of those is that you need to be able to depend on your build not just breaking randomly. Policies such as these are the only way to ensure that.

I can't agree about your definition of responsibility in this context. The only responsibility of nails is to hold a house up. If you can't depend on your nails to do that, then you need to find new nails. This incident quite literally made node and npm the butt of every joke for two weeks. It made the people I work for literally ask me, as the person primarily responsible for choosing node for an important project, can we depend on this technology? Was this the right choice?

The point of npm is to help people make software, not feel good. I make software for a living. I also contribute to the community both as a byproduct of doing that, and sometimes deliberately, in my spare time, expecting nothing in return. But at the end of all that, I rely on tools to be successful. If npm is not dependable, then it will not be taken seriously by software professionals. If all node and npm ever hope to be is a toy for hobbyists, then fine. But that is not all they hope to be, and I certainly think that is not all that enthusiasts and boosters of the technology, community and ecosystem hope for it to be.

I someone doesn't like the weight that comes with that necessity, then respectfully, I suggest that they simply opt out of participating in the system. But for the system to work there need to be ground rules. And one of those is that you need to be able to depend on your build not just breaking randomly. Policies such as these are the only way to ensure that.

@koresar

This comment has been minimized.

Show comment
Hide comment
@koresar

koresar Apr 1, 2016

My story of unpublish.

  1. I accidentally published a private package to be public. Noticed that in 48 hours. So, I quickly unpublished it.
  2. I accidentally published another public package as a different user. Noticed that in 7 days. So, I quickly unpublished it and republished as myself.

In both cases nothing depended on those packages. But the first case is a security issue.

I might repeat one of the numerous comments above, but here are my suggestions.

  1. If a package is not downloaded more than 10/100 times in a day then allow unpublishing it without going to NPM support.
  2. Allow any time unpublishing of private packages.

Thanks.

koresar commented Apr 1, 2016

My story of unpublish.

  1. I accidentally published a private package to be public. Noticed that in 48 hours. So, I quickly unpublished it.
  2. I accidentally published another public package as a different user. Noticed that in 7 days. So, I quickly unpublished it and republished as myself.

In both cases nothing depended on those packages. But the first case is a security issue.

I might repeat one of the numerous comments above, but here are my suggestions.

  1. If a package is not downloaded more than 10/100 times in a day then allow unpublishing it without going to NPM support.
  2. Allow any time unpublishing of private packages.

Thanks.

@ELLIOTTCABLE

This comment has been minimized.

Show comment
Hide comment
@ELLIOTTCABLE

ELLIOTTCABLE Apr 1, 2016

@koresar I'd posit that those security/emergency things don't need special, technical solutions: it's literally npm, Inc.'s job to be on-the-ball about those things.

What you don't want, is a technical solution that can be abused, there; what you're really asking for, even if you don't realize it, is a service-level agreement for the free-tier of npm's service re: security concerns. “If I accidentally publish a sensitive package, I need the lag-time between me realizing that, and it being resolved, to be as short as possible.” A technical solution may be fast, but a human can be just as fast, if there's a professional monitoring security@npmjs.com and whatnot, and can also be more flexible and helpful in that very-important situation.


edited security email address -- @isaacs

@koresar I'd posit that those security/emergency things don't need special, technical solutions: it's literally npm, Inc.'s job to be on-the-ball about those things.

What you don't want, is a technical solution that can be abused, there; what you're really asking for, even if you don't realize it, is a service-level agreement for the free-tier of npm's service re: security concerns. “If I accidentally publish a sensitive package, I need the lag-time between me realizing that, and it being resolved, to be as short as possible.” A technical solution may be fast, but a human can be just as fast, if there's a professional monitoring security@npmjs.com and whatnot, and can also be more flexible and helpful in that very-important situation.


edited security email address -- @isaacs

@getify

This comment has been minimized.

Show comment
Hide comment
@getify

getify Apr 1, 2016

Policies such as these are the only way to ensure that.

No, they aren't the only way. There's a spectrum of possible solutions that still fulfill keeping people's software from breaking on lost dependencies and balance that with author's desires.

getify commented Apr 1, 2016

Policies such as these are the only way to ensure that.

No, they aren't the only way. There's a spectrum of possible solutions that still fulfill keeping people's software from breaking on lost dependencies and balance that with author's desires.

@getify

This comment has been minimized.

Show comment
Hide comment
@getify

getify Apr 1, 2016

solution that is already pretty close

Does deprecation remove a package from the search results? Does it remove (actually unpublish!) any versions which weren't legitimately depended upon at that moment? Does it monitor in the future and remove versions as they become un-depended as devs migrate away from the deprecated package, such that eventually the package may go away entirely (as an author wanted)?

Not trying to be difficult -- I know you said "close" -- but I do think those parts are actually pretty important in swinging the balance of responsibility back toward the author. I appreciate your suggestion earlier of how to get part of the way there with a shell alias; I'm just explaining why I didn't drop the issue at that point, because in this case I don't personally think "close" is quite "close enough".

getify commented Apr 1, 2016

solution that is already pretty close

Does deprecation remove a package from the search results? Does it remove (actually unpublish!) any versions which weren't legitimately depended upon at that moment? Does it monitor in the future and remove versions as they become un-depended as devs migrate away from the deprecated package, such that eventually the package may go away entirely (as an author wanted)?

Not trying to be difficult -- I know you said "close" -- but I do think those parts are actually pretty important in swinging the balance of responsibility back toward the author. I appreciate your suggestion earlier of how to get part of the way there with a shell alias; I'm just explaining why I didn't drop the issue at that point, because in this case I don't personally think "close" is quite "close enough".

@getify

This comment has been minimized.

Show comment
Hide comment
@getify

getify Apr 1, 2016

What is the ethical framework that says the desires of a publisher to delete their work from a distribution (especially if they have explicitly granted all holders of the artifact with the right to distribute it!) are more important than the desires of a consumer who depends on that code to not break their build?

I am not basing my assertions on any formal ethical framework. I'm operating from a loose set of convictions and intuitions. I can try to illustrate my position informally, though, with some observations/assertions:

  1. npm continues to exist and thrive because of its users, both authors (publishers) and dependents. if either constituency wasn't strong, the value of npm overall would be severely diminished.

  2. however, some arbitrary % drop in authors wanting to trust npm enough to use it for distribution would harm the future of npm more than that same % drop in dependents choosing to install from npm. that is, to say, while both authors and dependents are important, it's not exactly a 50/50 importance. similar to the relationship conferences have with their speakers and attendees.

  3. authors place a profound and extensive trust (aka reliance) in the system when they choose to let it distribute their IP, and that trust is in large part based on knowing that the system will have to act on their behalf at the amplified scale.

    I as an individual author cannot practically manage to distribute my code to all my dependents without your help. I can't manage notifications to them when I've updated without your help. And, importantly, I can't signal to them that I am withdrawing support or endorsement of a particular package or version without your help.

    Your my proxy. I've trusted you with my most valuable asset, and in exchange, I need to know that you will proactively represent me at the consumer scale of your system in as many facets as is practical, even if that action is to withdraw.

  4. an asymmetric balance where your willingness to distribute on my behalf exceeds your willingness to withdraw on my behalf produces a chilling effect, even slightly, that causes me to question each time I want to publish. What if I want to remove this later? Will npm be less eager to represent me then than they are to help me now?

  5. If I as an author feel like the system isn't representing all my interests and desires, the main value (distributing my code) erodes. In other words, purely neutral hands-off distribution without representation of interests doesn't actually give me much that I couldn't get by just paying for a space on a cloud CDN (and some SEO efforts to be found in search engines).

    And as a side note on the topic of paying: I pay monthly for an npm account because I believe in and want to support npm and thank it for representing my interests -- I don't actually use any of the paid-account features. The same is true of my GH account, btw. I'm not suggesting that paying entitles me to anything, but just asserting that I'm not expecting npm to give me for free what I could be paying for. My ethics dictate that I want to pay you for representing me.

  6. Nothing that I've suggested actually de-stabilizes the system or harms dependent's ability to trust in and rely on npm. That is, you're able to serve their needs on a minimal basis by merely making sure an install never fails. They're not really relying on preservation of any of the bits that I'm suggesting you could swing back in favor of the author. In other words, how could a dependent possibly complain that npm let an author do any of the things I've suggested should be possible?

To sum this all up, my informal and arbitrary ethical framework asserts:

Authors have ceded control of their most valuable asset(s) to npm. In exchange, wherever it's possible to serve both author's and dependent's needs, authors need and rely on the system to protect as much of their interest as possible, up to yet not passing the point where a dependent is harmed.

getify commented Apr 1, 2016

What is the ethical framework that says the desires of a publisher to delete their work from a distribution (especially if they have explicitly granted all holders of the artifact with the right to distribute it!) are more important than the desires of a consumer who depends on that code to not break their build?

I am not basing my assertions on any formal ethical framework. I'm operating from a loose set of convictions and intuitions. I can try to illustrate my position informally, though, with some observations/assertions:

  1. npm continues to exist and thrive because of its users, both authors (publishers) and dependents. if either constituency wasn't strong, the value of npm overall would be severely diminished.

  2. however, some arbitrary % drop in authors wanting to trust npm enough to use it for distribution would harm the future of npm more than that same % drop in dependents choosing to install from npm. that is, to say, while both authors and dependents are important, it's not exactly a 50/50 importance. similar to the relationship conferences have with their speakers and attendees.

  3. authors place a profound and extensive trust (aka reliance) in the system when they choose to let it distribute their IP, and that trust is in large part based on knowing that the system will have to act on their behalf at the amplified scale.

    I as an individual author cannot practically manage to distribute my code to all my dependents without your help. I can't manage notifications to them when I've updated without your help. And, importantly, I can't signal to them that I am withdrawing support or endorsement of a particular package or version without your help.

    Your my proxy. I've trusted you with my most valuable asset, and in exchange, I need to know that you will proactively represent me at the consumer scale of your system in as many facets as is practical, even if that action is to withdraw.

  4. an asymmetric balance where your willingness to distribute on my behalf exceeds your willingness to withdraw on my behalf produces a chilling effect, even slightly, that causes me to question each time I want to publish. What if I want to remove this later? Will npm be less eager to represent me then than they are to help me now?

  5. If I as an author feel like the system isn't representing all my interests and desires, the main value (distributing my code) erodes. In other words, purely neutral hands-off distribution without representation of interests doesn't actually give me much that I couldn't get by just paying for a space on a cloud CDN (and some SEO efforts to be found in search engines).

    And as a side note on the topic of paying: I pay monthly for an npm account because I believe in and want to support npm and thank it for representing my interests -- I don't actually use any of the paid-account features. The same is true of my GH account, btw. I'm not suggesting that paying entitles me to anything, but just asserting that I'm not expecting npm to give me for free what I could be paying for. My ethics dictate that I want to pay you for representing me.

  6. Nothing that I've suggested actually de-stabilizes the system or harms dependent's ability to trust in and rely on npm. That is, you're able to serve their needs on a minimal basis by merely making sure an install never fails. They're not really relying on preservation of any of the bits that I'm suggesting you could swing back in favor of the author. In other words, how could a dependent possibly complain that npm let an author do any of the things I've suggested should be possible?

To sum this all up, my informal and arbitrary ethical framework asserts:

Authors have ceded control of their most valuable asset(s) to npm. In exchange, wherever it's possible to serve both author's and dependent's needs, authors need and rely on the system to protect as much of their interest as possible, up to yet not passing the point where a dependent is harmed.

@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Apr 1, 2016

What is the ethical framework that says the desires of a publisher to delete their work from a distribution (especially if they have explicitly granted all holders of the artifact with the right to distribute it!) are more important than the desires of a consumer who depends on that code to not break their build?

If you run a package repository that has only one package in it, are you, as an entity, relevant? What if you have no packages? You do not exist without the authors of software (and more importantly, you have dramatically less relevance the less ubiquitous you are - get back to that in a moment).

Did I write the code I submitted to you, or not? Is it mine or is it yours?

Your statement is "you submitted it to us, clearly it is now public domain" - bzzt, wrong answer. No, I wrote a license in it that says yes, it's open source, but you must not use it in a commercial product.

If I submitted it to you, should my rights, whatever they are, be respected?

On every single invocation a console.log message is created, "Don't add this package to dependencies! It's broken! I added it for whatever reason I chose to add it to npm but don't use it!"

Should my rights be respected? Your answer is a square "no" because someone decided to depend on my package... I sent them a mail telling (asking?) them to not do that, that they need to remove the dependency, their response is to tell me to go pound sand.

I forward that response to you. What is your decision? Is it case by case? Does it change based on the fact that the person depending on the package is a multimillion dollar entity and can kill "npm, Inc." at whim? What are the criteria you use to make your decision? When does your patience expire? What do I have to do to get rid of the package?

Publishers are everything, Isaac. It's really confusing that npm's team doesn't realize this. Some people seem to think the intent of npm is to always protect the rights of the consumers of that code, and that is what you (npm) states, over and over again, clearly and unambiguously. "Builds should never break!" "Code should always be available!" "No way, Spotify's been using that code forever, it can't be gone!"

Who cares? I wrote it. It's my code. Yes, I published on npm, but I didn't assume that meant it would be available forever. I didn't assume I was giving up all rights to it by publishing it. And more importantly, you are deciding that someone's decision to depend on my code is what controls its availability. I have no control over it - none. You cannot satisfy both worlds in a case like this. There are a lot of ways you CAN satisfy both worlds, but for whatever reason npm has just decided that's too much work and we're not going to bother, despite the well-reasoned objections in what has become a lengthy thread. There is a feeling that this is some sort of edge case, that we are somehow extremists because we believe you should respect our desires as the person that writes code that helps make you relevant. A feeling that somehow we are extreme because we believe we retain some rights to some code we published at some point.

Github is different from npm, we agree. But in many ways, github is just as useful as a model for the intent of a software author as anything else. If I remove my package from github, it is gone. The ability for github to continue serving my public repository ends at the moment I decide it does. And actually this serves as "removal" for a number of packaging systems that point to the actual location of code.

Let's take another example - say something like d3 or Imagemagick suddenly disappears wholesale because someone asserted a patent claim and it has to go away - it's depended on by a ton of things and it will break the whole world if it goes away. You gonna just up and remove that? No, because it will break the whole world? Now you are suddenly at odds with the patent holder AND the author. You disclaim liability? Okay, great, i'm sure that will work out well for you - here's a hint, it won't.

You have proposed (instituted) a process that clearly does not support the right of authors and lands squarely in favor of the consumer because you have decided (whether as a "clarification" or as a "new policy") that all packages submitted to npm are assumed to be "out in the wild". That is an assumption that authors did not always make. You are assuming they're okay with it because they (or someone else!) put their code in your packaging system. That is a bad assumption.

We just have folks who disagree. Those folks are probably going to go elsewhere. It's fine - but this approach hurts. It hurts you. It hurts npm. It hurts the community.

These are ways you can remediate the problem:

  1. Only licenses which grant perpetual distribution can be included in npm. This means if you see a license you don't understand or which doesn't include that statement then the package can't be submitted.
  2. If a package is "unpublished" by the author, provide increasing warnings until the package is finally removed. Set a finite timetable.
  3. Make it so that builds using 'npm install' fail intelligently in the face of dependencies, and consumers of npm are notified of these constraints when first installing an npm package (of course including a flag to say "yes, I understand")
  4. Allow an author to flag a "NODEPEND" entry in the npm entry for his package, indicating that the author does NOT want their package depended on by other packages. If existing packages come to depend on this, warn loudly. Unpublishes should be executed immediately for these packages.
  5. Allow authors to abandon a package (removing their name/identifier from the package).
  6. Combinations of the above.
  7. I'm sure a bunch of other stuff I haven't thought of.

And pretty much everything @getify said.

EDIT: Also want to add that I understand this is very personal for you and the whole team. But npm is not just you and the whole team. It is everyone that has ever contributed code to your repository. If you care about npm's success, and I know you do, you will revisit this policy and take a couple of weeks to think about how you can respect everyone's rights, and not just people who are too lazy to do the right thing by not thinking through the implications of depending on software they don't control, even if it looks, feels, sounds, smells, and behaves perfectly.

What is the ethical framework that says the desires of a publisher to delete their work from a distribution (especially if they have explicitly granted all holders of the artifact with the right to distribute it!) are more important than the desires of a consumer who depends on that code to not break their build?

If you run a package repository that has only one package in it, are you, as an entity, relevant? What if you have no packages? You do not exist without the authors of software (and more importantly, you have dramatically less relevance the less ubiquitous you are - get back to that in a moment).

Did I write the code I submitted to you, or not? Is it mine or is it yours?

Your statement is "you submitted it to us, clearly it is now public domain" - bzzt, wrong answer. No, I wrote a license in it that says yes, it's open source, but you must not use it in a commercial product.

If I submitted it to you, should my rights, whatever they are, be respected?

On every single invocation a console.log message is created, "Don't add this package to dependencies! It's broken! I added it for whatever reason I chose to add it to npm but don't use it!"

Should my rights be respected? Your answer is a square "no" because someone decided to depend on my package... I sent them a mail telling (asking?) them to not do that, that they need to remove the dependency, their response is to tell me to go pound sand.

I forward that response to you. What is your decision? Is it case by case? Does it change based on the fact that the person depending on the package is a multimillion dollar entity and can kill "npm, Inc." at whim? What are the criteria you use to make your decision? When does your patience expire? What do I have to do to get rid of the package?

Publishers are everything, Isaac. It's really confusing that npm's team doesn't realize this. Some people seem to think the intent of npm is to always protect the rights of the consumers of that code, and that is what you (npm) states, over and over again, clearly and unambiguously. "Builds should never break!" "Code should always be available!" "No way, Spotify's been using that code forever, it can't be gone!"

Who cares? I wrote it. It's my code. Yes, I published on npm, but I didn't assume that meant it would be available forever. I didn't assume I was giving up all rights to it by publishing it. And more importantly, you are deciding that someone's decision to depend on my code is what controls its availability. I have no control over it - none. You cannot satisfy both worlds in a case like this. There are a lot of ways you CAN satisfy both worlds, but for whatever reason npm has just decided that's too much work and we're not going to bother, despite the well-reasoned objections in what has become a lengthy thread. There is a feeling that this is some sort of edge case, that we are somehow extremists because we believe you should respect our desires as the person that writes code that helps make you relevant. A feeling that somehow we are extreme because we believe we retain some rights to some code we published at some point.

Github is different from npm, we agree. But in many ways, github is just as useful as a model for the intent of a software author as anything else. If I remove my package from github, it is gone. The ability for github to continue serving my public repository ends at the moment I decide it does. And actually this serves as "removal" for a number of packaging systems that point to the actual location of code.

Let's take another example - say something like d3 or Imagemagick suddenly disappears wholesale because someone asserted a patent claim and it has to go away - it's depended on by a ton of things and it will break the whole world if it goes away. You gonna just up and remove that? No, because it will break the whole world? Now you are suddenly at odds with the patent holder AND the author. You disclaim liability? Okay, great, i'm sure that will work out well for you - here's a hint, it won't.

You have proposed (instituted) a process that clearly does not support the right of authors and lands squarely in favor of the consumer because you have decided (whether as a "clarification" or as a "new policy") that all packages submitted to npm are assumed to be "out in the wild". That is an assumption that authors did not always make. You are assuming they're okay with it because they (or someone else!) put their code in your packaging system. That is a bad assumption.

We just have folks who disagree. Those folks are probably going to go elsewhere. It's fine - but this approach hurts. It hurts you. It hurts npm. It hurts the community.

These are ways you can remediate the problem:

  1. Only licenses which grant perpetual distribution can be included in npm. This means if you see a license you don't understand or which doesn't include that statement then the package can't be submitted.
  2. If a package is "unpublished" by the author, provide increasing warnings until the package is finally removed. Set a finite timetable.
  3. Make it so that builds using 'npm install' fail intelligently in the face of dependencies, and consumers of npm are notified of these constraints when first installing an npm package (of course including a flag to say "yes, I understand")
  4. Allow an author to flag a "NODEPEND" entry in the npm entry for his package, indicating that the author does NOT want their package depended on by other packages. If existing packages come to depend on this, warn loudly. Unpublishes should be executed immediately for these packages.
  5. Allow authors to abandon a package (removing their name/identifier from the package).
  6. Combinations of the above.
  7. I'm sure a bunch of other stuff I haven't thought of.

And pretty much everything @getify said.

EDIT: Also want to add that I understand this is very personal for you and the whole team. But npm is not just you and the whole team. It is everyone that has ever contributed code to your repository. If you care about npm's success, and I know you do, you will revisit this policy and take a couple of weeks to think about how you can respect everyone's rights, and not just people who are too lazy to do the right thing by not thinking through the implications of depending on software they don't control, even if it looks, feels, sounds, smells, and behaves perfectly.

@ELLIOTTCABLE

This comment has been minimized.

Show comment
Hide comment
@ELLIOTTCABLE

ELLIOTTCABLE Apr 1, 2016

So, this has been an excellent thread, and against all my expectations … I think @jmelfi et al. have actually swung me around. It's a really difficult question, and both sides keep bringing up really good points, but … there's something really important and subtle here:

Buried in all the (if I'm being honest … super-silly.) ‘legal’ and ‘licensing’ talk, there's a pretty valid concern being raised here; and I think @isaacs brushes up against it in his recent replies: I'm starting to think that it may be acting in bad faith, for npm to disallow unpublishing in the main registry.

I ended up taking so long to explain my point, here, that I've decided to leave it outside the thread. You can read the entirety here, should you wish to understand why I propose the following. All of the whys aside, though, here I've extracted a few excerpts from the actions I think might be taken, besides the obvious ‘retain unpublish’:

  • Enforce a cooling-off period. (Hell, make it proportional to some heuristic of the number of downloads-per-month / the number of dependant packages: something with no dependants disappears instantly, as unpublish currently works; something with very few disappears after an hour; and it tops out, for Lodash or Express, at, say, a week.) Every release of the package is immediately marked as deprecated / the equivalent of the above npm disown or whatever happens [...]

  • Finally [...] there's CPAN's solution: retain deprecated packages in a separate repository (obviously, one that is inherently ownership-less: as mentioned above, disown should be an inherent prerequisite / step of unpublish, no matter what else happens.) In fact, hell, I wouldn't even be opposed if you went the extra mile: allow packages to depend upon disowned-and-‘archived’ packages. As long as it requires active effort on the part of dependants, as long as it requires an active admission that the dependant is going against the wishes of the author, and drawing on that FOSS-licensed-right to continue to use the package ... then I think npm's due-diligence is fulfilled.

In fact, here, I think that bears repeating: The point here, is that I truly believe that the dependants need to each, individually, actively, indicate their willful acknowledgement of the package-authors' wishes in unpublishing. Whether that's by forking, vendoring, pointing your dependant-library's end-users to an alternative archive of the code, or in the case of that last suggestion, simply updating your dependency to "@archive/CoolPackage": "=1.2.3" and publishing a new version of your own library ... you're acknowledging the reasons the package was unpublished, as a human being, and asserting that after consideration, you still believe your needs to supersede those [...]

Basically: continuing to use⁶ (and more importantly, distribute) some software that the original author doesn't wish to be used anymore, should be the exception, not the rule. I think we should collectively very rarely resort to continuing to use / distribute / depend-on a package that the author wants to disappear, without a human actively taking upon the effort of forking, sanitizing, and maintaining the parent project. It should be an exceptional circumstance, and npm's technical decisions should reflect / encourage / even enforce that.


If I've completely failed to convince anybody with all of the above, and it's still the consensus that it's best that unpublish be removed entirely, [...]

  • [Packages] should be sanitized and hidden: remove author / description fields and similar from the package.json in an automated fashion; remove any versions not being depended upon at the time of the abandonment; and remove such packages from any public npm system (search features, the web-pages, the author-list, perhaps even remove them from the point of view of any API features; perhaps access-disowned should be a special-circumstances-granted API access-right, or smth?)
  • Finally, if possible, prevent new packages/versions from being published that depend upon them, even if you're going to allow already-dependant packages to continue to install.

So, this has been an excellent thread, and against all my expectations … I think @jmelfi et al. have actually swung me around. It's a really difficult question, and both sides keep bringing up really good points, but … there's something really important and subtle here:

Buried in all the (if I'm being honest … super-silly.) ‘legal’ and ‘licensing’ talk, there's a pretty valid concern being raised here; and I think @isaacs brushes up against it in his recent replies: I'm starting to think that it may be acting in bad faith, for npm to disallow unpublishing in the main registry.

I ended up taking so long to explain my point, here, that I've decided to leave it outside the thread. You can read the entirety here, should you wish to understand why I propose the following. All of the whys aside, though, here I've extracted a few excerpts from the actions I think might be taken, besides the obvious ‘retain unpublish’:

  • Enforce a cooling-off period. (Hell, make it proportional to some heuristic of the number of downloads-per-month / the number of dependant packages: something with no dependants disappears instantly, as unpublish currently works; something with very few disappears after an hour; and it tops out, for Lodash or Express, at, say, a week.) Every release of the package is immediately marked as deprecated / the equivalent of the above npm disown or whatever happens [...]

  • Finally [...] there's CPAN's solution: retain deprecated packages in a separate repository (obviously, one that is inherently ownership-less: as mentioned above, disown should be an inherent prerequisite / step of unpublish, no matter what else happens.) In fact, hell, I wouldn't even be opposed if you went the extra mile: allow packages to depend upon disowned-and-‘archived’ packages. As long as it requires active effort on the part of dependants, as long as it requires an active admission that the dependant is going against the wishes of the author, and drawing on that FOSS-licensed-right to continue to use the package ... then I think npm's due-diligence is fulfilled.

In fact, here, I think that bears repeating: The point here, is that I truly believe that the dependants need to each, individually, actively, indicate their willful acknowledgement of the package-authors' wishes in unpublishing. Whether that's by forking, vendoring, pointing your dependant-library's end-users to an alternative archive of the code, or in the case of that last suggestion, simply updating your dependency to "@archive/CoolPackage": "=1.2.3" and publishing a new version of your own library ... you're acknowledging the reasons the package was unpublished, as a human being, and asserting that after consideration, you still believe your needs to supersede those [...]

Basically: continuing to use⁶ (and more importantly, distribute) some software that the original author doesn't wish to be used anymore, should be the exception, not the rule. I think we should collectively very rarely resort to continuing to use / distribute / depend-on a package that the author wants to disappear, without a human actively taking upon the effort of forking, sanitizing, and maintaining the parent project. It should be an exceptional circumstance, and npm's technical decisions should reflect / encourage / even enforce that.


If I've completely failed to convince anybody with all of the above, and it's still the consensus that it's best that unpublish be removed entirely, [...]

  • [Packages] should be sanitized and hidden: remove author / description fields and similar from the package.json in an automated fashion; remove any versions not being depended upon at the time of the abandonment; and remove such packages from any public npm system (search features, the web-pages, the author-list, perhaps even remove them from the point of view of any API features; perhaps access-disowned should be a special-circumstances-granted API access-right, or smth?)
  • Finally, if possible, prevent new packages/versions from being published that depend upon them, even if you're going to allow already-dependant packages to continue to install.
@ELLIOTTCABLE

This comment has been minimized.

Show comment
Hide comment
@ELLIOTTCABLE

ELLIOTTCABLE Apr 1, 2016

(@arimathea also, most of the repetitive ‘this isn't about licensing’ parts in the above are basically written for you. While we agree in the practical, I think we differ strongly in theory: if anything, the legal side of this argument strongly supports the side of those who disagree with you, @getify, and I. You keep talking about the license of your code, and your rights … click the link above, but summarized: You, I, and Casey Arbitrary-Developer Smith all typed npm publish; and we're all bound by npm's Terms of Service. They're absolutely respecting the license of your software, in that you have already extended them the license do do what they're doing. I really think that's the wrong argument to be trying to make …)

(@arimathea also, most of the repetitive ‘this isn't about licensing’ parts in the above are basically written for you. While we agree in the practical, I think we differ strongly in theory: if anything, the legal side of this argument strongly supports the side of those who disagree with you, @getify, and I. You keep talking about the license of your code, and your rights … click the link above, but summarized: You, I, and Casey Arbitrary-Developer Smith all typed npm publish; and we're all bound by npm's Terms of Service. They're absolutely respecting the license of your software, in that you have already extended them the license do do what they're doing. I really think that's the wrong argument to be trying to make …)

@jmelfi

This comment has been minimized.

Show comment
Hide comment
@jmelfi

jmelfi Apr 1, 2016

I think that CPAN's solution is best since it requires consumers of the package to acknowledge the creator and they then also can understand why the package was archived/removed/disowned by the author/authors. It also allows authors the RTBF and such. Perhaps a way to flag a package/library as looking for a new author?

There should be I think a set time for how this will work. I'd even be happy with a transition period that the author would go thru to unpublish a package. I would support something described by @ELLIOTTCABLE where packages/libraries could be easily removed from the public npm registry but the npm registry archive would hold unpublished code, ownerless for code with properly open licenses. I feel that perhaps recommending best practice to vendor dependencies is just good practice.

@isaacs I fully acknowledge that the current language gives NPM, Inc the right to now do so. I was trying to make it clear that this was not the prior arrangement and is different from before. I really want to emphasize that community is the core of NPM. This is made up of the authors that publish to NPM and stewards of the NPM registry. This trust is paramount and, in my opinon, more important that users. I think the CPAN archive method gives a good and effective compromise as it allows users to still get code (by going thru some hoops though just really 1) and retains the important trust between publisher and distributer. Perhaps a flag that would trigger a notice if the package couldn't be placed in the archive (some non-copy-left-license that didn't allow distribution after, etc.) that would advise users to vendor that dependency? This would obviously make users wary of using the package, but that's the trade-off with packages. You now depend on it. As developers, we have to manage our dependencies that we create, either by vendoring, knowing where there is an archive, or some other method. It is not the author's job, but as publishers we should strive to always do this as gracefully as possible with warnings and notices. Author's should attempt to find other maintainers for the package if possible, especially if it is heavily used. While we can't force it, as professionals we should do what we can; though as hinted by @ELLIOTTCABLE's example, sometimes we just burn-out and need to go away, though I think an NPM archive nicely solves this for users and authors without NPM having to be an arbiter.

jmelfi commented Apr 1, 2016

I think that CPAN's solution is best since it requires consumers of the package to acknowledge the creator and they then also can understand why the package was archived/removed/disowned by the author/authors. It also allows authors the RTBF and such. Perhaps a way to flag a package/library as looking for a new author?

There should be I think a set time for how this will work. I'd even be happy with a transition period that the author would go thru to unpublish a package. I would support something described by @ELLIOTTCABLE where packages/libraries could be easily removed from the public npm registry but the npm registry archive would hold unpublished code, ownerless for code with properly open licenses. I feel that perhaps recommending best practice to vendor dependencies is just good practice.

@isaacs I fully acknowledge that the current language gives NPM, Inc the right to now do so. I was trying to make it clear that this was not the prior arrangement and is different from before. I really want to emphasize that community is the core of NPM. This is made up of the authors that publish to NPM and stewards of the NPM registry. This trust is paramount and, in my opinon, more important that users. I think the CPAN archive method gives a good and effective compromise as it allows users to still get code (by going thru some hoops though just really 1) and retains the important trust between publisher and distributer. Perhaps a flag that would trigger a notice if the package couldn't be placed in the archive (some non-copy-left-license that didn't allow distribution after, etc.) that would advise users to vendor that dependency? This would obviously make users wary of using the package, but that's the trade-off with packages. You now depend on it. As developers, we have to manage our dependencies that we create, either by vendoring, knowing where there is an archive, or some other method. It is not the author's job, but as publishers we should strive to always do this as gracefully as possible with warnings and notices. Author's should attempt to find other maintainers for the package if possible, especially if it is heavily used. While we can't force it, as professionals we should do what we can; though as hinted by @ELLIOTTCABLE's example, sometimes we just burn-out and need to go away, though I think an NPM archive nicely solves this for users and authors without NPM having to be an arbiter.

@spdustin

This comment has been minimized.

Show comment
Hide comment
@spdustin

spdustin Apr 1, 2016

@kemitchell and I have had some interesting dialog on this in #45, for those watchers who want to see another side of the discussion. I think npm is on the right path here, and with the conscientious counsel they have on their side, I think they'll get there.

spdustin commented Apr 1, 2016

@kemitchell and I have had some interesting dialog on this in #45, for those watchers who want to see another side of the discussion. I think npm is on the right path here, and with the conscientious counsel they have on their side, I think they'll get there.

@brentonhouse

This comment has been minimized.

Show comment
Hide comment
@brentonhouse

brentonhouse Apr 1, 2016

@isaacs

  1. Can ACLs for a private package be changed after 24 hours? (i.e. can the ACLs for a private package be changed to only allow owner after the 24 hour period, even if there are dependencies?)
  2. Can a public package be made private after 24 hours, if there are dependencies?

@isaacs

  1. Can ACLs for a private package be changed after 24 hours? (i.e. can the ACLs for a private package be changed to only allow owner after the 24 hour period, even if there are dependencies?)
  2. Can a public package be made private after 24 hours, if there are dependencies?
@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Apr 1, 2016

(@arimathea also, most of the repetitive ‘this isn't about licensing’ parts in the above are basically written for you. While we agree in the practical, I think we differ strongly in theory: if anything, the legal side of this argument strongly supports the side of those who disagree with you, @getify, and I. You keep talking about the license of your code, and your rights … click the link above, but summarized: You, I, and Casey Arbitrary-Developer Smith all typed npm publish; and we're all bound by npm's Terms of Service. They're absolutely respecting the license of your software, in that you have already extended them the license do do what they're doing. I really think that's the wrong argument to be trying to make …)

We agree that I have extended them the license and that we are bound by the (now clarified) terms of service. I think those are too broad. What I want is the sole authority as the author to revoke their continued distribution of my code, in as responsible a way as possible while still respecting my choice, and in a way that does not allow the potential for misinterpreted communication of my choice and does not allow outside actors to influence the process. I am making my points badly in language such as "rights", but I think there are plenty of other suitable analogies that have been made in this thread and many others that have not been made.

(@arimathea also, most of the repetitive ‘this isn't about licensing’ parts in the above are basically written for you. While we agree in the practical, I think we differ strongly in theory: if anything, the legal side of this argument strongly supports the side of those who disagree with you, @getify, and I. You keep talking about the license of your code, and your rights … click the link above, but summarized: You, I, and Casey Arbitrary-Developer Smith all typed npm publish; and we're all bound by npm's Terms of Service. They're absolutely respecting the license of your software, in that you have already extended them the license do do what they're doing. I really think that's the wrong argument to be trying to make …)

We agree that I have extended them the license and that we are bound by the (now clarified) terms of service. I think those are too broad. What I want is the sole authority as the author to revoke their continued distribution of my code, in as responsible a way as possible while still respecting my choice, and in a way that does not allow the potential for misinterpreted communication of my choice and does not allow outside actors to influence the process. I am making my points badly in language such as "rights", but I think there are plenty of other suitable analogies that have been made in this thread and many others that have not been made.

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Apr 1, 2016

@ELLIOTTCABLE Your #1 point:

"Users expect a particular behaviour. ... The user's expectations, in this case, are shaped by other package managers: and yes, crucially, most other package-managers I've dealt with, allow the users to remove their code² from the package manager"

This is an interesting comment, since I actually expected that packages could never be removed from npm, and was surprised when I learned by this incident that they could. So this statement is certainly not true for all users.

So what, actually do other package managers do? This is a good high-level overview of popular package managers. This is a list of all PMs with more than 10k registered packages:

Maven Central (Java) 137k
RubyGems (Ruby) 116k
Packagist (PHP) 90k
PyPi (Python) 77k
Nuget (.NET) 52k
Bower (JS) *** 50k
CPAN (Perl) * 33k
Drupal (PHP) **** 33k
Clojars (clojure) 15k

The package managers in bold do not permit authors to remove packages from the public registry themselves.

* CPAN is a mirror-based system: you can technically remove a package, but it does not remove it from the permanent public archive from which software can be installed automatically.

** RubyGems used to forbid unpublishing, but about a year ago changed this policy because of the administrative burden of having to unpublish manually.

*** Bower temporarily disabled unpublishing in response to the npm issue. The long-term status should be considered in debate as npm's is right now.

**** I know almost nothing about Drupal and why there would be more than one package manager for PHP, not sure if this is the same kind of system as the others,

So there is in fact a slight edge in popular PMs that do not allow unpublishing. I think it's reasonable to exclude Clojars, Drupal, CPAN, and Bower (simply because NPM is a Bower alternative, and they are also now having the same soul search) as far getting a list of the heavyweights in the room. CPAN is important, but... so.. old. So this brings it down to Maven, Nuget, Packagist, PyPi and RubyGems.

Personally i'd rather emulate the C-like langauge ecosystem than the whitespace-affects-code-flow language ecosystem ;)

jamietre commented Apr 1, 2016

@ELLIOTTCABLE Your #1 point:

"Users expect a particular behaviour. ... The user's expectations, in this case, are shaped by other package managers: and yes, crucially, most other package-managers I've dealt with, allow the users to remove their code² from the package manager"

This is an interesting comment, since I actually expected that packages could never be removed from npm, and was surprised when I learned by this incident that they could. So this statement is certainly not true for all users.

So what, actually do other package managers do? This is a good high-level overview of popular package managers. This is a list of all PMs with more than 10k registered packages:

Maven Central (Java) 137k
RubyGems (Ruby) 116k
Packagist (PHP) 90k
PyPi (Python) 77k
Nuget (.NET) 52k
Bower (JS) *** 50k
CPAN (Perl) * 33k
Drupal (PHP) **** 33k
Clojars (clojure) 15k

The package managers in bold do not permit authors to remove packages from the public registry themselves.

* CPAN is a mirror-based system: you can technically remove a package, but it does not remove it from the permanent public archive from which software can be installed automatically.

** RubyGems used to forbid unpublishing, but about a year ago changed this policy because of the administrative burden of having to unpublish manually.

*** Bower temporarily disabled unpublishing in response to the npm issue. The long-term status should be considered in debate as npm's is right now.

**** I know almost nothing about Drupal and why there would be more than one package manager for PHP, not sure if this is the same kind of system as the others,

So there is in fact a slight edge in popular PMs that do not allow unpublishing. I think it's reasonable to exclude Clojars, Drupal, CPAN, and Bower (simply because NPM is a Bower alternative, and they are also now having the same soul search) as far getting a list of the heavyweights in the room. CPAN is important, but... so.. old. So this brings it down to Maven, Nuget, Packagist, PyPi and RubyGems.

Personally i'd rather emulate the C-like langauge ecosystem than the whitespace-affects-code-flow language ecosystem ;)

@jmelfi

This comment has been minimized.

Show comment
Hide comment
@jmelfi

jmelfi Apr 1, 2016

@jamietre "This is an interesting comment, since I actually expected that packages could never be removed from npm, and was surprised when I learned by this incident that they could. So this statement is certainly not true for all users."

This comment is interesting as it is clear in the NPM documentation that packages can be unpublished. Also, NuGet and Maven allow unpublishing/removal though NuGet hides it until some time that there are no more requests for the package. (e.g. the package can't be used in new software and deprecation messages are sent to upgrade, etc.) I've had many java builds fail because those libraries became unavailable or were unpublished. You learn to manage it as a developer.

I'm not sure on Packagist. RubyGems as stated above allows unpublishing. I consider Bower a knee-jerk reaction to the NPM fallout to some degree. PyPi has always allowed unpublishing.

Drupal I do not believe allows this for all packages but any content that is not marked custom is shared under GPLv2 and custom is listed as being for that particular developer's use case only (aka. don't rely on it being there forever) This allows Drupal to distribute anything not marked custom.

I think that we are on a good path but there needs to be a way to respect authors requests that affect their work. As stated before, it can be done that if the license allows for continued distribution (including how NPM now reads unlike before) that NPM could archive it and have a process similar to CPAN where users do this for themselves to get the package and then can get context on why the package was unpublished.

Did the maintainer leave the community? not have time? huge security hole that they didn't want to fix? etc.

I think that users perception that packages are there forever is a serious flaw but not one that NPM or any registry should resolve in a way that does not respect authors wishes. I would like NPM to change back to something more akin to the current environment of package registries that developers use and contribute to.

jmelfi commented Apr 1, 2016

@jamietre "This is an interesting comment, since I actually expected that packages could never be removed from npm, and was surprised when I learned by this incident that they could. So this statement is certainly not true for all users."

This comment is interesting as it is clear in the NPM documentation that packages can be unpublished. Also, NuGet and Maven allow unpublishing/removal though NuGet hides it until some time that there are no more requests for the package. (e.g. the package can't be used in new software and deprecation messages are sent to upgrade, etc.) I've had many java builds fail because those libraries became unavailable or were unpublished. You learn to manage it as a developer.

I'm not sure on Packagist. RubyGems as stated above allows unpublishing. I consider Bower a knee-jerk reaction to the NPM fallout to some degree. PyPi has always allowed unpublishing.

Drupal I do not believe allows this for all packages but any content that is not marked custom is shared under GPLv2 and custom is listed as being for that particular developer's use case only (aka. don't rely on it being there forever) This allows Drupal to distribute anything not marked custom.

I think that we are on a good path but there needs to be a way to respect authors requests that affect their work. As stated before, it can be done that if the license allows for continued distribution (including how NPM now reads unlike before) that NPM could archive it and have a process similar to CPAN where users do this for themselves to get the package and then can get context on why the package was unpublished.

Did the maintainer leave the community? not have time? huge security hole that they didn't want to fix? etc.

I think that users perception that packages are there forever is a serious flaw but not one that NPM or any registry should resolve in a way that does not respect authors wishes. I would like NPM to change back to something more akin to the current environment of package registries that developers use and contribute to.

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Apr 1, 2016

"This comment is interesting as it is clear in the NPM documentation that packages can be unpublished."

I was responding to the notion that users expect a particular behaviour and observing that this is actually not true. The author expected a certain behavior. I expected a different one, so his assertion that "users in general" expect a thing to behave the way he did is incorrect.. Yes, I can read the documenation to see what the actual behaviour is, but that's irrelevant to the point I'm discussing here, about what people expect. As someone who has mostly used maven and nuget in the past, I expected it would act like them.

There is no automated method for a package owner to unpublish through maven or nuget. The polices are documented and crystal clear.

It's irrelevant that it is possible for packages to be removed by administrators; nobody here is saying "nobody shall ever remove anything, ever, through any means." Of course things can and will be removed in special cases. All that matters is the policy we are discussing here of whether package authors can remove their own packages for any reason at any time which is the current state, and is dangerous.

Again this notion of "respecting authors" is in direct contradiction with the terms of open source licenses. If an author wants to be "respected" in some way, than i submit, that they make those terms part of their license. Otherwise what's the point of the license at all? Why would we have a license that permits a certain action, but worry about offending someone by doing something that is entirely permissible under those terms? Our only job here is to determine what is the best course of action as far as providing a tool to end-users within the terms of use of the package and nothing more. We define these rules for exactly this purpose.

I think that policies of deprecation make a lot of sense.

  • We should allow people to remove their packages from the search engine freely.
  • We should use the already-present deprecation status
  • We should warn consumers when they install something that depends on deprecated packages, as we already do

But we cannot just stop providing access to something that people depend on, when there's no legal reason or security issue. This serves nobody.

jamietre commented Apr 1, 2016

"This comment is interesting as it is clear in the NPM documentation that packages can be unpublished."

I was responding to the notion that users expect a particular behaviour and observing that this is actually not true. The author expected a certain behavior. I expected a different one, so his assertion that "users in general" expect a thing to behave the way he did is incorrect.. Yes, I can read the documenation to see what the actual behaviour is, but that's irrelevant to the point I'm discussing here, about what people expect. As someone who has mostly used maven and nuget in the past, I expected it would act like them.

There is no automated method for a package owner to unpublish through maven or nuget. The polices are documented and crystal clear.

It's irrelevant that it is possible for packages to be removed by administrators; nobody here is saying "nobody shall ever remove anything, ever, through any means." Of course things can and will be removed in special cases. All that matters is the policy we are discussing here of whether package authors can remove their own packages for any reason at any time which is the current state, and is dangerous.

Again this notion of "respecting authors" is in direct contradiction with the terms of open source licenses. If an author wants to be "respected" in some way, than i submit, that they make those terms part of their license. Otherwise what's the point of the license at all? Why would we have a license that permits a certain action, but worry about offending someone by doing something that is entirely permissible under those terms? Our only job here is to determine what is the best course of action as far as providing a tool to end-users within the terms of use of the package and nothing more. We define these rules for exactly this purpose.

I think that policies of deprecation make a lot of sense.

  • We should allow people to remove their packages from the search engine freely.
  • We should use the already-present deprecation status
  • We should warn consumers when they install something that depends on deprecated packages, as we already do

But we cannot just stop providing access to something that people depend on, when there's no legal reason or security issue. This serves nobody.

@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Apr 1, 2016

But we cannot just stop providing access to something that people depend on, when there's no legal reason or security issue.

Do you have some good reasons why other than builds failing is inconvenient?

But we cannot just stop providing access to something that people depend on, when there's no legal reason or security issue.

Do you have some good reasons why other than builds failing is inconvenient?

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Apr 1, 2016

"Do you have some good reasons why other than builds failing is inconvenient?"

Because we want things to work as often as possible.

I'd pose the opposite question to you: given no legal or security reason to do so, why would we choose a policy that results in a less-stable ecosystem than a different policy?

Why would we not want to create the most reliable system possible, within the confines of the law, the terms of use of packages, and reasonable security concerns?

Again, you need to demonstrate some compelling reason why we should base policy on anything other than the licensing terms of a package. The license seems of little value if this is not the final arbiter of acceptable use.

jamietre commented Apr 1, 2016

"Do you have some good reasons why other than builds failing is inconvenient?"

Because we want things to work as often as possible.

I'd pose the opposite question to you: given no legal or security reason to do so, why would we choose a policy that results in a less-stable ecosystem than a different policy?

Why would we not want to create the most reliable system possible, within the confines of the law, the terms of use of packages, and reasonable security concerns?

Again, you need to demonstrate some compelling reason why we should base policy on anything other than the licensing terms of a package. The license seems of little value if this is not the final arbiter of acceptable use.

@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Apr 1, 2016

I'd pose the opposite question to you: given no legal or security reason to do so, why would we choose a policy that results in a less-stable ecosystem than a different policy?

We disagree soundly on what constitutes stability in ecosystems.
We disagree soundly on what constitutes reliable systems.
We disagree that the reasons I have provided are or are not compelling.
We disagree that things don't work better when builds based on code that have been removed from the repository by the author should, indeed, fail.

I'm not sure you will ever see my point if you don't see it already, particularly after @getify's phenomenal job of clarifying my poor prose.

Perhaps the right and final approach is that packages submitted to NPM should be accepted only if the license permits unrestricted and permanent redistribution in any form and that the author relinquishes all ability to change that license, and that packages are now available indefinitely irrespective of whether or not the author wishes them to be.

I'd support that. I think it harms npm, but that's what you want.
If you don't understand how it harms npm, then nothing I can say will truly explain that to you.

I'd pose the opposite question to you: given no legal or security reason to do so, why would we choose a policy that results in a less-stable ecosystem than a different policy?

We disagree soundly on what constitutes stability in ecosystems.
We disagree soundly on what constitutes reliable systems.
We disagree that the reasons I have provided are or are not compelling.
We disagree that things don't work better when builds based on code that have been removed from the repository by the author should, indeed, fail.

I'm not sure you will ever see my point if you don't see it already, particularly after @getify's phenomenal job of clarifying my poor prose.

Perhaps the right and final approach is that packages submitted to NPM should be accepted only if the license permits unrestricted and permanent redistribution in any form and that the author relinquishes all ability to change that license, and that packages are now available indefinitely irrespective of whether or not the author wishes them to be.

I'd support that. I think it harms npm, but that's what you want.
If you don't understand how it harms npm, then nothing I can say will truly explain that to you.

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Apr 1, 2016

Contributor

hey everyone!

thank you very much for this discussion, i'll be answering as many of the questions asked here as possible and then i will be closing this thread at the end of the day. your participation was useful and i appreciate you all taking the time to share.

a couple of things that i would like to make as clear as possible:

  1. the restriction 24hr unpublish window has not been shipped yet, the blog post announced our intentions and authors who wish to unpublish right now may. we will let you know when the window has been implemented.
  2. while the unpublish policy is new, our legal right to distribute code published to the registry indefinitely has been guaranteed by our terms for quite some time. you agree to our terms when you publish to the registry whether you read them or not. again, this legal right has everything to do with our terms and nothing to do with the license you assign your code.
  3. this is the first step in our policy. the idea of abandoning a project or removing your association is interesting and being considered.

now to the questions:

What is the expected round trip time of a support human?

24-72hrs, often faster

What percentage of NPM packages is "stale" e.g. not updated frequently? Are you tracking any statistics on the number of package authors whose linked github repositories are unresponsive?

We are not actively tracking, but have the ability to see user and package activity should we decide we needed such data.

Under what circumstances does a currently published package fall under these new rules? Did you send a note to all npm package authors asking them to consent?

This is policy does not require any change to our legal policy. We published this policy announcement and we ran this open issue as a forum for questions. support@npmjs.com and community@npmjs.com are also available for concerned users.

In the case of a legal challenge by the author(s) or rightsholders of the code in question, do you intend to defend your ability to continue publishing the package?

Our terms, which you agree to when you publish a package to the registry, allow us the ability to distribute your code indefinitely. We are, and have been, legally within our ability to continuing distributing the package.

What are the thresholds used? Using humans implies judgment. If the package has one listed dependency, is that an issue? Ten? Twenty?

Those thresholds will emerge from data we collect as we move forward with the policy. We don't claim to know which would be best at this point and will look to the data we collect for direction.

Are all packages treated the same way? What about a package that has never been downloaded?

Nope. Packages vary and that is why a human solution is best at this point until we can figure out if there are clearly defined thresholds.

If a package has no dependents and has never been downloaded unpublishing it is a pretty trivial decision. However, given the bot activity on the registry, there are no packages that have never been downloaded.

If your support humans get overloaded, at what point do you anticipate migrating to a more scalable system?

We'll have to see.

How long does your sensible support human expect to take resolving a package that desires an unpublish that other packages depend on and are unresponsive? Modulecounts suggests 250k packages currently in NPM. If we assume 1% of those packages might have these issues, that's 2500 total, with an additional 4 (1% of 446 average) a day, for a total of 1,460 over the course of a year. Are these statistics accurate?

We'll have to see how it goes.

What do you think the proposed contact mechanism is or should be for a developer who wishes to unpublish their package? "Any means necessary"? If a developer wishes to unpublish a package, do you accept their warranty that they've made all attempts to communicate to the people that depend on their package?

"Any means necessary" is hyperbolic and definitely not endorsed by npm. Email seems sufficient.

"do you accept their warranty that they've made all attempts to communicate to the people that depend on their package?" We trust our users and believe that they act in good faith. If there is little to no activity for packages dependent on the target of an unpublish, we may make a judgement call, though our tendency will be to be conservative in protecting the ecosystem. This is a Hard thing to do and this is why we are using people and not hard and fast thresholds.

If a developer thinks there is someone, or multiple someones, who are infringing their license for a given package, how does NPM propose to deal with that dispute? NPM's terms of service seem to indicate there is no avenue for a developer to stop such a package from being available to the infringer short of a lengthy court battle.

If you believe a person or persons is using your package in a way that infringes on your license you should get in touch with that person and ask them to stop. You may need to get a lawyer. npm is not, and has never promised to be, your lawyer. Our distribution of your code has nothing to do with the license you give it. Violations of the license you give your code is something you need to deal with.

If it happens that a package in the registry is infringing on your Trademark or Intellectual Property you can get in touch with us. If it is the case that your Trademark or IP is being violated, we will remove the package. An example of this is our DMCA policy: https://www.npmjs.com/policies/dmca

Does NPM intend to treat larger companies different than smaller companies when it comes to this policy?

No.

Thank you again for your participation and questions. This issue will be closed at the end of the day today (Midnight, EST).

Contributor

ashleygwilliams commented Apr 1, 2016

hey everyone!

thank you very much for this discussion, i'll be answering as many of the questions asked here as possible and then i will be closing this thread at the end of the day. your participation was useful and i appreciate you all taking the time to share.

a couple of things that i would like to make as clear as possible:

  1. the restriction 24hr unpublish window has not been shipped yet, the blog post announced our intentions and authors who wish to unpublish right now may. we will let you know when the window has been implemented.
  2. while the unpublish policy is new, our legal right to distribute code published to the registry indefinitely has been guaranteed by our terms for quite some time. you agree to our terms when you publish to the registry whether you read them or not. again, this legal right has everything to do with our terms and nothing to do with the license you assign your code.
  3. this is the first step in our policy. the idea of abandoning a project or removing your association is interesting and being considered.

now to the questions:

What is the expected round trip time of a support human?

24-72hrs, often faster

What percentage of NPM packages is "stale" e.g. not updated frequently? Are you tracking any statistics on the number of package authors whose linked github repositories are unresponsive?

We are not actively tracking, but have the ability to see user and package activity should we decide we needed such data.

Under what circumstances does a currently published package fall under these new rules? Did you send a note to all npm package authors asking them to consent?

This is policy does not require any change to our legal policy. We published this policy announcement and we ran this open issue as a forum for questions. support@npmjs.com and community@npmjs.com are also available for concerned users.

In the case of a legal challenge by the author(s) or rightsholders of the code in question, do you intend to defend your ability to continue publishing the package?

Our terms, which you agree to when you publish a package to the registry, allow us the ability to distribute your code indefinitely. We are, and have been, legally within our ability to continuing distributing the package.

What are the thresholds used? Using humans implies judgment. If the package has one listed dependency, is that an issue? Ten? Twenty?

Those thresholds will emerge from data we collect as we move forward with the policy. We don't claim to know which would be best at this point and will look to the data we collect for direction.

Are all packages treated the same way? What about a package that has never been downloaded?

Nope. Packages vary and that is why a human solution is best at this point until we can figure out if there are clearly defined thresholds.

If a package has no dependents and has never been downloaded unpublishing it is a pretty trivial decision. However, given the bot activity on the registry, there are no packages that have never been downloaded.

If your support humans get overloaded, at what point do you anticipate migrating to a more scalable system?

We'll have to see.

How long does your sensible support human expect to take resolving a package that desires an unpublish that other packages depend on and are unresponsive? Modulecounts suggests 250k packages currently in NPM. If we assume 1% of those packages might have these issues, that's 2500 total, with an additional 4 (1% of 446 average) a day, for a total of 1,460 over the course of a year. Are these statistics accurate?

We'll have to see how it goes.

What do you think the proposed contact mechanism is or should be for a developer who wishes to unpublish their package? "Any means necessary"? If a developer wishes to unpublish a package, do you accept their warranty that they've made all attempts to communicate to the people that depend on their package?

"Any means necessary" is hyperbolic and definitely not endorsed by npm. Email seems sufficient.

"do you accept their warranty that they've made all attempts to communicate to the people that depend on their package?" We trust our users and believe that they act in good faith. If there is little to no activity for packages dependent on the target of an unpublish, we may make a judgement call, though our tendency will be to be conservative in protecting the ecosystem. This is a Hard thing to do and this is why we are using people and not hard and fast thresholds.

If a developer thinks there is someone, or multiple someones, who are infringing their license for a given package, how does NPM propose to deal with that dispute? NPM's terms of service seem to indicate there is no avenue for a developer to stop such a package from being available to the infringer short of a lengthy court battle.

If you believe a person or persons is using your package in a way that infringes on your license you should get in touch with that person and ask them to stop. You may need to get a lawyer. npm is not, and has never promised to be, your lawyer. Our distribution of your code has nothing to do with the license you give it. Violations of the license you give your code is something you need to deal with.

If it happens that a package in the registry is infringing on your Trademark or Intellectual Property you can get in touch with us. If it is the case that your Trademark or IP is being violated, we will remove the package. An example of this is our DMCA policy: https://www.npmjs.com/policies/dmca

Does NPM intend to treat larger companies different than smaller companies when it comes to this policy?

No.

Thank you again for your participation and questions. This issue will be closed at the end of the day today (Midnight, EST).

@spdustin

This comment has been minimized.

Show comment
Hide comment
@spdustin

spdustin Apr 1, 2016

In response to @arimathea's comment

Do you have some good reasons why other than builds failing is inconvenient?

Continuous integration → continuous delivery → continuous deployment builds that are failing is more than an inconvenience. There are many, many moving parts to the things we all -- as users, not as developers -- use every day. Major and minor web applications that, as individuals, we have come to rely on to Get Shit Done™. Major organizations have, for better or worse, come to rely on package managers as their source of truth, ensuring that their builds are reproducible by others on their development teams.

When a failed build happens during this process — something that happens many, many times a day on sites you may be familiar with: Facebook, Amazon, Bing, Etsy, Netflix — you can bet that it may be more than an inconvenience to the users, and exponentially more than an inconvenience to the developers.

They may be rushing to push out a security fix, or an update that fixes a bug that's preventing a user from checking out on an e-commerce site, or fixing an issue preventing a particular browser from playing a video. But when a package is suddenly plucked from the registry... that build that is fixing an issue costing users and/or businesses actual money is now held up because some package has a third-generation dependency on another package that's been unpublished. Sure, they'll find a way around it. Maybe they'll use shrinkwrap on their integration builds so it won't happen again, but then again, they wouldn't get urgent updates to those packages. Maybe they'll manually pull the package source from an earlier build, but it takes time.

You're angry and frustrated because you feel like you'd lose control over something you've made, something you've crafted, something you've poured your heart into, and I understand that. I don't see any of those on this particular Github profile, but I can definitely empathize with the feeling that you don't have control over something you've made. Please understand that your feelings aren't being ignored or minimized here, really and truly — I think your frustration that you're not changing minds is because you're feeling like you're not being acknowledged, but you are. I've read the responses, and I've no particular stake in npm as a contributor (in fact, I'm primarily a consumer of the registry, though I do contribute to other projects, including some in the npm registry). I can see the replies that express an understanding of your point of view, but I feel strongly that you haven't done the same. Sometimes, you can be right, or you can be happy. Sometimes being right means ignoring others because you're pot-committed (to borrow a poker term), and that's a sad situation that I hope you can avoid.

Take a moment, just a moment, and ponder the other side of the debate: npm devotes considerable resources to a huge ecosystem of developers, many of whom build the things you use every day, and those developers have come to rely on npm's implicit promise that they'll deliver the packages you request, when you request them, every time. They risk completely up-ending the workdays of many developers, and potentially putting even more end-users at risk when an urgent update can't be pushed out to production quickly because of a missing package.

I hope you find your way to be more circumspect about this. There's a whole lot more to the Internet than it seems to most people, even those of us that contribute to projects used by many, many people (and even if those changes are trivial, like mine 😛).

spdustin commented Apr 1, 2016

In response to @arimathea's comment

Do you have some good reasons why other than builds failing is inconvenient?

Continuous integration → continuous delivery → continuous deployment builds that are failing is more than an inconvenience. There are many, many moving parts to the things we all -- as users, not as developers -- use every day. Major and minor web applications that, as individuals, we have come to rely on to Get Shit Done™. Major organizations have, for better or worse, come to rely on package managers as their source of truth, ensuring that their builds are reproducible by others on their development teams.

When a failed build happens during this process — something that happens many, many times a day on sites you may be familiar with: Facebook, Amazon, Bing, Etsy, Netflix — you can bet that it may be more than an inconvenience to the users, and exponentially more than an inconvenience to the developers.

They may be rushing to push out a security fix, or an update that fixes a bug that's preventing a user from checking out on an e-commerce site, or fixing an issue preventing a particular browser from playing a video. But when a package is suddenly plucked from the registry... that build that is fixing an issue costing users and/or businesses actual money is now held up because some package has a third-generation dependency on another package that's been unpublished. Sure, they'll find a way around it. Maybe they'll use shrinkwrap on their integration builds so it won't happen again, but then again, they wouldn't get urgent updates to those packages. Maybe they'll manually pull the package source from an earlier build, but it takes time.

You're angry and frustrated because you feel like you'd lose control over something you've made, something you've crafted, something you've poured your heart into, and I understand that. I don't see any of those on this particular Github profile, but I can definitely empathize with the feeling that you don't have control over something you've made. Please understand that your feelings aren't being ignored or minimized here, really and truly — I think your frustration that you're not changing minds is because you're feeling like you're not being acknowledged, but you are. I've read the responses, and I've no particular stake in npm as a contributor (in fact, I'm primarily a consumer of the registry, though I do contribute to other projects, including some in the npm registry). I can see the replies that express an understanding of your point of view, but I feel strongly that you haven't done the same. Sometimes, you can be right, or you can be happy. Sometimes being right means ignoring others because you're pot-committed (to borrow a poker term), and that's a sad situation that I hope you can avoid.

Take a moment, just a moment, and ponder the other side of the debate: npm devotes considerable resources to a huge ecosystem of developers, many of whom build the things you use every day, and those developers have come to rely on npm's implicit promise that they'll deliver the packages you request, when you request them, every time. They risk completely up-ending the workdays of many developers, and potentially putting even more end-users at risk when an urgent update can't be pushed out to production quickly because of a missing package.

I hope you find your way to be more circumspect about this. There's a whole lot more to the Internet than it seems to most people, even those of us that contribute to projects used by many, many people (and even if those changes are trivial, like mine 😛).

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Apr 1, 2016

@arimathea certainly I accept that our viewpoints are very different on how much package authors should be allowed to retain control of how their code is distributed, but I am astonished that we don't even share the same premise in this debate:

"We disagree soundly on what constitutes stability in ecosystems. We disagree soundly on what constitutes reliable systems."

The entire purpose of npm is to distribute code. That is it's only function. The only metric by which you can measure it's effectiveness at that is, how often does it succeed at providing code to consumers when they request it?

If you think that a policy which makes npm deliver code less often does not mean it is less reliable, I really don't know how else you could evaluate it. This is the entire premise of npm making this change in the first place.

I have appreciated reading others' perspectives on this. But I come from a very pragmatic one. I want things to work. I'm glad that the policies they are moving towards seem to similarly place "working" as the most important concern.

I do hope that the policies can evolve to include "delisting" capabilities as have been suggested by others. These have no impact on the delivery of previously-published code, and would seem to be useful for people who want to stop new people from finding their code. I see no downside to this.

jamietre commented Apr 1, 2016

@arimathea certainly I accept that our viewpoints are very different on how much package authors should be allowed to retain control of how their code is distributed, but I am astonished that we don't even share the same premise in this debate:

"We disagree soundly on what constitutes stability in ecosystems. We disagree soundly on what constitutes reliable systems."

The entire purpose of npm is to distribute code. That is it's only function. The only metric by which you can measure it's effectiveness at that is, how often does it succeed at providing code to consumers when they request it?

If you think that a policy which makes npm deliver code less often does not mean it is less reliable, I really don't know how else you could evaluate it. This is the entire premise of npm making this change in the first place.

I have appreciated reading others' perspectives on this. But I come from a very pragmatic one. I want things to work. I'm glad that the policies they are moving towards seem to similarly place "working" as the most important concern.

I do hope that the policies can evolve to include "delisting" capabilities as have been suggested by others. These have no impact on the delivery of previously-published code, and would seem to be useful for people who want to stop new people from finding their code. I see no downside to this.

@spdustin

This comment has been minimized.

Show comment
Hide comment
@spdustin

spdustin Apr 1, 2016

@jamietre, to put it simply, I think npm's ultimate terms should be "deprecation before deletion".

This would allow devs to de-list because they want to cut that responsibility from their lives, without frakking it up for the rest of us using their package (or a package that depends on their package). Let me know that it's been deprecated, and I can decide how to proceed as a user of the registry.

spdustin commented Apr 1, 2016

@jamietre, to put it simply, I think npm's ultimate terms should be "deprecation before deletion".

This would allow devs to de-list because they want to cut that responsibility from their lives, without frakking it up for the rest of us using their package (or a package that depends on their package). Let me know that it's been deprecated, and I can decide how to proceed as a user of the registry.

@jamietre

This comment has been minimized.

Show comment
Hide comment
@jamietre

jamietre Apr 1, 2016

@spdustin completely agree, and this is how Nuget works. You can already deprecate with npm, you just can't delist.

jamietre commented Apr 1, 2016

@spdustin completely agree, and this is how Nuget works. You can already deprecate with npm, you just can't delist.

@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Apr 1, 2016

The entire purpose of npm is to distribute code. That is it's only function. The only metric by which you can measure it's effectiveness at that is, how often does it succeed at providing code to consumers when they request it?

You are confusing the means of distribution (pushing bits either source or binary) with the role of a package manager (metadata and plumbing). I see why you would lump them together, but the two are functionally separate for me.

I could say more (a lot more!) about this, but I think it doesn't matter at this point.

When a failed build happens during this process — something that happens many, many times a day on sites you may be familiar with: Facebook, Amazon, Bing, Etsy, Netflix — you can bet that it may be more than an inconvenience to the users, and exponentially more than an inconvenience to the developers.

I know this all too well, because while I don't work for one of these five companies, I certainly work for a company in this class of companies, with some packages in npm both individually and at a corporate level, some of which are depended on (two quite heavily - 100s of dependencies). However, and perhaps this is the difference, we (my peers and company) have a much more pragmatic view of dependencies than it seems like some of the people that have been vocal on this thread have. We know what we are building on (we look at dependencies all the way up the tree) and the impact that it might have on us, both individually and collectively, should that code just "vanish". We also separate, as perhaps some of you do not, a development pipeline and a production pipeline. This is what I interpret as responsible development and behavior, and perhaps i'm an outlier, but I don't think so. I really don't (didn't) look at who depends on me too much - there wasn't a need - but now I must actively police a new dependent since they might prevent me (us) from removing my (our) code.

Does that sound like something that it is useful to spend your time doing?

You're angry and frustrated because you feel like you'd lose control over something you've made,

Actually, @spdustin, I am neither angry nor frustrated. I am 😥. In a way, I am enlightened, because this has made me understand that I need to only choose package management solutions that will respect my choices as an author of code, something that I erroneously assumed a business built on the backs of developers and users alike would do (the former much more than the latter, but they are both often one and the same). While npm, the entity, isn't "directly" taking that choice away from me and my company, they are certainly allowing other people to have a disproportionate say as to the final outcome of my contributions. If someone depends on my code, I lose control of it if it's in npm. There's no resolution process other than "we'll do our best", and I know they will do their best, but I know now, unequivocally, that bias will not ever be in my favor. No problem at all - just means I will choose to no longer contribute to npm and to deprecate my packages, unpublishing the ones that are not depended on. Or maybe I will adopt the approach some have suggested, which is unpublishing every 23 hours and then republishing.

I do know I now have to send out a couple of hundred emails to individuals, hope those emails still work, hope they still work at the companies that have packages that rely on ours/mine or that they can still exert some influence. That is a truly asinine amount of work to no purpose - after all, it is not my fault, and shouldn't be!, that their builds suddenly fail. But it costs me real money just because they arbitrarily decided that "hey, this code looks cool, and we're going to be within the terms of the license by adopting it, so let's go after it".

I have deeply considered the other side and I realize that other people have an opposing view - i'd even go so far as to say i've probably given both sides more thought than some of the responders here. I think I, and others, have presented numerous solutions in this thread that serve to keep both worlds happy and still respect an author's choices. npm has not chosen to adopt them at this time, even though I have not heard or seen any real opposition to any of the proposals for a middle ground. I think the "why" is pretty clear. Thankfully, there are alternatives to npm package management, and now i'm going to go adopt some and ignore the mumbled "good riddance" from others. 👍

The entire purpose of npm is to distribute code. That is it's only function. The only metric by which you can measure it's effectiveness at that is, how often does it succeed at providing code to consumers when they request it?

You are confusing the means of distribution (pushing bits either source or binary) with the role of a package manager (metadata and plumbing). I see why you would lump them together, but the two are functionally separate for me.

I could say more (a lot more!) about this, but I think it doesn't matter at this point.

When a failed build happens during this process — something that happens many, many times a day on sites you may be familiar with: Facebook, Amazon, Bing, Etsy, Netflix — you can bet that it may be more than an inconvenience to the users, and exponentially more than an inconvenience to the developers.

I know this all too well, because while I don't work for one of these five companies, I certainly work for a company in this class of companies, with some packages in npm both individually and at a corporate level, some of which are depended on (two quite heavily - 100s of dependencies). However, and perhaps this is the difference, we (my peers and company) have a much more pragmatic view of dependencies than it seems like some of the people that have been vocal on this thread have. We know what we are building on (we look at dependencies all the way up the tree) and the impact that it might have on us, both individually and collectively, should that code just "vanish". We also separate, as perhaps some of you do not, a development pipeline and a production pipeline. This is what I interpret as responsible development and behavior, and perhaps i'm an outlier, but I don't think so. I really don't (didn't) look at who depends on me too much - there wasn't a need - but now I must actively police a new dependent since they might prevent me (us) from removing my (our) code.

Does that sound like something that it is useful to spend your time doing?

You're angry and frustrated because you feel like you'd lose control over something you've made,

Actually, @spdustin, I am neither angry nor frustrated. I am 😥. In a way, I am enlightened, because this has made me understand that I need to only choose package management solutions that will respect my choices as an author of code, something that I erroneously assumed a business built on the backs of developers and users alike would do (the former much more than the latter, but they are both often one and the same). While npm, the entity, isn't "directly" taking that choice away from me and my company, they are certainly allowing other people to have a disproportionate say as to the final outcome of my contributions. If someone depends on my code, I lose control of it if it's in npm. There's no resolution process other than "we'll do our best", and I know they will do their best, but I know now, unequivocally, that bias will not ever be in my favor. No problem at all - just means I will choose to no longer contribute to npm and to deprecate my packages, unpublishing the ones that are not depended on. Or maybe I will adopt the approach some have suggested, which is unpublishing every 23 hours and then republishing.

I do know I now have to send out a couple of hundred emails to individuals, hope those emails still work, hope they still work at the companies that have packages that rely on ours/mine or that they can still exert some influence. That is a truly asinine amount of work to no purpose - after all, it is not my fault, and shouldn't be!, that their builds suddenly fail. But it costs me real money just because they arbitrarily decided that "hey, this code looks cool, and we're going to be within the terms of the license by adopting it, so let's go after it".

I have deeply considered the other side and I realize that other people have an opposing view - i'd even go so far as to say i've probably given both sides more thought than some of the responders here. I think I, and others, have presented numerous solutions in this thread that serve to keep both worlds happy and still respect an author's choices. npm has not chosen to adopt them at this time, even though I have not heard or seen any real opposition to any of the proposals for a middle ground. I think the "why" is pretty clear. Thankfully, there are alternatives to npm package management, and now i'm going to go adopt some and ignore the mumbled "good riddance" from others. 👍

@garfieldnate

This comment has been minimized.

Show comment
Hide comment
@garfieldnate

garfieldnate Apr 1, 2016

I would have liked a response/explanation about what happened with kik and how you plan to protect users in the future. That's what started this whole fiasco in the first place.

I would have liked a response/explanation about what happened with kik and how you plan to protect users in the future. That's what started this whole fiasco in the first place.

@ljharb

This comment has been minimized.

Show comment
Hide comment
@ljharb

ljharb Apr 1, 2016

@garfieldnate there was already one. http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy that's what this entire thread is about. (edit: perhaps you meant the original package, in which case the response is http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm , and there's nothing users need protection against imo)

ljharb commented Apr 1, 2016

@garfieldnate there was already one. http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy that's what this entire thread is about. (edit: perhaps you meant the original package, in which case the response is http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm , and there's nothing users need protection against imo)

@spdustin

This comment has been minimized.

Show comment
Hide comment
@spdustin

spdustin Apr 1, 2016

@arimathea, thanks for your reply. I, personally, partially agree with some of the mitigating solutions you presented in this comment

And partially (I'll explain why):

Make it so that builds using 'npm install' fail intelligently in the face of dependencies, and consumers of npm are notified of these constraints when first installing an npm package (of course including a flag to say "yes, I understand")

Well, I don't know what you mean by _fail intelligently in the face of dependencies_. Do you mean a warn and a re-use from cache of existing modules? Is this feasible? Is it a workable alternative to simply failing the build?

And the whole _flag to say "yes, I understand"_ is dead on arrival, and I think you and everyone else reading this knows it. Either builds are automated (and thus a wildcard "accept whatever" flag is set in the build process), or the user manually running npm install is going to get "banner blindness", and they'll quickly tune out all those warnings.

Allow an author to flag a "NODEPEND" entry in the npm entry for his package, indicating that the author does NOT want their package depended on by other packages. If existing packages come to depend on this, warn loudly. Unpublishes should be executed immediately for these packages.

A NODEPEND flag, that's smart. Why not just a @nodepend scope, with a change to how npm handles that scope, rather than a new flag? I'm not smart enough to come up with reasons why it would be used, but I'm smart enough to recognize that there exists reasons why it would be used.

It falls apart, however, when you take away the binary nature of a "flag", and allow other packages to depend on a NODEPEND package. If it's NODEPEND, it's NODEPEND and we mean it! If this solution is implemented, don't let packages depend on NODEPEND packages, period.

Then, we get to the "unpublish" part of the suggestion, and we find ourselves back in the quandary of "you're asking us to break shit all over the place because you changed your mind about sharing your code".

For better or worse (and I think it's arguable either way, and thus, not really helpful to argue that it's a poor choice on the developers' part), there are development/delivery/deployment procedures adopted by companies large and small that will fail if npm install just says "nope", and that's a Very Bad Thing™ — even if your procedures insulate you from it. npm shrinkwrap plus checking node_modules into your repo is common prescriptive guidance, for example, so how should npm handle shrinkwrapped packages? Should it remove them if unpublished, thus invalidating the entire shrinkwrap?

Allow authors to abandon a package (removing their name/identifier from the package)

I can get behind this, too. I think it's a smart idea, actually, and removes the cognitive burden from a developer who takes his or her code and (╯°□°)╯︵ ┻━┻. I propose the ALANSMITHEE flag, after (of course) Alan Smithee. Or, we can avoid a flag by changing the author field to Alan Smithee <alansmithee@abandonedcode.example.com> (http://abandonedcode.npmjs.com/) with an appropriate page describing the abandonware state of the package at that URL. Also, homepage set to http://abandonedcode.npmjs.com/

Finally, I don't think there's a moral or ethical problem with the presumption that your code, once published via npm, is out of your hands. It's true with absolutely everything else you send out to the Internet, like someone's Ashley Madison login and preferences, or like your comments here. Sure, you could delete it all (the whole "right to be forgotten"), but you know they're archived somewhere, by archive.org, or in Googlebot's cache. The Internet by its nature is the Wild West. I think a strict NODEPEND (or use a reserved @nodepend scope) plus ALANSMITHEE (or my example author and homepage combo) would be a good middle ground.

spdustin commented Apr 1, 2016

@arimathea, thanks for your reply. I, personally, partially agree with some of the mitigating solutions you presented in this comment

And partially (I'll explain why):

Make it so that builds using 'npm install' fail intelligently in the face of dependencies, and consumers of npm are notified of these constraints when first installing an npm package (of course including a flag to say "yes, I understand")

Well, I don't know what you mean by _fail intelligently in the face of dependencies_. Do you mean a warn and a re-use from cache of existing modules? Is this feasible? Is it a workable alternative to simply failing the build?

And the whole _flag to say "yes, I understand"_ is dead on arrival, and I think you and everyone else reading this knows it. Either builds are automated (and thus a wildcard "accept whatever" flag is set in the build process), or the user manually running npm install is going to get "banner blindness", and they'll quickly tune out all those warnings.

Allow an author to flag a "NODEPEND" entry in the npm entry for his package, indicating that the author does NOT want their package depended on by other packages. If existing packages come to depend on this, warn loudly. Unpublishes should be executed immediately for these packages.

A NODEPEND flag, that's smart. Why not just a @nodepend scope, with a change to how npm handles that scope, rather than a new flag? I'm not smart enough to come up with reasons why it would be used, but I'm smart enough to recognize that there exists reasons why it would be used.

It falls apart, however, when you take away the binary nature of a "flag", and allow other packages to depend on a NODEPEND package. If it's NODEPEND, it's NODEPEND and we mean it! If this solution is implemented, don't let packages depend on NODEPEND packages, period.

Then, we get to the "unpublish" part of the suggestion, and we find ourselves back in the quandary of "you're asking us to break shit all over the place because you changed your mind about sharing your code".

For better or worse (and I think it's arguable either way, and thus, not really helpful to argue that it's a poor choice on the developers' part), there are development/delivery/deployment procedures adopted by companies large and small that will fail if npm install just says "nope", and that's a Very Bad Thing™ — even if your procedures insulate you from it. npm shrinkwrap plus checking node_modules into your repo is common prescriptive guidance, for example, so how should npm handle shrinkwrapped packages? Should it remove them if unpublished, thus invalidating the entire shrinkwrap?

Allow authors to abandon a package (removing their name/identifier from the package)

I can get behind this, too. I think it's a smart idea, actually, and removes the cognitive burden from a developer who takes his or her code and (╯°□°)╯︵ ┻━┻. I propose the ALANSMITHEE flag, after (of course) Alan Smithee. Or, we can avoid a flag by changing the author field to Alan Smithee <alansmithee@abandonedcode.example.com> (http://abandonedcode.npmjs.com/) with an appropriate page describing the abandonware state of the package at that URL. Also, homepage set to http://abandonedcode.npmjs.com/

Finally, I don't think there's a moral or ethical problem with the presumption that your code, once published via npm, is out of your hands. It's true with absolutely everything else you send out to the Internet, like someone's Ashley Madison login and preferences, or like your comments here. Sure, you could delete it all (the whole "right to be forgotten"), but you know they're archived somewhere, by archive.org, or in Googlebot's cache. The Internet by its nature is the Wild West. I think a strict NODEPEND (or use a reserved @nodepend scope) plus ALANSMITHEE (or my example author and homepage combo) would be a good middle ground.

@benburkhart1

This comment has been minimized.

Show comment
Hide comment
@benburkhart1

benburkhart1 Apr 1, 2016

I think there's a lot of commonalities between npm and Sourceforge on this issue. Sourceforge, like npm, was a repository for open source projects. Sourceforge provided a place for developers to commit and distribute their software to other people. They dictated which licenses they were willing to work with, so they were able to guarantee their legal right to distribute this software. Over time, Sourceforge made numerous questionable policy changes, and developers started migrating away from their platform. Sourceforge, like npm allowed developers to delete their projects, but then after realizing their revenue loss from popular projects leaving, they decided to implement a "mirroring" process. They did this for nmap, GIMP and VLC after they had officially left Sourceforge. Despite the fact Sourceforge had legal ability to redistribute this software (as per GPLed terms or other less restrictive licenses), it was met with mass community problems and eventually they suspended this practice due to the amount of negative feedback they received about it. I only bring up this because this sounds exactly like what npm is proposing doing. It's not always about the legality of doing it, if all your contributions come from developers, honoring what the developers/copyright holders want to do with their code should be your priority.

http://www.infoworld.com/article/2929732/open-source-software/sourceforge-commits-reputational-suicide.html

http://arstechnica.com/information-technology/2015/06/black-mirror-sourceforge-has-now-siezed-nmap-audit-tool-project/

http://www.infoworld.com/article/2927973/linux/is-sourceforge-hijacking-project-accounts.html

I think there's a lot of commonalities between npm and Sourceforge on this issue. Sourceforge, like npm, was a repository for open source projects. Sourceforge provided a place for developers to commit and distribute their software to other people. They dictated which licenses they were willing to work with, so they were able to guarantee their legal right to distribute this software. Over time, Sourceforge made numerous questionable policy changes, and developers started migrating away from their platform. Sourceforge, like npm allowed developers to delete their projects, but then after realizing their revenue loss from popular projects leaving, they decided to implement a "mirroring" process. They did this for nmap, GIMP and VLC after they had officially left Sourceforge. Despite the fact Sourceforge had legal ability to redistribute this software (as per GPLed terms or other less restrictive licenses), it was met with mass community problems and eventually they suspended this practice due to the amount of negative feedback they received about it. I only bring up this because this sounds exactly like what npm is proposing doing. It's not always about the legality of doing it, if all your contributions come from developers, honoring what the developers/copyright holders want to do with their code should be your priority.

http://www.infoworld.com/article/2929732/open-source-software/sourceforge-commits-reputational-suicide.html

http://arstechnica.com/information-technology/2015/06/black-mirror-sourceforge-has-now-siezed-nmap-audit-tool-project/

http://www.infoworld.com/article/2927973/linux/is-sourceforge-hijacking-project-accounts.html

@arimathea

This comment has been minimized.

Show comment
Hide comment
@arimathea

arimathea Apr 2, 2016

@spdustin:

Some of my mitigating solutions were not intended to mean we should do them all at once, just for clarity.

Well, I don't know what you mean by fail intelligently in the face of dependencies. Do you mean a warn and a re-use from cache of existing modules? Is this feasible? Is it a workable alternative to simply failing the build?

100%, with the continued warning that hey, this package is totally gone, and you're in bat country! (I would prefer it to be an error to insure the user has to pay attention to it, but that's splitting hairs).

A NODEPEND flag, that's smart. Why not just a @nodepend scope, with a change to how npm handles that scope, rather than a new flag? I'm not smart enough to come up with reasons why it would be used, but I'm smart enough to recognize that there exists reasons why it would be used.

It falls apart, however, when you take away the binary nature of a "flag", and allow other packages to depend on a NODEPEND package. If it's NODEPEND, it's NODEPEND and we mean it! If this solution is implemented, don't let packages depend on NODEPEND packages, period.

Then, we get to the "unpublish" part of the suggestion, and we find ourselves back in the quandary of "you're asking us to break shit all over the place because you changed your mind about sharing your code".

No. What i'm communicating to you is that I don't ever want you to depend on this code in your libraries because I want to reserve the right of removing it at any time. If you continue to depend on it, then npm should not step in to save your ass.

I think this flag should be set only when a package is newly entered into the repository, but there should be some grandfathering mechanism for older packages.

I'm not asking you to break shit all over the place. If I put up warning tape and signs around a giant hole in the earth, and you still fall into it despite my best efforts to prevent you from doing so, that is on you. I am shifting the responsibility directly to you, and not relying on the package repository to fix your dumb decision to depend on my code even when I told you, explicitly, ten ways, not to. Note that this is a totally different thing than me "abandoning" my code. Would that have fixed left-pad? No, of course not. Does it allow me to further express my desires as an author of code that you only depend on it if you REALLY KNOW what you're doing and you know that I can remove it at any time? Yes it does! Does it support npm's desire to "not break things that shouldn't be broken"? Yes it does. Is azer's code still available? Yes it is.

how should npm handle shrinkwrapped packages? Should it remove them if unpublished, thus invalidating the entire shrinkwrap?

I can't answer that question. The only point I am truly making here is that as the author of code I want to be able to remove it from a repository, and I want to be the person that controls that. I don't want you, as an arbitrary faceless person or corporation, to suddenly start depending on my code for your little one person side project, fall off the face of the earth so that I can never reach you, and have npm decide, "well arimathea, that's just too bad! Because this dependency still exists, we can't respect your desire to unpublish. Sorry!" And this gets monumentally more complex the more people depend on my package - something I no longer have any control over once I put it in npm.

Finally, I don't think there's a moral or ethical problem with the presumption that your code, once published via npm, is out of your hands. It's true with absolutely everything else you send out to the Internet, like someone's Ashley Madison login and preferences, or like your comments here.

I agree with you that once my code is published out on the Internet (anywhere) someone can just take it and download it and do whatever the license says they can do with it. I do not agree with you that npm, the commercial entity whose terms of service I agree with, should have the ultimate say on the disposition of my code. If I delete my code from github, yes, there are some artifacts there. There are plenty of other mechanisms, though, for me to exert control over the caching behavior of other parties (robots.txt as one example), and just as many other mechanisms for people to ignore them. There are plenty of legal mechanisms for me to stop a party from using my code if I have discovered that they are using it (and just as many mechanisms for them to fight me on it). But there is no way for me to say, unequivocally, that I want npm to stop pushing my bits with this new policy. I am really not trying to solve any other problems. I am trying to solve one and a half, and only one and a half, which is I don't want npm, the company, to have the ability to continue to publish my software if I don't want them to, and certainly to not link the actions of a third party to whether they continue to distribute the software I put up at one point when I want them not to. If you were using best practices and grabbed my code and just said "the hell with it, we're going to include arimathea's modules directly because npm is trash", then that's what it is. That isn't the problem I'm trying to solve.

It really isn't any more difficult than that.

@spdustin:

Some of my mitigating solutions were not intended to mean we should do them all at once, just for clarity.

Well, I don't know what you mean by fail intelligently in the face of dependencies. Do you mean a warn and a re-use from cache of existing modules? Is this feasible? Is it a workable alternative to simply failing the build?

100%, with the continued warning that hey, this package is totally gone, and you're in bat country! (I would prefer it to be an error to insure the user has to pay attention to it, but that's splitting hairs).

A NODEPEND flag, that's smart. Why not just a @nodepend scope, with a change to how npm handles that scope, rather than a new flag? I'm not smart enough to come up with reasons why it would be used, but I'm smart enough to recognize that there exists reasons why it would be used.

It falls apart, however, when you take away the binary nature of a "flag", and allow other packages to depend on a NODEPEND package. If it's NODEPEND, it's NODEPEND and we mean it! If this solution is implemented, don't let packages depend on NODEPEND packages, period.

Then, we get to the "unpublish" part of the suggestion, and we find ourselves back in the quandary of "you're asking us to break shit all over the place because you changed your mind about sharing your code".

No. What i'm communicating to you is that I don't ever want you to depend on this code in your libraries because I want to reserve the right of removing it at any time. If you continue to depend on it, then npm should not step in to save your ass.

I think this flag should be set only when a package is newly entered into the repository, but there should be some grandfathering mechanism for older packages.

I'm not asking you to break shit all over the place. If I put up warning tape and signs around a giant hole in the earth, and you still fall into it despite my best efforts to prevent you from doing so, that is on you. I am shifting the responsibility directly to you, and not relying on the package repository to fix your dumb decision to depend on my code even when I told you, explicitly, ten ways, not to. Note that this is a totally different thing than me "abandoning" my code. Would that have fixed left-pad? No, of course not. Does it allow me to further express my desires as an author of code that you only depend on it if you REALLY KNOW what you're doing and you know that I can remove it at any time? Yes it does! Does it support npm's desire to "not break things that shouldn't be broken"? Yes it does. Is azer's code still available? Yes it is.

how should npm handle shrinkwrapped packages? Should it remove them if unpublished, thus invalidating the entire shrinkwrap?

I can't answer that question. The only point I am truly making here is that as the author of code I want to be able to remove it from a repository, and I want to be the person that controls that. I don't want you, as an arbitrary faceless person or corporation, to suddenly start depending on my code for your little one person side project, fall off the face of the earth so that I can never reach you, and have npm decide, "well arimathea, that's just too bad! Because this dependency still exists, we can't respect your desire to unpublish. Sorry!" And this gets monumentally more complex the more people depend on my package - something I no longer have any control over once I put it in npm.

Finally, I don't think there's a moral or ethical problem with the presumption that your code, once published via npm, is out of your hands. It's true with absolutely everything else you send out to the Internet, like someone's Ashley Madison login and preferences, or like your comments here.

I agree with you that once my code is published out on the Internet (anywhere) someone can just take it and download it and do whatever the license says they can do with it. I do not agree with you that npm, the commercial entity whose terms of service I agree with, should have the ultimate say on the disposition of my code. If I delete my code from github, yes, there are some artifacts there. There are plenty of other mechanisms, though, for me to exert control over the caching behavior of other parties (robots.txt as one example), and just as many other mechanisms for people to ignore them. There are plenty of legal mechanisms for me to stop a party from using my code if I have discovered that they are using it (and just as many mechanisms for them to fight me on it). But there is no way for me to say, unequivocally, that I want npm to stop pushing my bits with this new policy. I am really not trying to solve any other problems. I am trying to solve one and a half, and only one and a half, which is I don't want npm, the company, to have the ability to continue to publish my software if I don't want them to, and certainly to not link the actions of a third party to whether they continue to distribute the software I put up at one point when I want them not to. If you were using best practices and grabbed my code and just said "the hell with it, we're going to include arimathea's modules directly because npm is trash", then that's what it is. That isn't the problem I'm trying to solve.

It really isn't any more difficult than that.

@npm npm locked and limited conversation to collaborators Apr 2, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.