No more npm publish -f
#148
Comments
|
👍 right on right on. it should have been this way from the start. |
|
+10000000 |
|
\o/ thank you! |
|
Oh god, about time. |
|
I've used |
|
How does this work with completely removing a package from npm? Is the package name burned forever? |
|
@benjamn My first go-to in this situation is to use the @jkrems Fully-removed packages are whittled down to a state where anyone can publish, but the old version numbers are still kept around as permanent tombstones that can never be reused. |
|
Note that server admins can still UPDATE 2015: PLEASE SEE COMMENT AT THE BOTTOM OF THIS THREAD. |
|
👍 Sounds great! |
|
I think @jdalton uses this feature to republish previous versions of Lo-Dash when needed. He might want to add something to this discussion. |
|
I was literally just about to do this to republish an old version of a module with a different dist-tag. Turned out I wanted to fix the readme, so I bumped the version anyway, but it still seems like a valid reason to |
|
@danmactough You can use the |
Ah. I thought that was the case and tried it, but I forgot to add
Tough call. I can see an argument that what's done is done -- even if it's only the readme. Wouldn't bother me, though. But, I never, ever look at the readme that's sitting in the |
|
Yeah, it's mostly about what goes into the doc itself, since that shows up on the website, can be searched, etc. |
Good points. Still kind of hinky for that shasum to change, though. |
|
Right, so, I add a new command, |
|
👍 Some secondary thoughts around
Assuming this is designed to prevent the naive work-around where a user would unpublish the entire package and then republish all versions (including the one single version they wanted to force push). e.g.: This is a good thing as preventing deletes all together is the only way to prevent this behavior (or at least to make it auditable which is really what most package consumers want). There are a couple of side-effects though:
Although (3) feels like a nit, given that developers (like @jesusabdullah) see |
Yap, I've used it before. It was handy to back port patches when things like a Node compat issue cropped up, or an external resource changed under our feet, or I had a non-functional doc/package typo. It was a win for me at least :) I've shot myself in the foot before, but recovered. It was useful.
Yikes, so devs using a package name, unaware it was previously owned and deleted, may run into issues with their versioning? |
|
expanding on @benjamn's point a bit, it seems that |
👍 |
Per @indexzero's comment above, someone new coming along would be able to publish any version they want. But that's probably by mistake… |
|
The more I think about it there doesn't seem to be any good way to preventing force publish without preventing unpublish of entire packages. Any checking we do around users could easily be circumvented with multiple accounts by anyone who care enough to do so. @isaacs what are your thoughts on removing this API all together? i.e. only unpublish of specific versions is allowed. It makes the name squatting case more prolific, but I'm pretty sure it's the only way to be certain. |
Yeah, that sucks. Since an admin usually has to step in there anyway, they can just delete the doc entirely.
Either way, it's still in the db until a compaction, and previous revs are just as gone as _deleted revs after a compaction, so nothing changes.
No, actually, but thanks for making me walk through this. The error message was completely useless there :) Fixed on eff893e |
|
@isaacs Can you explain a little bit why force publish is being removed (I know you had a summary at the top but I'm wondering if there there was a long history of abuse/misuse to point to) and why it was prioritized over something like getting npm stats back up? I found force publish useful. It wasn't something to abuse but I was glad it existed when I needed it. |
|
@isaacs Ok I see, very clever. For the other 90% of people who don't understand CouchApps this is how it works:
{
_id: "winston",
_rev: "42-somerev",
name: "winston",
time: {
// All the old time stuff
"unpublished": {
"name": "indexzero",
"time": 1392186915505
}
}
}
for (var v in oldTime) {
if (v === "modified" || v === "unpublished") continue
assert(doc.time[v] === oldTime[v],
"Cannot replace previously published version: "+v)
}
@isaacs ramp up that support team on social engineering sir because I suspect they're going to get a fair amount of requests for this. Any thoughts on say ... only doing this for packages with more than 10 versions? I know lots of folks who publish to the same test package name and then unpublish. Preventing that is going to get pretty ... frustrating. |
|
@indexzero You could use a few things to determine if a package shouldn't be able to force push (dependents, number of installs, how long it has existed, is it 1.0+ etc.). IMO I think removing it completely is the best way to go and documenting this properly is going to be really crucial to prevent frustration. |
I don't think that's the case. But I'd love a patch that adds a test. Try checking out this repository and running whatever tests against it that you'd like.
Are any of them on this thread? If not, can you please @-mention them so that they can be a part of this discussion? Thanks. (The npm tests themselves do this, but clearly I'm ok with this, so we can refactor those tests.)
Why is that?
@jdalton force publish changes the bits that are returned for a specific version number. This means that pinning your version is not a reliable way to ensure that you have a specific version of a package. It reduces data integrity, and can lead to difficult-to-resolve conflicts in the database. Many users have asked for this over the years. This wasn't prioritized "over download stats". I worked a bunch on the npm stats system recently, in fact, and @seldo is researching time-series databases for us to use to efficiently deliver this information more up to date and more on-demand that it was before. The raw data is still all backed up, and we will be providing the counters again soon, and better. Hooray for multiple brains doing multiple things in parallel! We're not going to do anything where some arbitrary limit means you get to start doing forced publishes. Changed bits in a package with a single version is still changed bits that can break your program in mysterious ways that are hard to detect. |
Yaaa buuut... it was pretty useful, & I'm glad I had it available when it was needed.
The reason I ask is because there are a few issues open for the download stats and it seems like a more pressing issue than removing working/useful features, like force publish, from npm. Was there an open issue to remove this feature or was it on some roadmap? This kind of looks like you're dropping it on your users, 🎊, and using the issue tracker as a way to broadcast the change. Seems a bit backwards (usually goes the other way, an issue is opened, discussed, PRs made, & then merged). |
|
To explain the checking algorithm: For
There might still be a hole in here somewhere, so if anyone has a suggestion or patch, I'd be happy to review it. AFAICT, this ensures that unpublishes are always accountable, and that versions can never be re-used, causing weird surprises for other users of those modules. |
|
Sounds great. I've seen a few module authors abuse republishing in the past. +1 |
|
It may be hard, but that doesn't mean it's not important. Being able to depend on SHAs rather than semvers for specific instances of packages seems like it would have been a much less disruptive way to prevent the surprise code change issue for projects that require/desire that level of lock down, and it's a more semantically correct use of the available information about a package (SemVer rule 3 notwithstanding). |
|
Hi, what I know from Sonatype Central maven repo releases is a "staging" I think, most of the problem people have with this npm issue would Best wishes, On Fri, Apr 18, 2014 at 6:53 PM, Jason Cheatham notifications@github.comwrote:
|
|
I like the staging area idea. It's another checkpoint to catch mistakes, and it would take care of failed publishes. |
|
A similar, but potentially simpler, option to the staging area would be to have public and private/unlisted publishing options. A "private" package would be installable by anyone using the exact semver, but it wouldn't be listed when running |
|
How about having that private version just be worked on locally, and not published until it's ready? Doesn't seem that that actually solves a problem. |
|
Say you have a project that depends on one of your npm packages, and you run CI on something like Travis. It'd be awesome if you could test your final configuration so that when you publish, you're publishing what you tested. |
|
Exactly! That's what git is good at! |
|
Github in particular came to my mind. You can push anonymous commits to your repo as much as you want, and then you tag a release when it's ready to go. You don't have to work offline until you get it just right before making a final push. Of course, if you screw something up on Github you can still force push if necessary. |
|
npm is not Sonatype Nexus or Maven. It has different semantics, different implementation, and serves a different platform. Expecting an exact 1-to-1 mapping of Nexus or Maven to npm is not going to happen. Instead, let's zoom out and talk about use-cases. You want to publish something, but not release it to the world. Only people explicitly asking for this thing should get it. You can do that with npm, pretty easily. In the "foo" package folder: Then, somewhere else, you can do: to get the beta version. Later on, let's say you want to promote the "staging" package to become the new default install for people who do Here's how this works:
None of this necessitates re-publishing an already-used version number, and the benefits of preventing overwriting are too great to justify changing this decision in order to accommodate a use case in the same way as Maven, when npm already does accommodate that use case in a different way. |
|
Note that |
|
@isaacs Thanks for the clear explanation of tags! Is there perhaps an environment variable that would set the default |
|
I have relatively little experience with other package repositories, so I'm not trying to say npm should be equivalent to something else. I'm just suggesting that the experience could be improved, both for users and admins. A developer shouldn't have to burn a version number (this is not always trivial) simply because of a publishing error, and it's unfortunate that correcting mistakes requires the intervention of an admin. Tagging sounds promising; I'll have to think about it some more. If I publish with tag "staging" (or whatever), can I publish the same version again with a new tag (until it's "latest"), or will I still have to version bump? I get the sense that that version number is still gone, which is the primary thing I'm trying to avoid. |
So you cannot publish the same version twice but with a different tag. You Michał Gołębiowski |
|
I understand that they're distinct concepts, although the fact that you can publish and tag at the same time ( The decision to disallow updates to a particular version is going to occasionally cause problems, either due to human or technical error. This is known, and there is a mitigation strategy in place, which is to email support and ask for the version number to be manually deleted from the repository. All I'm suggesting is that there could be relatively simple technical measures in place that would accomplish the same goals without requiring 1) a version bump when package contents haven't substantively changed, and 2) the sometimes lengthy wait entailed in getting human-in-the-loop support. |
No. You can never publish that same version again. But Good news! You don't have to! Because you can point the tags at whatever published versions you want. There is no need to re-publish version
In practice, this type of error is extremely rare, especially using the latest version of npm, because now publishes are a single atomic PUT that will either succeed or fail, 100%. There is not any need to publish the same version again. If the PUT fails, then it didn't make it to the database, and so the version number is not "taken". If it doesn't fail, then you don't have to republish, because it won't ever end up "half published", no matter what kinds of network errors might occur. If the issue is: "Oh, no, I published version 1.2.3, but I forgot that I was supposed to have updated thus and so before doing that!" well, ok, get your processes in order. If you email support, we can help you with that. Or you can just accept it, and bump the version number.
Sorry, that is not how npm works. A If you want to "stage" a publish in this way, then you can do this with git tags or branches quite easily. |
|
I've read the thread and the blog post, and can see the reasoning behind this change. It just seems like that goal could have been achieved without entirely taking away a valuable ability from developers or necessitating significant changes to the registry's architecture. I suppose I assumed this kind of issue would be more common than it apparently is, though, so it probably wasn't worth the effort. Ah well. I'm not opposed to documentation trails or human intervention, but it's not nearly as fast as being able to self-correct. Is support@npmjs.com the best place to go for admin intervention? |
|
Tagging plus atomic publishes probably solve my problems. |
|
Well, if nothing else I have a better idea of how tags and versions work now. :) One workflow that seems to work well is to always use patch versions for your package, and use tags to denote canonical versions. So you'd use 0.1.1-1, 0.1.1-2, etc. for the package, and just tag the last one as 0.1.1. That lets both 0.1.1 and ~0.1.1 work properly as dependencies. |
|
Is there a way to undo an unpublish? New to npmjs and just getting started, there was zero warning or (y/n) confirmation when doing an "npm unpublish" that this irrevocably discarded a version number. Perhaps it would be a good idea to warn and ask confirmation in such a drastic case? I was just trying to update the docs/README because they were malformed. In any case the project is: https://www.npmjs.org/package/bokehjs which is the front-end component of the Bokeh (http://bokeh.pydata.org, https://github.com/ContinuumIO/bokeh) library. The js version and the the python version are synced, so a different version number is not feasible. |
|
I'm sorry for this. A confirmation on unpublish is a good idea. No, there is not a way to do this. Once published, the version number is taken, for good. You can remove it, but you can't ever re-use it. The costs of allowing this are simply too great. The only viable solution is to either bump the version of the python lib as well, or design Bokeh so that it doesn't rely on the versions being identical. |
|
It's sad to see doctrinaire policies become an excuse to abdicate real judgment in all instances.The original upload was also only playing around to gain familiarity with npmjs. I will extend my suggestion to warnings and confirmations on publish as well, so that your users may have some indication, any idea at all, that the publication is also an irrevocable action from which they may not recover simple issues. |
|
It makes sense not to allow differently publishing to an unpublished version - but is there any way to revert an unpublish? |
Not without significant human intervention. @bryevdv Perhaps I misunderstood, if so, I apologize. Is there something you would like un-un-published? Can you please email support@npmjs.com with the details? |
|
@isaacs I sent a message there the other day but I will resend thanks. |
I fucked up and can't republish the broken 0.1.0 version (npm/npm-registry-couchapp#148)
|
This issue is resolved. If you have a rare issue that requires re-publishing over an old version of a package, please email us at support@npmjs.com and explain your situation. However, note that if you are tying the npm version to some external versioned thing, such that The change discussed in this issue has been in production for several months. Locking it now, so that further discussion is directed to other channels. |
|
UPDATE 2015: Back in the winter and spring of 2014, the npm registry was a much smaller and simpler piece of technology. Now, it is much more dangerous and difficult to properly remove every trace of a package in such a way that it does not introduce data conflicts. Removing and re-publishing old versions is not supported. If your situation is truly unique, and for some reason it is fundamentally impossible to bump the version number and proceed, please let us know by emailing support@npmjs.com. |
In b054e48 and 516262d, the ability to publish over a previously published version is removed. This hasn't been published live yet, but it has been tested extensively, and I think it's a Good Thing.
With this change, if you publish
foo@1.2.3, you can still un-publishfoo@1.2.3. But then, you will not be able to publish something else to that same package identifier. Not now, not never. Deleting documents entirely will be forbidden (though of course old revs still get compacted out) and we'll have a lot more visibility into what each change in the db does.I wouldn't characterize it as a security fix, per se, but it does reduce a significant gotcha that users have run into over the years, which is especially annoying when deploying Node servers to production. If the bits are gone, ok, it's an error, you notice and deal with it. But if the bits are changed, then this can be really hazardous.
I'm about 85% sure that this is the right approach. It enforces a best-practice that is pretty much universally agreed upon. But, since it is reducing functionality in a way, I think it'd be good for people to speak up if they have a use case that'll be impacted.
The text was updated successfully, but these errors were encountered: