deps: remove corepack

[darcyclarke]

"Package managers" manage packages & their versions. Package managers are packages themselves. This makes a "package manager version manager" redundant. Improvements can be made to the existing default tooling (ex. devEngines).

Users can still access Corepack via npm, npx (ex. npm i -g corepack or npx corepack) or source as they were prior to #39608.

Fixes: #51888
Related to: #51951

@nodejs/tsc @nodejs/corepack @nodejs/npm @nodejs/package-maintenance

[nodejs-github-bot]

Review requested:

• @nodejs/actions
• @nodejs/security-wg
• @nodejs/startup
• @nodejs/tsc

[ronag]

I mean if we are not removing npm then this is kind of a valid point...

[mcollina]

I don't think this is feasible, as there is massive user support behind having yarn and pnpm binaries.

However, I think you could implement the proposal in #51931, which would solve have the same user need without corepack.

[ronag]

I don't think this is feasible, as there is massive user support behind having yarn and pnpm binaries.

However, I think you could implement the proposal in #51931, which would solve have the same user need without corepack.

Isn't it possible to install both yarn and pnpn with npm?

[targos]

Corepack doesn't only exist to install yarn and pnpm. It manages multiple installed versions and ensures the right one is used based on the current project.

[mcollina]

Isn't it possible to install both yarn and pnpm with npm?

One command in a bash script/Dockerfile/instruction is too hard/burdensome for many users.

Corepack doesn't only exist to install yarn and pnpm. It manages multiple installed versions and ensures the right one is used based on the current project.

The perceived value for users is that it allows them to use yarn and pnpm without a specific install command. They are mostly oblivious to the problems of "multiple installed versions" and how to manage this safely.

[anonrig]

But they can't pin versions. I'm -1.

[darcyclarke]

You can pin versions.

[GeoffreyBooth]

One command in a bash script/Dockerfile/instruction is too hard/burdensome for many users.

So these users are sophisticated enough to use a package manager but too novice to install a package manager?

[aduh95]

I think I agree with what @mcollina meant, but I don't really agree with the phrasing in #51981 (comment).

IMO, it's more useful to think about the "least friction path":

• without Corepack in the Node.js distribution, the path of least friction is to use a globally managed version of your package manager (either the provided npm binary, but another package manager installed by e.g. npm i -g yarn).
• with Corepack in the Node.js distribution, the path of least friction (for non-npm project at least) is to pin package manager per project, effectively treating the package manager as a dev dependency. (And even more so since feat: Pins the package manager as it's used for the first time corepack#413 landed.)

Of course, whether treating a package manager as a dependency is desirable is up for debate (but let's try not to do that in this thread to keep the conversation about the proposed change), but from what I heard it's one reason that Corepack users have found compelling; and personally, I think the whole ecosystem would benefit from that pattern being more wide-spread (for the same reasons lock files were a good thing for the ecosystem).

Also, it does reduce some friction for users of Yarn and pnpm, which is also a positive IMO. (I reckon that point is debatable, but at least for Yarn Berry users it's pretty clear the UX is better with Corepack).

Certainly Corepack is not strictly necessary to achieve what its users are doing with it, it's not the only tool that can do it, and it's far from perfect. However Corepack users are still getting value out of it, and certainly it would not have gotten the same interest if it wasn't maintained within the project and distributed with Node.js.

TL;DR Having Corepack removes some friction for some of our users. Removing it adds friction, I don't think we should do that.

[ShogunPanda]

I'm -1 here as well.

Not applicable anymore. See #51981 (comment)

[GeoffreyBooth]

Having Corepack removes some friction for some of our users. Removing it adds friction, I don’t think we should do that.

I don’t think this is really in dispute; the question is whether the reduced friction is worth the cost. We could bundle CoffeeScript, too, and that would also reduce friction for some of our users. We could bundle any number of things, some of which might arguably be more useful or serve more users; I think the general rules of what should go into core should apply to what goes into the bundle. Arguably those rules should mean we exclude npm, but npm got included long before those rules were written; and besides, each addition should be judged on its own merits.

[GeoffreyBooth]

It’s not clear to me what goals Corepack aims to achieve, nor why inclusion in the Node bundle is necessary for achieving them. Therefore I think we should just remove it and spare ourselves the maintenance burden and security risk associated with Corepack, and users who wish to continue using it can install it directly, such as via npm install --global corepack or via the equivalent command in the package manager of their choice.

[ShogunPanda]

After a careful thought I changed my mind. LGTM.

[ronag]

I think the DX is a good idea and wouldn't mind pursuing it. But this corepack question is taking too much effort.

I think there already are alternatives (though maybe not as user friendly)

You can install corepack:

npx corepack
or
npm i -g corepack

You can install a package manager

npm i -g yarn
or
npx yarn

You can pin a package manager version to your project.

npm i yarn@{version}
npm run yarn {command}

[mcollina]

I’m -1. corepack is part of the flow of many devs.

[arcanis]

Sure, except it's never been a supported way to choose npm? This is pretty well documented so I don't feel like I need to go into why, but the current most used package manager not having a good path in that feels like exceptionally bad DX that will not lead to anywhere except where we currently are.

Different projects have different DevX; the DevX choices made by the Yarn team have no impact on the npm team, and vice versa. If you want npm to be distributed primarily through Corepack you're free to open a feature request on their repository and lobby them, but I don't see that as a reason to deprive other package managers from a tool useful to their users.

From the user's perspective I don't see how this can be significant

It's a breaking change, and it clearly signals that Corepack will never be enabled by default, which is not the intended end state (don't forget that this all started from an issue from one of your users asking for Corepack to be enabled by default).

A third reason is that such moves can erode some of the trust that users have in the project. That's not specific to Node.js, it's just how open-source projects work in general. End users who started using Corepack to manage their package managers and sold it to their teams because it was supported by Node.js won't forget if the carpet is pulled from under them, which may make it harder for them to trust what you ship.

Can they trust require(esm) to eventually be unflagged, or is there a chance it'll get removed if someone has an ideological issues with it? Same with fs.glob, can it be used in our scripts, or should we expect it to be removed altogether? And none of those two features had a TSC vote requesting them, or had been distributed for the past two years.

From what I've understood is that the concerns with this PR is mostly related to the politics/feelings of "demoting" corepack than any technical or user experience issue

I think it's rather the other way around: this PR is obviously about politics and completely ignore the opinion of the pnpm team, the Yarn team, the Node.js TSC, the impacted users, and of course the Corepack team. It doesn't engage in a constructive way, and handwaves away users already using the feature.

Tbh in any other project I'd have seen such PRs as trolls; attempting to rollback changes made by contributor X without their approval, or adding in the project core documents that "we promise to never do the very thing contributor X is working on" by pretexting the status quo, all that feels grotesque to me.

[ronag]

I mean we flag things experimental for a reason.

[ronag]

It's a breaking change,

Of an experimental feature. It's not like we never do breaking changes and there is a reason why things are flagged as experimental. But yea, it has an affect on users, that's obvious, whether it's signficiant or not is another question.

and it clearly signals that Corepack will never be enabled by default, which is not the intended end state (don't forget that this all started from #50963).

This has been discussed so many times now that I have nothing further to add.

I think it's rather the other way around: this PR is obviously about politics and

Yes it's political. Either direction is political at this point.

completely ignore the opinion of the pnpm team, the Yarn team, the Node.js TSC, the impacted users, and of course the Corepack team.

This is a bit unfair IMHO.

It doesn't engage in a constructive way, and handwaves away users already using the feature.

There has been constructive engagements in other forums.

[gulbanana]

Pivoting from "corepack is the new direction of node package management, check it out" to "corepack has been removed and its developers ejected" in the space of two years would be really bizarre.

[ronag]

Pivoting from "corepack is the new direction of node package management, check it out"

Where/When this communicated by the project? I wasn't involved back then so I am missing some context there.

to "corepack has been removed and its developers ejected" in the space of two years would be really bizarre.

What do you mean by "developers ejected"?

[bnb]

Different projects have different DevX; the DevX choices made by the Yarn team have no impact on the npm team, and vice versa. If you want npm to be distributed primarily through Corepack you're free to open a feature request on their repository and lobby them, but I don't see that as a reason to deprive other package managers from a tool useful to their users.

Sure, I wasn't arguing the choices - I was pointing out that an assertion made that could lead folks down a certain logical path was incorrect. Your response further supports my message?

Can they trust require(esm) to eventually be unflagged, or is there a chance it'll get removed if someone has an ideological issues with it? Same with fs.glob, can it be used in our scripts, or should we expect it to be removed altogether?

Deeply confused by this point. Users should absolutely not rely on experimental features for production, and if there are substantial issues with either of those features, they should absolutely not ship. There are a number of historical instances of this happening in different ways, I think the first one I experienced was Async Hooks and that being rewritten... two or three times? It was so long ago that I was cis, straight, and married.

And none of those two features had a TSC vote requesting them, or had been distributed for the past two years.

A TSC vote is generally something that should try to be avoided, it's not a good thing that we got to the point with a feature that we were required to call a vote - it's a last resort. I don't think this comes off how you meant it to.

While I personally don't think Corepack is an ideal solution, I do respect that both Yarn and pnpm want it as their delivery method and have put time and effort into it and am relatively fine with it shipping if we actually do the work to document our reasoning behind certain aspects of it (which I have documented at length in other spaces, and don't feel the need to reiterate here).

That said, there seems to be a bit of sunk cost in the points being made in this thread - we've gotten too far, too much work has been put in, users won't accept Corepack if it's not in core anymore. I do not think this should ever be a valid argument for not shipping something that's currently experimental, and would really strongly advise another rhetorical avenue as it's not one that's been historically successful in making progress, and I don't think that's going to change.

[GeoffreyBooth]

don't forget that this all started from an issue from one of your users asking for Corepack to be enabled by default).

And that user's problem was that different patch versions of pnpm were writing different lockfiles, and they wanted consistency for their project to avoid different teammates constantly updating the lockfile. Which is basically a bug in pnpm, and should be fixed by pnpm. It's very far outside the scope of the Node project, and shipping a package manager version manager is an extremely overengineered solution if your only goal is to fix this bug.

Which gets back to my point: I don't see what problems Corepack solves that are appropriate to be solved by Node. And I've asked so many times by this point, on multiple threads and in multiple meetings, that I don't think I'm ever getting an answer. So my conclusion is that Corepack doesn't align with the goals of the Node project and should live as its own project, which it already is, and we can recommend installing it alongside installing Node for users who want it as their choice for package manager management solution.

[gurgunday]

Will there be a vote on this matter tomorrow?

It looks like most members of the TSC agree that moving from corepack enable to npm install -g corepack is reasonable

[wesleytodd]

Will there be a vote on this matter tomorrow?

With the planning under way to propose the PMWG take on this domain of work, the lack of a replacement workflow, the npm team working toward devEngines or similar, next weeks collab summit session, and the many still open PRs around policy (whew thats a lot), I believe it would be unwise to have a vote this soon. Just like the original vote should not have happened, neither should this in the current state. @bnb is absolutely correct:

A TSC vote is generally something that should try to be avoided, it's not a good thing that we got to the point with a feature that we were required to call a vote - it's a last resort.

[BridgeAR]

I believe we have to have corepack or an alternative that provides a similar functionality. I won't agree on removing corepack as long as there is no alternative as it is important for multiple developers and it would require additional effort for them to change their workflow. I don't think that's good for those users.

[GeoffreyBooth]

@mcollina @BridgeAR You both mention wanting to provide a potential alternative. An alternative that does what? What are the use cases that you think the Node project should be supporting? What do you think of the alternative proposed in nodejs/package-maintenance#591 (comment)?

All that this PR does is change how users get Corepack. Instead of running corepack enable, they run npm install -g corepack. This would resolve all the security and maintenance concerns raised in #52107.

[marco-ippolito]

I'm -1 on removing corepack. I feel this issue is causing a lot of unnecessary stress to collaborators even not involved directly with corepack. I dont see how the removal would make a positive impact on the project. People are working on corepack to address the current issue preventing it being enabled by default.
I think there are benefits of having a way to handle multiple package managers and even if the current solution is not the best, its a drive to bring the development forward and improve the ecosystem. Removing corepack means halting the development of something potentially beneficial.

[ronag]

I think there are benefits of having a way to handle multiple package managers and even if the current solution is not the best, its a drive to bring the development forward and improve the ecosystem.

How does this PR technically relate to what you are writing?

Removing corepack means halting the development of something potentially beneficial.

Why does removing corepack halt the development?

EDIT: To avoid going further off-topic, I have removed the last sentence as it is not vital to the point I'm trying to make.

[Trott]

A TSC vote is generally something that should try to be avoided, it's not a good thing that we got to the point with a feature that we were required to call a vote - it's a last resort.

Feel free to mark/hide this comment as off-topic if you think it is wandering too far from the core conversation, but I think "vote as a last resort only" is a common refrain--I've said it many times myself!--that is overstated and can be misguided.

It has often been the case that conversation has grown fruitless, repetitive, and toxic on some topic, but the "we only vote when we absolutely have to" approach results in the TSC returning the conversation to GitHub, week after week. Weeks turn into months and people burn out. In my opinion, there have been many times where it would have been better for the project and the people in the project had the TSC stepped in sooner and made a decision with a vote. Participating in votes is the only thing we require "Voting TSC members" to do! They don't need to attend meetings or contribute to the project in any other way. But they must participate in votes. That's because votes are important to maintaining the health of the project.

A point comes where difficult decisions must be made and not deferred. Things get to the point where making the "wrong" decision is better than not making a decision at all. It sure seems plausible that the best approach here is to see what progress can be made at the collaborator summit, and then make a decision with a TSC vote. This doesn't seem (to me, at least) like a conversation where consensus is likely to be reached or some extraordinary new approach is likely to come out of a few more days/weeks/months of conversation. The maintainers of corepack, the maintainers of Node.js, and the users of Node.js and corepack all deserve to have a reasonable degree of confidence about the support or absence of support for corepack in Node.js on the part of the ultimate arbiters of the project. I don't envy the voting members of the TSC having to make this decision, but these decisions are exactly the only significant role of the TSC and they should not shy away from this one, IMO.

Commit (or re-commit) to corepack or don't, but please don't make the people who love corepack and the people who think corepack is a mistake endure more of this.

(And yes, even with a vote, this might come up again in six months. That's OK. Even if it buys people only 6 months of breathing room, it's worth it. Have another vote in six months. It will be fine.)

[bnb]

Similar "feel free to mark this as off topic".

I think "vote as a last resort only" is a common refrain--I've said it many times myself!--that is overstated and misguided.

I don't disagree with this, but I also don't think that changes the point of my message which was that a vote isn't (currently!) something that's a positive indicator of the sentiment of the project/progress of something people want added, which is how it was being referred to.

In my opinion, there have been many times where it would have been better for the project and the people in the project had the TSC stepped in sooner and made a decision with a vote.

Agree.

They don't need to attend meetings or contribute to the project in any other way. But they must participate in votes. That's because votes are important to maintaining the health of the project.

I genuinely think this is unhealthy for the project and needs to be looked at. I have been told that a TSC member told their direct reports at their employer that they actively do the absolute minimum to keep the role, nothing more. This is not a healthy state of existence for leadership of this project.

Commit (or re-commit) to corepack or don't, but please don't make the people who love corepack and the people who think corepack is a mistake endure more of this.

I agree with this, and hope that's obvious throughout my various comments across the handful of corepack issues.

[ShogunPanda]

A TSC vote is generally something that should try to be avoided, it's not a good thing that we got to the point with a feature that we were required to call a vote - it's a last resort.

Feel free to mark/hide this comment as off-topic if you think it is wandering too far from the core conversation, but I think "vote as a last resort only" is a common refrain--I've said it many times myself!--that is overstated and misguided.

It has often been the case that conversation has grown fruitless, repetitive, and toxic on some topic, but the "we only vote when we absolutely have to" approach results in the TSC returning the conversation to GitHub, week after week. Weeks turn into months and people burn out. In my opinion, there have been many times where it would have been better for the project and the people in the project had the TSC stepped in sooner and made a decision with a vote. Participating in votes is the only thing we require "Voting TSC members" to do! They don't need to attend meetings or contribute to the project in any other way. But they must participate in votes. That's because votes are important to maintaining the health of the project.

A point comes where difficult decisions must be made and not deferred. Things get to the point where making the "wrong" decision is better than not making a decision at all. It sure seems plausible that the best approach here is to see what progress can be made at the collaborator summit, and then make a decision with a TSC vote. This doesn't seem (to me, at least) like a conversation where consensus is likely to be reached or some extraordinary new approach is likely to come out of a few more days/weeks/months of conversation. The maintainers of corepack, the maintainers of Node.js, and the users of Node.js and corepack all deserve to have a reasonable degree of confidence about the support or absence of support for corepack in Node.js on the part of the ultimate arbiters of the project. I don't envy the voting members of the TSC having to make this decision, but these decisions are exactly the only significant role of the TSC and they should not shy away from this one, IMO.

Commit (or re-commit) to corepack or don't, but please don't make the people who love corepack and the people who think corepack is a mistake endure more of this.

(And yes, even with a vote, this will come up again in six months. That's OK. Even if it buys people only 6 months of breathing room, it's worth it. Have another vote in six months. It will be fine.)

I agree with @Trott.

While consensus would be beneficial, we can't halt the TSC because we're afraid of passing a vote by 50%+1 result.

I'm very new in TSC but honestly I don't even understand what are we fearing. Do we fear we make our users not happy? If that happens and we receive complaints we will revert it and just move on. Many people often emphasizes that nothing is set on stone, including our decisions.

If we don't have consensus we can decide by majority and it is totally ok. Folks, we do everything in life via "trials and errors" approach. Why should TSC activities be exempted from this?

On the other side, forgive me for the strong position here, by not deciding and by arguing with each other we are definitely showing our worst side. I think that people from the outside perceive that we are unable to decide and thus our function as technical guidance is totally impaired. In other words: we are here to coordinate and decide. If we don't do that, we are we here for?

On the final and more important side I sense (but I might be wrong) that several fellows in the TSC to be close to burn out or at least lose their temper, due to arguing or due to inability to decide. Nobody wants it as we care about persons first and the project then.

So let's make sure that at the Collab Summit we finalize his once and forever. And by "forever" I mean to vote and now and then move on other topics. As Rich said, this might come back in 3, 4, 6 or 12 months and it is totally ok to revisit.

Love you all! ❤️

[gurgunday]

I agree with everything you said @ShogunPanda ♥️

Just want to add one comment to this:

I think that people from the outside perceive that we are unable to decide and thus our function as technical guidance is totally impaired. In other words: we are here to coordinate and decide. If we don't do that, what are we here for?

I don't think trying to exhaust debating and showing the world that TSC members understand the power of the vote and want to see others to participate as much as possible before potentially making a change that'll result in a big reaction is perceived as a weakness but a strength, because it is a very good demonstration of the unique and open nature of the Node.js project — that decisions are not being made behind closed doors and forced on others — this is very important to a lot of users

With that said however, I fully believe that the vote is the only solution here 😁