Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Discussion around release strategy of Yarn v2 #766

Closed
2 tasks
brentvatne opened this issue Jan 25, 2020 · 83 comments
Closed
2 tasks

[Feature] Discussion around release strategy of Yarn v2 #766

brentvatne opened this issue Jan 25, 2020 · 83 comments
Labels
enhancement

Comments

@brentvatne
Copy link

@brentvatne brentvatne commented Jan 25, 2020

  • I'd be willing to implement this feature
  • This feature can already be implemented through a plugin

Describe the user story

As a library author, I want to recommend the best and most reliable way to use my libraries, so I recommend users use yarn. With yarn v2 I can no longer do this because it is not compatible with React Native. I would like to continue to recommend yarn.

As a JS developer, I want to use one package manager everywhere. I use yarn in all of my projects and I would like to continue to do this but I won't be able to because yarn v2 will not work in many of them.

Describe the solution you'd like

I think that until yarn pnp is ready to be used in all major projects in the existing ecosystem, it should be released under another name. In environments where yarn pnp works well, people can adopt this easily and enjoy the benefits. Once people experience it, library authors will start to make sure their libraries work with it. When it has become very popular and widely supported, release it under a new major version for yarn or keep the name if the branding is working.

Describe the drawbacks of your solution

It is going to be unpleasant and tedious work to rename everything and create some different branding and domains. Probably there are more things I haven't considered.

Describe alternatives you've considered

The main alternative is to continue to release yarn v2 as-is. This would be less work but it runs a serious risk of hurting the trust that the community has gained in yarn. People will no longer be able to depend on yarn working on all of their projects, and library authors will move away from recommending it. There will always be a concern about large breaking changes potentially disrupting workflows and users will build around other tools instead.

@arcanis
Copy link
Member

@arcanis arcanis commented Jan 25, 2020

Yes, we are discussing this at the moment. I can see the problem and definitely don't want to break workflows (hence why we hold off on updating the latest tag, for example).

The rational behind reusing the name is mostly to make clear that no evolutions will happen on the 1.x trunk (I mean, maybe someone will step up and take over, but I would find this highly unlikely). That being said, maybe a different release name doesn't necessarily preclude from this.

What I think could be a good way to go forward would be to:

But to release the CLI binary under the berry package name on npm (which I already control), with the berry binary name. To prevent the confusion I talked about earlier, the yarn package would also be marked as deprecated (deprecated isn't the right word, but it's the name of the npm feature. The actual message would say "frozen", or similar word we could discuss) and would recommend to try out the berry package.

Once the ecosystem is ready (for example we can put a reconductible hard reevaluation date on Jan 24 2021) we would then consider to take back the yarn package name.

Would that answer your concerns?

@janicduplessis
Copy link

@janicduplessis janicduplessis commented Jan 25, 2020

What about using v1 instead of legacy for now to refer to yarn v1? I think it is a bit early to call it legacy, at least until v2 is more widely supported. I also think v2 / PnP should not be the default mode until supported by react-native and flow which are 2 very large communities.

@fubhy
Copy link

@fubhy fubhy commented Jan 25, 2020

I don't think that that would be sufficient. It would still technically live under the yarn branding with all the packages and also the domain being yarnpkg.com. Wouldn't that be even more confusing?

Instead, I think the right move would be to entirely embrace the name berry and use it everywhere (including domain). You can still technically deprecate yarn (I personally wouldn't) by adding a nice large banner on yarnpkg.com and in the installer that leads ppl to the berry website so they can check it out voluntarily and switch over once the migration from yarn to berry is sufficiently simple.

@cpojer
Copy link

@cpojer cpojer commented Jan 25, 2020

Yarn was primarily built to solve problems at large companies who had trouble managing their dependencies. They have most likely made significant investments into the integration of Yarn. Now, these companies do not have a viable migration path and will be stuck with Yarn v1 for years to come given Berry's change in dependency management behavior. I believe deprecating Yarn v1 is premature and I am confident that various people and companies will keep supporting it if necessary. There is no need to turn the existing community away given that they are quite happy with the current version of Yarn.

I am also in favor of releasing Berry as a separate project. Let's reconsider replacing Yarn with Berry once it gained significant community adoption, and is known to support all parts of the JavaScript ecosystem just like Yarn v1. At that point in time we'll all be able to confidently agree that Yarn v1 is obsolete.

@cwhitten
Copy link

@cwhitten cwhitten commented Jan 25, 2020

If the governance believes that berry will in fact subsume yarn but important pieces currently labeled experimental are not yet complete (pnp, compatbility, etc), why not use a "next" metaphor to denote where the tool is going and set good expectations while keeping the branding? It feels premature to immediately label yarn 1 "legacy" at the moment.

@brentvatne
Copy link
Author

@brentvatne brentvatne commented Jan 25, 2020

I think your proposal will largely solve my concerns @arcanis, with a few changes:

  • yarnpkg.com leads to yarn v1. the landing page includes information about it being in maintenance mode with major feature work being done now on berry, and provide a pitch for berry.
  • berry.yarnpkg.com leads to berry/yarnv2
  • leave out "deprecating" npm package for now, this may cause some concern from users installing it and i think won't be the best way to help with adoption of berry

@cpojer
Copy link

@cpojer cpojer commented Jan 25, 2020

@paularmstrong
Copy link

@paularmstrong paularmstrong commented Jan 25, 2020

Have to agree strongly with everyone here. At Twitter, we're going to be stuck on Yarn v1 or else need to move back to npm and hope that the improvements they've put in over the last couple of years will be enough that our tooling built on top of yarn can be ported over.

@janicduplessis
Copy link

@janicduplessis janicduplessis commented Jan 25, 2020

Thinking about this more, I'm not a fan of changing the name and adding another package manager to the JS ecosystem. Let's not get JS fatigue trending again. Is there a technical reason for not supporting the node_modules structure and defaulting to PnP?

I'm sure a lot of people and companies would love to use the new yarn v2 features without PnP until it is more proven and widely supported.

There should be a transition period, especially considering this is completely rewritten. I suggest

  • Holding on on making v2 the default until node_modules support is stable.
  • Make v2 the default in node_modules resolution mode, PnP opt-in.
  • When PnP is widely supported make it the default.

This allows everyone to benefit from v2 improvements while keeping breaking changes and migration very minimal.

@kaatt
Copy link

@kaatt kaatt commented Jan 25, 2020

From the original author of yarn:

Yarn 2 shouldn’t even be called Yarn. Change every piece of a ship and it’s a completely new ship, not an evolution. Want to shed previous baggage? New name. Want to make a completely new codebase with a different project philosophy? New name. Otherwise? Thrash the community.

@fubhy
Copy link

@fubhy fubhy commented Jan 25, 2020

I mentioned this already on Twitter to you @arcanis but I'd like to repeat it here: I think what you did with Berry is amazing and if this was a green field and Node and the ecosystem was in it's infancy this would definitely be a much easier transition. But with 1.2 million packages currently published on the registry and so many projects using Node, npm and yarn (v1) as their backbone, this is waaaay too much of a drastic sweeping change for a single major release while also immediately deprecating the "old" version. Especially considering that it's currently absolutely non trivial to migrate from v1 to v2. Just ask yourself this: Would you expect React v17 to "simply work" after upgrading from v16? Or with a minimal amount of easy fixes? I would.

@arcanis
Copy link
Member

@arcanis arcanis commented Jan 25, 2020

Is there a technical reason for not supporting the node_modules structure and defaulting to PnP?

It's already the case, as pointed out in the blog post. We are investing in both areas. Some improvements are on their way in this regard, and we fully intend to support node_modules if you wish to keep using them.

@janicduplessis
Copy link

@janicduplessis janicduplessis commented Jan 25, 2020

@arcanis In this case does the migration plan I suggest makes sense. PnP looks promising but sadly years of infra work depends on the node_modules resolution. I think this is the only reason for all the push back around yarn v2.

@brentvatne
Copy link
Author

@brentvatne brentvatne commented Jan 25, 2020

I like @janicduplessis's migration plan proposal as well. If people can update to the latest yarn version and continue using it the same as they are today then this will get pnp just a few keystrokes away from users and they will be much more likely to try it out.

@cpojer
Copy link

@cpojer cpojer commented Jan 25, 2020

@janicduplessis @brentvatne Given the complete rewrite of all algorithms in Berry (rather than an incremental major version bump with some breaking changes, bug fixes and new features), what framework could we use to ensure that it meets the bar in terms of behavior and functionality that will enable >95% of users to upgrade without any work involved? It seems that Brent's initial proposal is healthier for the community in the short term.

@janicduplessis
Copy link

@janicduplessis janicduplessis commented Jan 25, 2020

@cpojer

  • Holding on on making v2 the default until node_modules support is stable.

That would be the first point of the migration strategy. Ideally we'd get to the point where node_modules mode in yarn v2 produces a close enough output to yarn v1 that we are confident to make it the default.

If we can get a couple large projects to test v2 to provide feedback on the node_modules mode that would be great and give enough confidence to be able to replace yarn v1 with it.

Do you think facebook would be interested in testing yarn v2 with node_modules mode?

@cpojer
Copy link

@cpojer cpojer commented Jan 25, 2020

We can definitely commit to testing it, but anything that's not a drop-in-replacement at this time will probably not work for us.

Another approach could be to focus on improving Yarn 1's PnP mode and gaining significant adoption first (See: https://yarnpkg.com/lang/en/docs/pnp/ ). However, Given that PnP was released more than one year ago and hasn't gained significant traction makes me worried that the approach may not be viable to become a default at any time in the future and that we'll be stuck with materialized node_modules.

@larixer
Copy link
Member

@larixer larixer commented Jan 25, 2020

@cpojer The node_modules support in v2 is being worked on. It is certainly not ready at the moment, but hopefully will be there soon enough. Even in this case the perfect drop-in replacement might be not possible, more like the replacement in the sense of switching from npm to yarn v1 should be achievable.

@arcanis
Copy link
Member

@arcanis arcanis commented Jan 25, 2020

To be crystal clear, PnP will remain as the default on this repository and the 2.x releases, this isn't something that's going to change.

That being said - I'm ok with dropping the idea of adding a "deprecated" notice on the yarn package. I understand what you say regarding people being unreasonably warning-adverse (honestly the case already presented before with optional peer dependencies and here to we had to find a workaround to make it happen). I don't think it will do them any favour, but it's not a fight I want to have.

The website however should stay as it is. We all have to make compromises, I think asking you to click on an extra link to access the confusing documentation that isn't maintained about the version that isn't maintained is reasonable. Everything is already redirected, so referenced links will keep working and search engines will pick it up.

@jaredpalmer
Copy link

@jaredpalmer jaredpalmer commented Jan 25, 2020

@arcanis I highly suggest you reconsider @brentvatne’s suggested solution. My beliefs are that Yarn 2 is a different package manager at the moment since it has no upgrade path for a significant amount of the userbase. Big players such as FB and Twitter have publicly stated that they cannot use Yarn 2 because of PnP. Like it or not, they have large sway when it comes to JS standardization.

My worry with your proposed plan is that you will damage the reputation of Yarn and kill the thing you have been working so hard on as well. That would be a shame, since what you have built is awesome. However, maintainers including myself, will immediately remove yarn from all docs so as to avoid confusing people around installation. The website needs to be Yarn 1, with a big banner stating that berry is the blessed successor and linking to its docs.

@JounQin
Copy link

@JounQin JounQin commented Jan 26, 2020

Why not just call it yarn2 like python3 so that we can have yarn and yarn2 installed at the same time with compatibility?

@Cretezy
Copy link

@Cretezy Cretezy commented Jan 26, 2020

One current problem is testing Yarn 2. I would love to try it in some of my projects, but doing npm i -g yarn@berry and npm i -g yarn@latest over and over while switching is not viable currently.

It would be nice to have a separate (temporary) package with the yarn binary renamed to berry, maybe under @yarnpkg/berry, to allow both Yarn 1 and 2 installed and usable at the same time.

(This could be done with some bash hacks for now, but a new package would help adoption of v2 until it is "ready" for prime time)

@olingern
Copy link
Contributor

@olingern olingern commented Jan 26, 2020

(I mean, maybe someone will step up and take over, but I would find this highly unlikely

I'm not sure about yarn's governance and its relationship to Facebook (since I believe it was started as a Facebook project), but, having done some and just a bit of work on v1 and v2, I would be interested in maintaining yarn v1. Happy to chat more elsewhere about this.


Not aiming to be controversial by any means, but to highlight that Yarn gained community adoption where many other JS package managers did not ( and many, like pnpm, are quite good!). I think it's worth noting the stability and momentum v1 has while having virtually no work done on it since last year.

When yarn v1 released, people happily migrated because npm, at the time, had some performance issues and it didn't seem like there was a priority around fixing them. If the same migration happened for yarn to berry, then I would say adding deprecation notices to yarn v1 and rolling out [x] months from that deprecation notice would be something most people could get behind.

At the moment, I think the ergonomics and benefits of berry are yet to be realized by most people (including myself) so if there's no governing body to decide how / when to rollout, I would say allowing the community to decide overtime would be the safest option that would not risk fracturing the current user base.

@Daniel15
Copy link
Member

@Daniel15 Daniel15 commented Jan 26, 2020

One current problem is testing Yarn 2. I would love to try it in some of my projects, but doing npm i -g yarn@berry and npm i -g yarn@latest over and over while switching is not viable currently.

@Cretezy FWIW you can specify Yarn version per-project by using yarn policies set-version and committing the Yarn version to the repo. If you do that, the local version overrides the globally installed version. https://legacy.yarnpkg.com/en/docs/cli/policies/

@Daniel15
Copy link
Member

@Daniel15 Daniel15 commented Jan 26, 2020

It would be nice to have a separate (temporary) package with the yarn binary renamed to berry, maybe under @yarnpkg/berry, to allow both Yarn 1 and 2 installed and usable at the same time

This will definitely be the case for the non-npm installation methods (Debian/Ubuntu package, Windows installer, Chocolatey and Homebrew), that part of the release infra just hasn't been finalized yet. It might be a good idea for the npm package too!

@arcanis
Copy link
Member

@arcanis arcanis commented Jan 26, 2020

I've been in a plane for a few hours and got some time to think more in depth about this issue.

First, regarding using a different binary name: I rescind what I said. Changing the package name would be a bad idea for the project health. The reason for that is that many tools are hardcoding Yarn in their logic - for example VSCode, but also WebStorm, Husky, project generators, etc. A different binary name would bring a lot of additional maintenance hurdle not only to us but to the global ecosystem we all care about.

But there's another option: stop releasing Yarn as a global binary. With 2.x we are in a state where yarn set version is already in a good position to be the preferred way to upgrade Yarn, so let's use that. My thinking is:

  • Yarn 1.x will stay as the package on latest.
  • It (almost certainly) won't be updated anymore apart from the occasional security fix.
  • Yarn 2.x will be published on GitHub as a single binary, and yarn set version will install it to the local project.
  • The main documentation will target the 2.x, and the install instructions will be updated to recommend the workflow I described.
  • Once the node-modules plugin is working well enough, we can consider moving 2.x into the binary again. May not be necessary if the yarn set version adoption went well.

With this plan things will stay basically the same as they are right now. Existing applications will keep working, people following instructions on third-party website will keep using the 1.x workflow. The website will stay on the 2.x however, this isn't something I'm willing to change considering how likely it is that I'll be doing most of the support, if this year is a good indication.

@Cretezy FWIW you can specify Yarn version per-project by using yarn policies set-version and committing the Yarn version to the repo. If you do that, the local version overrides the globally installed version. https://legacy.yarnpkg.com/en/docs/cli/policies/

Yes, this is what I have in mind. Together with for example yarn init -2 which would do this automatically it would likely be the best way to go forward with the smallest amount of risk.

@accidentaldeveloper
Copy link
Contributor

@accidentaldeveloper accidentaldeveloper commented Jan 26, 2020

I like the idea of keeping the global binary on Yarn 1. This will allow users and projects to opt-in to Yarn 2 as they are ready.

This creates an opportunity to encourage projects to "pin" their desired version of yarn in the repo using policies set-version. Yarn version pinning is a fantastic feature and creates an excellent environment for gradual adoption.

I'm discovering that there are still a lot of packages that are not ready for PnP. Now that Yarn 2 is stable, most of these packages will start resolving these issues and make the migration smoother.

@fubhy
Copy link

@fubhy fubhy commented Jan 26, 2020

First, regarding using a different binary name: I rescind what I said. Changing the package name would be a bad idea for the project health. The reason for that is that many tools are hardcoding Yarn in their logic - for example VSCode, but also WebStorm, Husky, project generators, etc. A different binary name would bring a lot of additional maintenance hurdle not only to us but to the global ecosystem we all care about.

The global ecosystem is currently built around yarn (v1), npm and ... node_modules. The one thing that will be bad for the project's health, imho, will be the loss of trust in yarn if all these integrations that you mentioned stop working or simply aren't compatible with the documentation and installation guides found on the website and branded as yarn.

Piggybacking on the success and wide spread adoption of yarn v1 to force such a drastic change for the entire ecosystem without any sort of upfront warning, long deprecation phase and, most importantly, a bearable upgrade path is wrong. And it's still wrong even if, technically, the underlying changes are sound.

By branding berry as a direct successor of yarn and, even worse, labeling yarn v1 as deprecated while also hijacking the main website (that pretty much everyone who advertises yarn in their own packages' docs, blog posts, etc. links to) is simply creating a level of confusion that would be hard to recover from.

The level of friction this creates will simply be too much for the vast majority of the community and will eventually drive them away. You are righteously imposing a monumental amount of work on the entire ecosystem with such a sweeping change. I appreciate the effort to fix the brokeness of the current status quo but you can't force this over night.

@gaearon
Copy link

@gaearon gaearon commented Jan 30, 2020

I’d like to throw in a historical anecdote from the React side. Not that it’s the same situation, but might be helpful to consider.

We knew in 2015 that we wanted incremental reconciliation. This is something Jordan prototyped and it seems like a fundamental shift that challenges many assumptions about how React works.

We started on a rewrite around that time. However, we knew we can’t change how React apps work over a short period of time. That would be a large blow to people’s confidence in React. If we create a bad impression, no matter how superior the solution is, the lost goodwill is a major hurdle for adoption. Which would set us backwards rather than forwards.

So instead, we rewrote React to make incremental reconciliation possible. But we kept the old behavior. We even “reimplemented” some of the old bugs to make the transition more seamless. That was the React 16 release: https://engineering.fb.com/web/react-16-a-look-inside-an-api-compatible-rewrite-of-our-frontend-ui-library/

Soon after React 16 release, we added a “strict mode”. It allowed people to opt into a stricter set of warnings that tell you about patterns that won’t work with incremental reconciliation. Judging by the issues we see in open source libraries, Strict Mode is widely adopted. People push component libraries to pass Strict Mode with no warnings. On the other hand, using something incompatible isn’t insurmountable. Some libraries are unmaintained, and it takes months of community effort to get them fixed or migrate away. If we warned without an opt in, everything would be so noisy that people would give up. So this has been a really effective strategy in getting people upgraded.

Now we have an opt-in to incremental reconciliation as well. It’s still experimental but people are trying it and the ecosystem is so much more ready for it than years ago. We also released a set of APIs (Hooks) that make it easier to write code that’s friendly to incremental reconciliation. I doubt we would be in this place if we simply turned out incremental reconciliation in React 16 by default.

I’d like to add that I deeply sympathise with the idea that sometimes, you just need to move forward. And that requires dedication and, sometimes, cutting losses. Still, it’s important to remember that technical innovation can only succeed when there is a climate for it to happen. There is a real cost to scaring the ecosystem, and that cost is that people won’t consider the technical aspects of the solution altogether. Some smart solutions from 1960s still aren’t in use today because the adoption strategy hasn’t been considered as closely as technical brilliance.

Hope this helps.

@freiksenet
Copy link

@freiksenet freiksenet commented Jan 30, 2020

Currently Yarn 2 (at least with PnP) doesn't work for most Gatsby websites. For example themes don't work with pnp, some plugins that write to node_modules don't work with pnp (eg gatsby-plugin-typography), there are many peer dependency issues. We also rely on node_modules being availaible internally. At current stage we wouldn't be recommending any of our users to use Yarn 2 (in fact all our documentation is using npm).

Gatsby project by itself, however, uses Yarn (because of workspaces). We are also interested in leveraging new features in Yarn 2, like releases.

For us @cpojer solution would be preferable, as it would allow us to continue supporting Yarn with Gatsby projects and at the same time use new features inside our monorepo. However, I'm worried that a minor release after a full rewrite will cause too many problems. Releasing Yarn version 2 without PnP as default could be a safer alternative.

@devongovett
Copy link

@devongovett devongovett commented Jan 30, 2020

I disagree with the proposals mentioned here. I think the Yarn maintainers had the right idea with the original launch. It might be painful at first as libraries and tools become compatible, but it’s the only way to ensure the adoption of PnP over the long term.

Shipping with PnP off by default will result in PnP never gaining adoption. We’ve seen this already with Yarn 1, which has had PnP support for over a year. So, the question is, if it is turned off again, how long do we wait to turn it on by default? Another year? Two years? Three?

Yarn 2 is a major version bump, so it won’t break anything unless you upgrade. If you try it out and it doesn’t work for you, you can always downgrade! But, the bug reports that are sent to libraries and tools with incompatibilities are very important to PnP’s adoption, and won’t happen without this process.

I think we need to suck up the short term pain for the long term gain. This is the way. 😉

@Haroenv
Copy link
Member

@Haroenv Haroenv commented Jan 30, 2020

I think one thing to note why pnp might not be gaining mainstream adoption on v1 is because there are no warnings to fix what should be fixed for pnp to work, rather than it being an option

@fbartho
Copy link

@fbartho fbartho commented Jan 30, 2020

I saw the announcement post and was knocked over by how much amazing stuff is coming. Several parts of it solve real problems my team are fighting today.

I’m also conceptually a fan of PnP and have been looking forwards to its future progress since it came out.

Unfortunately, about 1/5 of my apps’ direct dependencies need patches making them incompatible with PnP. We use React-Native, and TypeScript, so we won’t be able to upgrade until those projects are fixed. This feels like a large hurdle that will keep these great tools away from us, and the nature of this migration could fracture the community. I think many people dislike using a “legacy” product, and many people can’t fix some of the stumbling blocks under the planned rollout.

I’d personally love @cpojer or similar proposals of delaying the hard PnP switch until more community investments iron out the rough spots. This lets the community come to know and love all of Berry’s other awesome features, and ideally even better understand the effort of switching to PnP.

I still like @brentvatne‘s proposal as a second fallback plan, it recognizes that community pain may have been underestimated, and might be easier (on a technical level) if slower for the maintainers to advance.

I have enjoyed yarn’s existence and have advocated for it at several shops, and feel it has produced a better npm community by its very existence, not to mention that npm literally adopted some of yarn’s learnings. I hope that that continues rather than making “giving up on yarn” the more attractive road for the majority of users; this pain is real, and if PnP is the future, then we can’t force it on people when the ecosystem isn’t ready or yarn itself doesn’t have enough tools to bridge us there.

@gaearon
Copy link

@gaearon gaearon commented Jan 30, 2020

Yarn 2 is a major version bump, so it won’t break anything unless you upgrade.

That makes sense for libraries. But does it make sense for global commands? If everyone installing Yarn today gets something very different from what people who added yarn to their READMEs meant, this has to be at least a little bit confusing. Maybe I'm missing something.

Shipping with PnP off by default will result in PnP never gaining adoption.

Do you find the opposite of this statement believable? That turning it on by default for users who have no technical expertise to fix PnP issues will lead to PnP gaining adoption? I'm somewhat skeptical of this, although I'd love to be proven wrong.

@devongovett
Copy link

@devongovett devongovett commented Jan 30, 2020

I don’t think installation instructions in readmes have ever been that specific about which package manager to use. Many document npm install but that doesn’t stop people from using yarn or pnpm. pnpm in particular already works pretty differently, which causes some bugs for library/tool authors already, so there is some historical precedent here.

@FezVrasta
Copy link

@FezVrasta FezVrasta commented Jan 30, 2020

When you read npm i pkg-name and you convert it to yarn by yourself, you can expect to get a different outcome.

But if the readme already reads yarn add pkg-name and running this exact command you get a different outcome, people will for sure get confused, or at least, I would.

Personally, in all my packages' READMEs I put both npm and yarn install commands, and I can think of a lot of other popular OSS projects doing the same.

@gaearon
Copy link

@gaearon gaearon commented Jan 30, 2020

Personally, in all my packages' READMEs I put both npm and yarn install commands, and I can think of a lot of other popular OSS projects doing the same.

Yeah.

My concern is that if people's first experiences with Yarn V2 look like #843, open source maintainers will likely start removing the Yarn-specific instructions to avoid confusing their users.

Personally, I don't have investment in what happens either way. But it would be a shame if this set the project back even if the solution is technically superior and the pain is supposed to be short term.

@devongovett
Copy link

@devongovett devongovett commented Jan 30, 2020

Do you find the opposite of this statement believable? That turning it on by default for users who have no technical expertise to fix PnP issues will lead to PnP gaining adoption?

It’s possible I guess, but that’s a pretty pessimistic viewpoint. We can’t move the status quo forward without trying.

Also remember: it’s opt in, so users are choosing to turn it on by upgrading. They also have a way out by downgrading. Users don’t need to “fix” the issues themselves, they can report them, and downgrade if needed.

@mgcrea
Copy link

@mgcrea mgcrea commented Jan 30, 2020

I don't quite get why this has to be forced on end users, if PnP is a superior solution it will eventually find its way to widespread adoption. Why the rush?

I think the famous Linus quote about "never ever breaking userspace" is quite fitting for yarn, as it has become a central piece of software at the core of our projects.

Forcing breaking changes to end users that mostly only care about stability and have actual work to do is wrong in my opinion, moreover, I'm afraid it would irremediably damage this project.

Please ship berry as a standalone new project and put yarn in maintenance mode. Dealing with a third package manager would be quite painless, most of the hard work has already been done for the npm/yarn split.

@0xjjpa
Copy link

@0xjjpa 0xjjpa commented Jan 30, 2020

I second the idea of having berry as a standalone new project. The changes are just too many to keep them in the same brand/binary IMHO. Even if it comes with huge benefits, it’s confusing for newcomers due to how different its architecture is.

The thing is that we get so used to work with these tools daily that we forget that every day new developers come to the ecosystem, meaning they have to go through the whole JS-fatigue process. How are we going to explain to them that yarn is one thing from version 1.x and utterly different beast on version 2.x? If berry had evolved in the same way react with 16.x did as @gaearon mentioned, then I would support the version upgrade. However, even for react 16.x you could easily upgrade, give it a try, and rollback when needed. I do not see that happening at all with berry as it is right now.

I don’t think it’s fair to only worry about the current community and avoid thinking about the ones to come. Let’s learn from other communities that went through similar hurdles (e.g. python ended up choosing python3 and pip3) and see how adoption goes. As an industry, I think we are more used to tackle package diversification than breaking changes on version upgrades. For instance, there are many task runners/bundlers in the industry (e.g. grunt, gulp, broccoli, parcel, rollup) that were forks/inspirations from previous packages rather than upgrades. As of today, many companies use each of these runners because it fits their knowledge base, needs, etc.

In short, I would love to try berry as a standalone project that allows me to have yarn or npm live at the side, the same way yarn can be run in a project with package-lock.json and just be like “Hey, so, here’s the thing, I noticed you have package-lock.json, that might not be ideal”. Getting a taste of the new kick-ass features berry has in existing projects would be for my team and me the best path to use it from the get-go in new ones.

As a closing note, in particular, to @arcanis, take all these comments with a grain of salt. The JS community is extremely grateful for the work individuals like you and the @yarnpkg maintainers (and companies like @DataDog that sponsor this!), that it’s easy to get lost in the many “Well, I think that...” comments when a good chunk of developers sit on the side just looking to try the new toys. Projects like Babel got their fair share of hate on an upgrade (can’t remember which), that got their maintainers to feel pretty bad about work they did for (guess what) free. Do not let impostor syndrome kick in: you are doing awesome work here.

@devongovett
Copy link

@devongovett devongovett commented Jan 30, 2020

There is so much FUD being spread in this thread.

Forcing breaking changes to end users

Nothing is being forced. You need to explicitly upgrade. That's why it's a major version release.

if PnP is a superior solution it will eventually find its way to widespread adoption. Why the rush?

PnP has been available for over a year already with only a small amount of adoption. Enabling it by default is the next step. If not now, when?

Please ship berry as a standalone new project and put yarn in maintenance mode.

How is this any different? Yarn 2 is already opt-in, just like a separate project would be.

@jorgegonzalez
Copy link

@jorgegonzalez jorgegonzalez commented Jan 30, 2020

Of all the proposed solutions, releasing berry as a stand-alone project will, in my opinion, cause the most friction. Do we really want to our language community to have four fundamentally-different-yet-similar package managers? (npm/yarn/pnpm/berry)

We need less fragmentation in our ecosystem, not more.

@cevr
Copy link

@cevr cevr commented Jan 30, 2020

@devongovett I think the issue lies in the all or nothing approach of v2. It comes with many improvements that no one would find an issue with, yet a large portion of the community will have to downgrade because node_modules support is not ready yet in v2. A compromise would be to release v2 without PnP enabled by default, so as to not force people to stay on v1 and miss out on other improvements.

I don't know if the v2 improvements are possible without PnP, if not then that's another problem

@brentvatne
Copy link
Author

@brentvatne brentvatne commented Jan 30, 2020

@devongovett - a comparable example that comes to mind for me here is Node / Deno — what if once Deno has partial compatibility with Node ("It's unlikely we can make Node code work 100% of the time") it were to ship as the next major version of Node? I think this would create a massive split in the community between pre-Deno Node, like Python2/3. Deno is, in my mind, a technically superior solution — but that doesn't mean it should outright replace Node. There is room for both.

@jorgegonzalez - regardless of the outcome there will be friction here, the difference is whether it will be more clear for users to differentiate between yarn@1 and yarn@2, which behave completely differently despite having the same name, or between yarn and berry.

@devongovett
Copy link

@devongovett devongovett commented Jan 30, 2020

You don't have to upgrade on day one. You can choose to wait for stability in the ecosystem. Nothing is forcing you.

@mgcrea
Copy link

@mgcrea mgcrea commented Jan 30, 2020

Nothing is being forced. You need to explicitly upgrade.

A simple brew update; brew upgrade would pull the new major version, most people wouldn't even notice that they have upgraded to berry.

Do we really want to our language community to have four fundamentally-different-yet-similar package managers?

I'm fine with it, @jjperezaguinaga said it quite well, the JS ecosystem is built around package diversification, with forks and new projects popping up every other day, we're used to it.

Keep the same name and I bet you'll see people forking away yarn v1 as a new package manager. We will end up in the same state but the damage will be done.

@devongovett
Copy link

@devongovett devongovett commented Jan 30, 2020

@brentvatne

a comparable example that comes to mind for me here is Node / Deno

The difference there is that Node is actively maintained whereas Yarn 1 will not be.

@mgcrea

A simple brew update; brew upgrade

@arcanis already addressed this in a previous comment: #766 (comment). Yarn 1 will stay as the latest package on package managers like brew, with yarn set version to opt into Yarn 2 for a specific project rather than globally.

@brentvatne
Copy link
Author

@brentvatne brentvatne commented Jan 30, 2020

The difference there is that Node is actively maintained whereas Yarn 1 will not be.

Someone in this thread who has contributed to yarn in the past has already stepped up and offered to maintain it. I believe that others will get involved too.

@cwhitten
Copy link

@cwhitten cwhitten commented Jan 30, 2020

I tend to agree with @devongovett 's thinking, hence my suggestion earlier in the discussion to adopt a "next" metaphor to represent the state of Berry instead of Yarn 2. It is the direction where the tool is headed but important components are still being discussed and reasoned about.

@larixer
Copy link
Member

@larixer larixer commented Jan 30, 2020

Nothing is being forced. You need to explicitly upgrade.

A simple brew update; brew upgrade would pull the new major version, most people wouldn't even notice that they have upgraded to berry.

This comment contributes to spreading fears.

➜  ~ yarn --version
1.21.1
➜  ~ brew update
➜  ~ brew upgrade
➜  ~ yarn --version
1.21.1

brew will not upgrade yarn version to v2

This is exactly what @devongovett talks about.

This thread has too many fears and too few rational thoughts. There are plenty of options if pnp doesn't work for you from day one.

  1. Wait until pnp will be adopted by major packages
  2. Use yarn 1
  3. Use yarn 2 with node_modules plugin (experimental at the moment, but its state improves every day)
  4. Wait for more tools from v2 that will relieve compatibility issues that are being worked on right now
  5. Come join v2 team and help to improve v2 pain points!

@arslivinski
Copy link

@arslivinski arslivinski commented Jan 30, 2020

Please correct me if I'm wrong, but the node_modules plugin should provide the compatibility layer to whom is not able to upgrade to PnP, right? If so, IMHO, all we need to do is wait (and help, if able) until it is stable, then change the latest tag to v2, so we can have an easy migration path.

Library maintainers can also document this on their README, pointing that if you are using Yarn V2 you need this plugin.

@jorgegonzalez
Copy link

@jorgegonzalez jorgegonzalez commented Jan 30, 2020

the JS ecosystem is built around package diversification, with forks and new projects popping up every other day, we're used to it.

https://en.m.wikipedia.org/wiki/Stockholm_syndrome

@yarnpkg yarnpkg locked as too heated and limited conversation to collaborators Jan 30, 2020
@arcanis
Copy link
Member

@arcanis arcanis commented Jan 30, 2020

Thanks everyone, I think all arguments have been made. I'll lock this issue for tonight, and will come back tomorrow to share a longer retrospective and detail the plans our team discussed.

@cpojer
Copy link

@cpojer cpojer commented Jan 30, 2020

Edit by @arcanis: When I lock a thread as "too heated", I expect discussions to stop and people to go home - people with repository write-access included. One-sided arguments do nothing to "stop the flamewar". Thanks for understanding.


Yarn 1 will stay as the latest package on package managers like brew, with yarn set version to opt into Yarn 2 for a specific project rather than globally.

If we do this long term, then PnP will still not get significant adoption. PnP has been an opt-in for Yarn 1 for the past 18 months and we've seen that people do not opt-in to stricter guarantees. An opt-in to Berry and PnP is effectively the same except the opportunity cost is higher for a marginal benefit to use additional features.

If we only do this short term as the announcement indicates, then the concerns above are valid and people are justifiably worried about being forced to migrate. This is a real concern that can't be brushed off with "don't use it". Many people here would love to just not use it, and they want guarantees that they won't be forced to today or in the future, or be able to use a drop-in replacement by running a single upgrade command.

There is so much FUD being spread in this thread.

Many of the people in this issue like @brentvatne and @janicduplessis have been involved since the very beginning of Yarn. The React Native community embraced Yarn on day one because of the huge install time wins. I have been involved with Yarn since before it was open sourced, funded the work on PnP at Facebook, monitored its deployment, helped fix problems and eventually removed it. I have seen the Berry codebase evolve from a prototype almost three years ago to what is in this repo today.

What we all presented above was entirely facts based from our heavy involvement with Yarn and early adoption of PnP. I do not believe you can claim that we are spreading misinformation or invoking fear to disadvantage this project. We are here because we care about the future of this community.

@arcanis
Copy link
Member

@arcanis arcanis commented Jan 31, 2020

Hi everyone. I didn't want to answer this thread immediatly because the release is still young and, quite frankly, I didn't want to take decisions over the course of a week-end. @brentvatne was right to start the discussion, but that didn't mean we needed to act in a hurry. The release itself took weeks to plan (the blog post itself got reviewed by at least four people from the community, plus our own team), follow ups should be thought through with similar care.

This out of the way, our team (@yarnpkg/berry) discussed and we agreed on the following points. I believe most of you will be satisfied, but just as importantly we are satisfied with it and we think it's the best way for the project to move forward. Regardless of how you feel about it, I'd ask you to show us the respect and empathy we deserve.

Yarn 2.x and forward - our goals

Javascript has been a wild ride for the past twenty-or-so years. From browsers to servers, from desktops to smartphones, from a handful of developers building cute stuff to word-class applications with billions of users.

I myself remember my first open-source experiences - they were in ... 2006? I was writing tutorials for a phpBB forum hosting platform, writing small scripts to mark topics as closed and stuff. Distribution was funny: we had to tell users which files to modify, and where to put the code. It often didn't go very well...

Anyway, years later appeared npm, followed by Yarn, and the ecosystem took a very different turn. I was a bit skeptical when I first learned about Yarn, btw - speed was nice, but it failed to install the app I was working on at the time (it eventually did - go figure, it’s almost like projects improve given time 😉). Then in 2017 I started working on it, and something clicked. I was still seeing broken installs that we couldn't control. My coworkers still had to rm -rf their modules regularly. I had to context switch during installs. We hit file casing issues, file permission issues, filesystem i/o issues, we had to write layers upon layers to make it do what we wanted, ...

All this is what we are set to change with Yarn 2.x. We want installs to be fast (a typical install of our repository is 5s). We want installs to be light (a stock CRA app is now 60MB, vs 230MB before). We want installs to be fully deterministic (flat cache with full integrity checks, no chance for different behaviours) and stable (boundary checks, no more delta between prod and dev).

Today, what we need the most isn't code. It isn't users. And it most definitely isn't drama or trolls or jokes about design decisions. It's support. We've been working incredibly hard to give the JS ecosystem a chance to get rid of problems that plagued it for a decade. Our work is literally worth millions in computer resources and developer efficiency, and for that we'll get little to no rewards (guess where is Yarn on this chart).

So we now need you to care enough to make a change by yourself. If you don't, fine - we'll spend our resources elsewhere helping those who do.

About the initial release strategy

I did a mistake by not anticipating how our words would be interpreted. From my very pragmatical viewpoint, I didn't expect much to change due to the release itself: the latest tag stayed the same, the 1.x releases are available through the same URLs as before, the 1.x repository had already been in effective maintenance mode for a year, the 2.x repository had already received numerous RCs during the past months ... practically speaking, the only real change between then and now was the website homepage, which pointed to a refreshed documentation.

This reasoning unfortunately didn't account for the weight that words like “legacy“ or “deprecated“ would bear on some of our users. What we meant to express was that the 1.x branch was from our perspective considered “feature-complete”, but some instead interpreted it as an “upgrade now“ sign surrounded by flashing lights. Words have weight, and maybe choosing our words more carefully would have helped.

Is React Native abandoned by Yarn? /
Is React Native compatible with Yarn 2.x?

No to the first question, and yes to the second one. To suggest otherwise is frankly insulting to the amazing efforts that people like @larixer have put into ensuring that projects like React Native could still upgrade to the Yarn 2.x releases. If you're wondering why some might see FUD, consider that this might be the reason (together with the whole “forced“ narrative that maybe we contributed to with our poor choice of word, more on that later).

Yarn already supports an early version of our node_modules linker which we intend to support and keep improving over time. This linker will be the perfect way for 1.x projects to use the 2.x releases without having to deal with the PnP compatibility. If at some point you think the opportunity cost is low enough, or the value proposition high enough, then you'll be able to consider implementing the PnP compatibility at your own pace.

Let me rephrase it for good measure: if you use 2.x and don't like PnP, create a .yarnrc.yml file and add the following line. You're set!

nodeLinker: node-modules

Again, whether you think this should be the default or not is largely irrelevant to us; not being the default doesn't mean it doesn't exist or that we don't invest in it. Our defaults are what our team suggests - it's then up to you to pick the tools you think are the best for the job.

Will an RFC thread be opened?

This very thread has been a Request for Comment. Discussions have been going strong for almost a week, gathering contributions from the core team and users alike (now almost 40 people). Things are starting to go in circle, so now it’s up to our team to take a decision regarding how we want to handle the project. We will not open another thread on another repository just for the sake of rebooting the discussion.

Should this be named differently?

This is Yarn, it will stay Yarn. More than an implementation, we see this project as a focus. Yarn was first released focused on providing a “fast, reliable, and secure“ package manager, and this is the foundation upon which we will keep building.

Will installing the `yarn` package globally break 1.x projects? /
Do I need to upgrade now?

The yarn package will remain frozen on 1.x, and will only receive security updates (from our team; other contributors are welcome to merge functional PRs if they wish to). Similarly, we've asked the Docker image to stay on 1.x for the global binary. Generally speaking, we’ll recommend the global Yarn to remain 1.x.

Our team will continue releasing the 2.x by recommending per-project installations (also known as yarn set version, or yarn policies set-version). Part of our work will consist in improving this workflow to turn it into the preferred way to install Yarn.

Put clearly: applications that work now will keep their behaviour. There will be no “oh crap” moment caused by an unexpected upgrade on brew install yarn. Which brings me to my next point.

Is Yarn 1.x legacy? Deprecated?

No. We are scaling back the word we use. Earlier in the development we used to call Yarn 1.x “Classic“. I believe this is a good word we should have kept using for the release.

We will henceforth start referring to Yarn 1.x as “Classic“ in our materials. Yarn Classic will remain a tool that you can recommend to your users (whether external or internal, thinking of you @paularmstrong) free of the worry of being seen as recommending a legacy or deprecated tool. I mentioned it before: we see Classic as being feature-complete, so you can expect it to always be as good as it currently is. Over time we’ll keep making the 2.x branch more and more compelling, but it’ll be an organic process that we don’t intend to rush.

The Yarn Classic documentation will be moved once again to be hosted at classic.yarnpkg.com. We hope that this change will better convey our intent and will be in line with what people like @cwhitten suggested.

And no, I don't actually play World of Warcraft.

Will PnP be the default?

First, let me be clear: this is ours to define. You are free to have your opinions and to share them respectfully (as some did here), but ultimately it's up to the people spending their nights on Yarn to decide the direction they wish to give to their work.

We decided that Yarn 2.1 would implement what we call a PnP-loose mode, inspired in idea by the loose mode in Babel. In PnP-loose mode, which will be the default, Yarn will print warnings should a package rely on undefined behaviours (instead of throwing flat-out exceptions). We will strive to do something sensible (usually resolving to a compatible version of the package), but the warnings won't let you ignore the underlying issues.

Note that PnP-loose installs will be slower and will generate heavier files than PnP-strict installs (which will be available as well although not as the default install they currently are).

Is Yarn a community project?

We see ourselves as a community of contributors. We work together towards similar goals, and we're happy to welcome into our team anyone willing to contribute in good faith - be it by doing support, development, or documentation.

This doesn't mean, however, that we'll let a Twitter community or any one company or project dictate our roadmap. Yarn doesn't generate any monetary contribution for any of us, and our incentives lie in making a software we would recommend rather than a software built around user acquisition.

Whether you prefer to see that as a positive or a negative is now up to you.

To summarize:

  • Yarn 1.x isn't legacy
  • Yarn 1.x will keep being the preferred global binary
  • Yarn 2.x will be distributed through yarn set version
  • Yarn 2.x will use a PnP-loose mode which will warn on danger
  • Yarn 2.x supports node_modules installs if you prefer that
  • Decisions regarding the project are up to our active contributors

@arcanis arcanis closed this as completed Jan 31, 2020
@Haroenv Haroenv changed the title [Feature] Release yarn v2 under a different name [Feature] Discussion around release strategy of Yarn v2 Jan 31, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement
Projects
None yet
Development

No branches or pull requests