Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Singleton Packages RFC. #23

Open
wants to merge 3 commits into
base: latest
from

Conversation

@usergenic
Copy link

commented Dec 3, 2018

Suggested feature for support of a singleton property in package.json to guard against installation of multiple versions/instances of a package the package tree.

Anticipates, but does not define course for resolving version conflicts among singleton packages. (That would be a separate RFC.)

usergenic added some commits Dec 3, 2018

@arcanis

This comment has been minimized.

Copy link

commented Dec 12, 2018

This is something that we (Yarn) are interested in supporting as well. My comments:

  • One question I have is regarding the expected efficiency. Finding the right set of packages that satisfy multiple constraints is I think NP-complete, so I'd prefer any spec to mention that the package managers are allowed to ask for an explicit version lock even if a theoretical solution was possible. I know that you've said the resolution conflict was to be defined in a different RFC, but this particular point affects the semantic of the field, so I think it should be answered here.

  • I'm a bit concerned that this change will prompt everyone to add the singleton field to all their packages because "hey the package manager will optimize it better this way!", which will cause a lot of conflicts issues and potential pain that will last forever. It should be made crystal clear that this feature should only be used if there is no way for the package to work properly without it and that using it for optimization is absolutely the wrong thing to do.

  • While it doesn't affect npm directly since they don't implement this feature yet, this also has ramifications regarding the Yarn workspaces. Is a singleton unique in a single workspace, or across the whole worktree? I need to think a bit more before having a recommendation on this topic, but I'd appreciate if you could think about it as well.

@mantoni

This comment has been minimized.

Copy link

commented Dec 12, 2018

There has been a related discussion some years ago.

@dbatiste

This comment has been minimized.

Copy link

commented Dec 12, 2018

@arcanis : your last point reminded me of this discussion on Scoped Custom Element Registries, which has some support. If such scoped element registry becomes possible, the scenario of a component that defines a custom registry scope and a dependency on a component that, which today, must only be "defined" once, then an approach that makes these components unique to the whole dependency tree could be problematic.

@bmeck

This comment has been minimized.

Copy link

commented Dec 12, 2018

Another discussion of this topic happened as well. It seems there are multiple of these.

@arcanis

This comment has been minimized.

Copy link

commented Dec 12, 2018

I remember - the suggestion to use peer dependencies to define singletons looked interesting (but unfortunately relies on the ecosystem doing the right thing, which is a assumption that's been proven wrong multiple times). I wonder if the preferPeer you mentioned would help.

@usergenic

This comment has been minimized.

Copy link
Author

commented Dec 13, 2018

Thanks for feedback and questions, @arcanis.

One question I have is regarding the expected efficiency. Finding the right set of packages that satisfy multiple constraints is I think NP-complete, so I'd prefer any spec to mention that the package managers are allowed to ask for an explicit version lock even if a theoretical solution was possible. I know that you've said the resolution conflict was to be defined in a different RFC, but this particular point affects the semantic of the field, so I think it should be answered here.

I am assuming that package managers already produce an effective union of semver constraints such that transitive dependencies on a@^1.2.0 and a@=1.3.0 would resolve to a@1.3.0 even if a@1.4.0 were available? Is this not correct? I wasn't operating under the assumption that singletons would add a deeper behavior to these calculations.

I'm working on separate RFC to encourage npm to essentially implement a fully compatible resolutions property to Yarn's including its' Selective Version Resolutions, since this provides the necessary top-level conflict resolution specification.

I'm a bit concerned that this change will prompt everyone to add the singleton field to all their packages because "hey the package manager will optimize it better this way!", which will cause a lot of conflicts issues and potential pain that will last forever. It should be made crystal clear that this feature should only be used if there is no way for the package to work properly without it and that using it for optimization is absolutely the wrong thing to do.

I agree. Optimized package semver union resolutioning, if not a thing, should be a thing that is separate from this thing.

While it doesn't affect npm directly since they don't implement this feature yet, this also has ramifications regarding the Yarn workspaces. Is a singleton unique in a single workspace, or across the whole worktree? I need to think a bit more before having a recommendation on this topic, but I'd appreciate if you could think about it as well.

I am going to have to get more familiar with Yarn workspaces. I'm quite familiar with Lerna but also haven't used it in conjunction with workspaces. Will explore.

@steveklabnik

This comment has been minimized.

Copy link

commented Dec 13, 2018

Just to chip in briefly here: Rust and Cargo have this distinction; https://doc.rust-lang.org/stable/cargo/reference/build-scripts.html#the-links-manifest-key

It works well, but there's one big issue: this means that major version bumps of these kinds of libraries split the ecosystem. When those are big, foundational libraries, it can lead to a lot of pain. In Rust, this has mostly manifested as the libc package. This is transitively used by the vast majority of the ecosystem, and can only be included once. This means that the upgrade from 0.1 to 0.2 was a really big pain. You could only use packages that depended on identical versions to each other.

The JavaScript ecosystem may not have situations like this, but I figured it was worth mentioning. Happy to answer any specific questions about it. Good luck!

@ljharb

This comment has been minimized.

Copy link

commented Dec 13, 2018

The JS ecosystem definitely already does - React, eslint, babel, etc - and it's generally handled by every participant in that "singleton" ecosystem adding a peer dep on it. (not that it's the best solution)

@daKmoR

This comment has been minimized.

Copy link

commented Jan 2, 2019

It's a new year 🎉
So anything we can do to help this along?

@zkat zkat referenced this pull request Jan 10, 2019
@daKmoR

This comment has been minimized.

Copy link

commented Jan 21, 2019

a friendly reminder if there is anything we can do to help?

@zkat

This comment has been minimized.

Copy link
Contributor

commented Jan 24, 2019

Hey y'all! Thanks a lot for submitting this!

Overall, we think this is a good idea. This feature does require us to land #27's implementation first, imo. Our plan right now is to land that implementation, see how it feels, and then see about this RFC and implementing it.

Historically, npm has been a package manager that worked pretty hard to guarantee that you wouldn't run into dependency hell[1], and a feature like this one is a major step back into that hole (after we stopped installing peerDependencies by itself, which sort-of-kind-of pushed out the issue). If we're gonna introduce dependency hell into npm, I really want to make sure we have a good way to resolve conflicts.

I appreciate the discussion that's been going on in this thread, and I hope it keeps going while we take the necessary steps to accept it. Thanks everyone for adding the context, and thanks again to @usergenic for taking the time to write and submit it. Hang in there, we'll get this in, and I think this looks a lot like what we'll end up accepting in the end, if not exactly like it!

[1]: In npm parlance, dependency hell is when you have two different packages that require the same package at incompatible versions, a problem that's universal to package managers and that npm resolves through dependency nesting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.