peerDependencies (plugins) #1400

Closed
isaacs opened this Issue Sep 14, 2011 · 54 comments

Comments

Projects
None yet
@isaacs
Member

isaacs commented Sep 14, 2011

New package.json field, "peerDependencies"

Just like "dependencies" hash, but:

  1. Each entry must be installed in the family (at the same level or higher) as the package. (If the package is the top target, and not global, then install as normal deps, maybe, just for testing/development purposes?)
  2. If not found, then fail, unless --force flag is set.
  3. Never ever include in the package when publishing.

This enables express middleware and other plugin-type things to specify that they depend on a certain version of express being in-play, but not bundled underneath them.

Open questions:

Should peerDeps be installed by default if missing? Leaning towards no, since this brings up all kinds of cleanup issues on uninstallation.

@dfellis

This comment has been minimized.

Show comment
Hide comment
@dfellis

dfellis Sep 14, 2011

What about taking a page from apt-get and print something like

express-route-util RECOMMENDS the following packages:
    express

and then have something like --recommended to auto-install these packages, and perhaps display a warning message like:

The following RECOMMENDED applications must be uninstalled manually:
    express

dfellis commented Sep 14, 2011

What about taking a page from apt-get and print something like

express-route-util RECOMMENDS the following packages:
    express

and then have something like --recommended to auto-install these packages, and perhaps display a warning message like:

The following RECOMMENDED applications must be uninstalled manually:
    express
@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Sep 14, 2011

Member

Yeah, a flag to install them automatically would be good. Then at least it'll feel like a separate step.

However, I'd really like to make the installation actually fail if they're missing, and require the use of -f, since it probably means that the plugin or whatever actually won't work.

Member

isaacs commented Sep 14, 2011

Yeah, a flag to install them automatically would be good. Then at least it'll feel like a separate step.

However, I'd really like to make the installation actually fail if they're missing, and require the use of -f, since it probably means that the plugin or whatever actually won't work.

@dfellis

This comment has been minimized.

Show comment
Hide comment
@dfellis

dfellis Sep 14, 2011

That makes sense; express-route-util isn't actually recommending express; it really is depending on it.

A RECOMMENDS type of thing would be the other way around; a package stating that it can optionally use the library if present (by wrapping require() in a try-catch block, for instance).

dfellis commented Sep 14, 2011

That makes sense; express-route-util isn't actually recommending express; it really is depending on it.

A RECOMMENDS type of thing would be the other way around; a package stating that it can optionally use the library if present (by wrapping require() in a try-catch block, for instance).

@dfellis

This comment has been minimized.

Show comment
Hide comment
@dfellis

dfellis Sep 14, 2011

Perhaps --peers would be the right flag name?

dfellis commented Sep 14, 2011

Perhaps --peers would be the right flag name?

@indexzero

This comment has been minimized.

Show comment
Hide comment
@indexzero

indexzero Sep 17, 2011

Contributor

Additional discussion is at #930

Contributor

indexzero commented Sep 17, 2011

Additional discussion is at #930

@indexzero

This comment has been minimized.

Show comment
Hide comment
@indexzero

indexzero Nov 28, 2011

Contributor

@isaacs Any progress here? What can I do to help?

Contributor

indexzero commented Nov 28, 2011

@isaacs Any progress here? What can I do to help?

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic May 22, 2012

Member

Ping.

We've run into this while working on plugins for Chai. The plugins need to be able to specify that they will only work with (e.g.) Chai >= 1.0.1.

Member

domenic commented May 22, 2012

Ping.

We've run into this while working on plugins for Chai. The plugins need to be able to specify that they will only work with (e.g.) Chai >= 1.0.1.

@drewfish

This comment has been minimized.

Show comment
Hide comment
@drewfish

drewfish May 25, 2012

Ditto. We're trying to get plugins for mojito going as well.

The only thought we'd had so far was to have the plugin specify the host (i.e., express, mojito) as an"engines", as in

"engines": {
    "mojito": "0.3.x"
}

Since (I believe) NPM doesn't check the engines (like "dependencies") the host would need to do that itself. Not a clean approach :(

Ditto. We're trying to get plugins for mojito going as well.

The only thought we'd had so far was to have the plugin specify the host (i.e., express, mojito) as an"engines", as in

"engines": {
    "mojito": "0.3.x"
}

Since (I believe) NPM doesn't check the engines (like "dependencies") the host would need to do that itself. Not a clean approach :(

@indexzero

This comment has been minimized.

Show comment
Hide comment
@indexzero

indexzero May 25, 2012

Contributor

@drewfish That doesn't work. See the discussion on #930. @isaacs correct me if I'm wrong, but we've basically decided on the package.json API, which for your case @drewfish would be:

  {
    "peerDependencies": {
      "mojito": "0.3.x"
    }
  }
Contributor

indexzero commented May 25, 2012

@drewfish That doesn't work. See the discussion on #930. @isaacs correct me if I'm wrong, but we've basically decided on the package.json API, which for your case @drewfish would be:

  {
    "peerDependencies": {
      "mojito": "0.3.x"
    }
  }
@drewfish

This comment has been minimized.

Show comment
Hide comment
@drewfish

drewfish May 25, 2012

Sorry, yeah, just read #930.

As for discoverability, mojito right now walks the app's node_modules/ directory looking for modules that are extensions to mojito. (We never thought that this was a great solution, but it works for now.) If npm had some kind of API that a host could use to list its plugins, then the host doesn't care where those plugins are installed.

Sorry, yeah, just read #930.

As for discoverability, mojito right now walks the app's node_modules/ directory looking for modules that are extensions to mojito. (We never thought that this was a great solution, but it works for now.) If npm had some kind of API that a host could use to list its plugins, then the host doesn't care where those plugins are installed.

@indexzero

This comment has been minimized.

Show comment
Hide comment
@indexzero

indexzero May 25, 2012

Contributor

@drewfish Cool. Anyone on the mojito team up to taking on implementing this?

Contributor

indexzero commented May 25, 2012

@drewfish Cool. Anyone on the mojito team up to taking on implementing this?

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic May 25, 2012

Member

@indexzero, @isaacs I'd be happy to give this a shot over the weekend. I'd implement the proposal from the OP without the parenthetical for point 1 and without install-by-default-if-missing. Sound good?

Member

domenic commented May 25, 2012

@indexzero, @isaacs I'd be happy to give this a shot over the weekend. I'd implement the proposal from the OP without the parenthetical for point 1 and without install-by-default-if-missing. Sound good?

@indexzero

This comment has been minimized.

Show comment
Hide comment
@indexzero

indexzero May 25, 2012

Contributor

@domenic Sounds good. A couple of things:

  1. The parenthetical from point one is actually much simpler than it sounds. Just use semver.satisfies instead of semver.gt.
  2. install-by-default-if-missing is critical for anything we merge in, but if it gets implemented later that's not a problem
Contributor

indexzero commented May 25, 2012

@domenic Sounds good. A couple of things:

  1. The parenthetical from point one is actually much simpler than it sounds. Just use semver.satisfies instead of semver.gt.
  2. install-by-default-if-missing is critical for anything we merge in, but if it gets implemented later that's not a problem
@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic May 25, 2012

Member

@indexzero by the parenthetical I meant

(If the package is the top target, and not global, then install as normal deps, maybe, just for testing/development purposes?)

Is that semver related?

Member

domenic commented May 25, 2012

@indexzero by the parenthetical I meant

(If the package is the top target, and not global, then install as normal deps, maybe, just for testing/development purposes?)

Is that semver related?

@indexzero

This comment has been minimized.

Show comment
Hide comment
@indexzero

indexzero May 25, 2012

Contributor

@domenic No. I misunderstood. Look forward to seeing what you put together :-D

Contributor

indexzero commented May 25, 2012

@domenic No. I misunderstood. Look forward to seeing what you put together :-D

@aseemk

This comment has been minimized.

Show comment
Hide comment
@aseemk

aseemk Jun 6, 2012

Sweet! Have been wondering about this too. (And like @drewfish, used engines to specify it informally.)

aseemk commented Jun 6, 2012

Sweet! Have been wondering about this too. (And like @drewfish, used engines to specify it informally.)

@vojtajina

This comment has been minimized.

Show comment
Hide comment
@vojtajina

vojtajina Aug 21, 2012

👍 Totally support this effort, it would be great for plugins.

👍 Totally support this effort, it would be great for plugins.

@carlos8f

This comment has been minimized.

Show comment
Hide comment
@carlos8f

carlos8f Aug 31, 2012

I've needed this feature for a long time. Going to give it a shot.

I've needed this feature for a long time. Going to give it a shot.

@carlos8f

This comment has been minimized.

Show comment
Hide comment
@carlos8f

carlos8f Sep 1, 2012

Regarding point 3: does that really need addressing? Isn't it required to list things in bundledDependencies to include them when publishing? If so, wouldn't you have to include the package in both peer and bundled deps to have that problem?

Also re: install by default if missing, I would be wary. Mostly because plugins often share the same state of the parent by require()ing something in the parent's family, and if there is a different/local copy of that module in the plugin's family, it breaks the state sharing.

carlos8f commented Sep 1, 2012

Regarding point 3: does that really need addressing? Isn't it required to list things in bundledDependencies to include them when publishing? If so, wouldn't you have to include the package in both peer and bundled deps to have that problem?

Also re: install by default if missing, I would be wary. Mostly because plugins often share the same state of the parent by require()ing something in the parent's family, and if there is a different/local copy of that module in the plugin's family, it breaks the state sharing.

@marcelklehr marcelklehr referenced this issue in ether/etherpad-lite Sep 29, 2012

Open

Make node modules available for plugins #823

@Raynos Raynos referenced this issue in Level/levelup Nov 20, 2012

Closed

hooks #29

@indexzero

This comment has been minimized.

Show comment
Hide comment
@indexzero

indexzero Nov 29, 2012

Contributor

@domenic @isaacs Did this ever get a full-on PR somewhere?

Contributor

indexzero commented Nov 29, 2012

@domenic @isaacs Did this ever get a full-on PR somewhere?

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Nov 29, 2012

Member

@indexzero no, it's waiting on isaacs/read-installed#6, which has a review comment I need to take care of.

You can see the installation code here, but it's not really ready to be a proper PR until I have a version of read-installed I can work with. It probably also needs a new test to take care of the review comment.

Member

domenic commented Nov 29, 2012

@indexzero no, it's waiting on isaacs/read-installed#6, which has a review comment I need to take care of.

You can see the installation code here, but it's not really ready to be a proper PR until I have a version of read-installed I can work with. It probably also needs a new test to take care of the review comment.

@vojtajina vojtajina referenced this issue in karma-runner/karma Dec 10, 2012

Closed

Reorganize Testacular #158

@mfncooper

This comment has been minimized.

Show comment
Hide comment
@mfncooper

mfncooper Dec 15, 2012

Contributor

Once we have peerDependencies support in npm, the next step would be the host auto-discovering its peer-installed plugins. The host could certainly be told, by the app, which plugins to use, but it seems like it would be nice to be able to (perhaps optionally) auto-discover them.

Let's say I have an application that uses some-framework and some-cool-framework-plugin, and the latter declares a peer dependency on the former. What we need is some-framework to walk the packages that are its peers (i.e. in the same node_modules folder), and discover those that declare a peer dependency on it.

Unfortunately, it doesn't seem that easy to robustly locate the package's containing node_modules folder within the app. For one thing, the package may have been symlinked in there, using npm link, so it's not simply the parent folder. It'd be possible for the framework to walk module.paths, looking for itself, but that seems messy. And readInstalled() still needs to be told where the root is.

Thoughts? Apologies if this seems a tad off-topic, but it seems to me that it's part of a complete "we have plugin support" story, so it'd be good to think it through before we commit to a peer dependencies mechanism for npm.

Contributor

mfncooper commented Dec 15, 2012

Once we have peerDependencies support in npm, the next step would be the host auto-discovering its peer-installed plugins. The host could certainly be told, by the app, which plugins to use, but it seems like it would be nice to be able to (perhaps optionally) auto-discover them.

Let's say I have an application that uses some-framework and some-cool-framework-plugin, and the latter declares a peer dependency on the former. What we need is some-framework to walk the packages that are its peers (i.e. in the same node_modules folder), and discover those that declare a peer dependency on it.

Unfortunately, it doesn't seem that easy to robustly locate the package's containing node_modules folder within the app. For one thing, the package may have been symlinked in there, using npm link, so it's not simply the parent folder. It'd be possible for the framework to walk module.paths, looking for itself, but that seems messy. And readInstalled() still needs to be told where the root is.

Thoughts? Apologies if this seems a tad off-topic, but it seems to me that it's part of a complete "we have plugin support" story, so it'd be good to think it through before we commit to a peer dependencies mechanism for npm.

@rvagg

This comment has been minimized.

Show comment
Hide comment
@rvagg

rvagg Dec 16, 2012

Contributor

@mfncooper this is something we've considered for Ender. I'd like to be able to scan all accessible modules and see if any are defined (somehow, something in package.json I suppose) as Ender-compatible plugins to extend the core CLI functionality. Having to discover the current node_modules directory and perhaps the global one(s) is probably not the most robust method and it'd be neat to have a way to get npm to help us out.

I think Grunt is even more of a candidate for this kind of thing at the moment though.

Contributor

rvagg commented Dec 16, 2012

@mfncooper this is something we've considered for Ender. I'd like to be able to scan all accessible modules and see if any are defined (somehow, something in package.json I suppose) as Ender-compatible plugins to extend the core CLI functionality. Having to discover the current node_modules directory and perhaps the global one(s) is probably not the most robust method and it'd be neat to have a way to get npm to help us out.

I think Grunt is even more of a candidate for this kind of thing at the moment though.

@balupton

This comment has been minimized.

Show comment
Hide comment
@balupton

balupton Jan 2, 2013

Sweet. This will be really useful for abstracting out components out of https://github.com/bevry/docpad

Currently we have our own plugin loader that checks the plugin's package.json file and scans the engines and verifies them ourselves. https://github.com/bevry/docpad/blob/master/src/lib/plugin-loader.coffee#L133-L136 and https://github.com/docpad/docpad-plugin-livereload/blob/master/package.json#L31-L34 - so what @drewfish suggested.

However, having this peerDependencies would be amazing. Our use case would be for DocPad Plugins, DocPad Modules, as well as Backbone.js extensions.

balupton commented Jan 2, 2013

Sweet. This will be really useful for abstracting out components out of https://github.com/bevry/docpad

Currently we have our own plugin loader that checks the plugin's package.json file and scans the engines and verifies them ourselves. https://github.com/bevry/docpad/blob/master/src/lib/plugin-loader.coffee#L133-L136 and https://github.com/docpad/docpad-plugin-livereload/blob/master/package.json#L31-L34 - so what @drewfish suggested.

However, having this peerDependencies would be amazing. Our use case would be for DocPad Plugins, DocPad Modules, as well as Backbone.js extensions.

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Jan 9, 2013

Member

Landed in 1.2.0

Member

isaacs commented Jan 9, 2013

Landed in 1.2.0

@isaacs isaacs closed this Jan 9, 2013

@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Jan 10, 2013

Lacking docs, should likely be here: https://npmjs.org/doc/json.html

Is there something to look at until that is fixed? The original ticket description is hardly useful.

Lacking docs, should likely be here: https://npmjs.org/doc/json.html

Is there something to look at until that is fixed? The original ticket description is hardly useful.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 10, 2013

Member

@jzaefferer blog post coming, this weekend at the latest. As well as some followup PRs with docs and niceties like --save-peer. In the meantime, here's a quick example that may or may not be helpful, depending on how familiar you are with the problem.


grunt-contrib-stylus@0.3.1 is a plugin to grunt. In particular, "plugin" means that when the user plugs it into their grunt configuration, it uses certain grunt features and exposes grunt tasks.

However, grunt-contrib-stylus@0.3.1 is compatible with grunt@0.4.0rc4, but not with grunt@0.4.0rc5. That is, when the user plugs it in, it uses grunt features that only exist in grunt@0.4.0rc4, and were removed in grunt@0.4.0rc5. Argh, that's bad! Maybe we should specify grunt@0.4.0rc4 as a dependency?

But that's not quite right, for two reasons. One, the code for grunt-contrib-stylus never actually does a require("grunt"), so there's no point in depending on grunt. Two, if the user has grunt@0.4.0rc5 installed, then installs a hypothetical grunt-contrib-stylus that depends on grunt@0.4.0rc4, he ends up with a node_modules directory looking like:

grunt@0.4.0rc5
grunt-contrib-stylus@hypotheticalVersion
|- grunt@0.4.0rc4

which isn't what he wanted at all!

What the user really wanted was to be told "hey, I see you have grunt@0.4.0rc5, but this thing you're installing peer depends on grunt@0.4.0rc4. This isn't gonna work!" We accomplish this by saying that grunt@0.4.0rc4 is a peer dependency of grunt-contrib-stylus@0.3.1, i.e. adding an entry to the latter's peerDependencies hash in package.json.

(Note that no grunt plugins I know of use the peerDependencies field yet. Expect this to change! Starting with grunt-contrib-stylus, the bastard that bit me yesterday at work -_-)

Member

domenic commented Jan 10, 2013

@jzaefferer blog post coming, this weekend at the latest. As well as some followup PRs with docs and niceties like --save-peer. In the meantime, here's a quick example that may or may not be helpful, depending on how familiar you are with the problem.


grunt-contrib-stylus@0.3.1 is a plugin to grunt. In particular, "plugin" means that when the user plugs it into their grunt configuration, it uses certain grunt features and exposes grunt tasks.

However, grunt-contrib-stylus@0.3.1 is compatible with grunt@0.4.0rc4, but not with grunt@0.4.0rc5. That is, when the user plugs it in, it uses grunt features that only exist in grunt@0.4.0rc4, and were removed in grunt@0.4.0rc5. Argh, that's bad! Maybe we should specify grunt@0.4.0rc4 as a dependency?

But that's not quite right, for two reasons. One, the code for grunt-contrib-stylus never actually does a require("grunt"), so there's no point in depending on grunt. Two, if the user has grunt@0.4.0rc5 installed, then installs a hypothetical grunt-contrib-stylus that depends on grunt@0.4.0rc4, he ends up with a node_modules directory looking like:

grunt@0.4.0rc5
grunt-contrib-stylus@hypotheticalVersion
|- grunt@0.4.0rc4

which isn't what he wanted at all!

What the user really wanted was to be told "hey, I see you have grunt@0.4.0rc5, but this thing you're installing peer depends on grunt@0.4.0rc4. This isn't gonna work!" We accomplish this by saying that grunt@0.4.0rc4 is a peer dependency of grunt-contrib-stylus@0.3.1, i.e. adding an entry to the latter's peerDependencies hash in package.json.

(Note that no grunt plugins I know of use the peerDependencies field yet. Expect this to change! Starting with grunt-contrib-stylus, the bastard that bit me yesterday at work -_-)

@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Jan 10, 2013

Thanks @domenic, that helps. Since I maintain a few grunt plugins myself, I'm gonna start using that.

Recently I actually moved grunt from dependencies to devDependencies, since I also use grunt for testing the grunt plugin. Would I move it now from devDependencies to peerDependencies? Or do I need both?

Thanks @domenic, that helps. Since I maintain a few grunt plugins myself, I'm gonna start using that.

Recently I actually moved grunt from dependencies to devDependencies, since I also use grunt for testing the grunt plugin. Would I move it now from devDependencies to peerDependencies? Or do I need both?

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 10, 2013

Member

Recently I actually moved grunt from dependencies to devDependencies, since I also use grunt for testing the grunt plugin. Would I move it now from devDependencies to peerDependencies? Or do I need both?

I'd say both. peerDependencies don't get installed in your node_modules folder, only in your parents'. Good question!

Member

domenic commented Jan 10, 2013

Recently I actually moved grunt from dependencies to devDependencies, since I also use grunt for testing the grunt plugin. Would I move it now from devDependencies to peerDependencies? Or do I need both?

I'd say both. peerDependencies don't get installed in your node_modules folder, only in your parents'. Good question!

@naholyr

This comment has been minimized.

Show comment
Hide comment
@naholyr

naholyr Jan 10, 2013

That's two different concepts:

  • The third-party libs I need to test and/or develop
  • The third-party libs I'm supposed to work with, like "side by side"

If a lib has both roles, it definitely should be in both lists ;)

naholyr commented Jan 10, 2013

That's two different concepts:

  • The third-party libs I need to test and/or develop
  • The third-party libs I'm supposed to work with, like "side by side"

If a lib has both roles, it definitely should be in both lists ;)

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 10, 2013

Member

@naholyr yes, that's a great way of putting it.

In general to everyone out there, keep the comments/questions coming---it helps me figure out what aspects need the most explanation, and the best ways to explain them.

Member

domenic commented Jan 10, 2013

@naholyr yes, that's a great way of putting it.

In general to everyone out there, keep the comments/questions coming---it helps me figure out what aspects need the most explanation, and the best ways to explain them.

@ForbesLindesay

This comment has been minimized.

Show comment
Hide comment
@ForbesLindesay

ForbesLindesay Jan 10, 2013

Member

Can a module have peerDependencies that are optional.

E.g. Both QEJS and consolidate.js work fine on their own, but if you install both then you can us consolidate.js's API to interact with QEJS. However the latest version of QEJS can't be used via the current version of consolidate (without warning messages) should consolidate.js peer depend on QEJS or would that cause QEJS to be installed every time you install consolidate.js?

Member

ForbesLindesay commented Jan 10, 2013

Can a module have peerDependencies that are optional.

E.g. Both QEJS and consolidate.js work fine on their own, but if you install both then you can us consolidate.js's API to interact with QEJS. However the latest version of QEJS can't be used via the current version of consolidate (without warning messages) should consolidate.js peer depend on QEJS or would that cause QEJS to be installed every time you install consolidate.js?

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 10, 2013

Member

Can a module have peerDependencies that are optional.

No, peer dependencies are (peer-)installed whenever you npm install a package.

That's an interesting use case, however. Maybe let peerDependencies sit for a while and get some usage; if enough people find this to be a missing feature, we should probably add support for it. Also, @isaacs and @indexzero might have pro/con arguments based on their experiences.

Member

domenic commented Jan 10, 2013

Can a module have peerDependencies that are optional.

No, peer dependencies are (peer-)installed whenever you npm install a package.

That's an interesting use case, however. Maybe let peerDependencies sit for a while and get some usage; if enough people find this to be a missing feature, we should probably add support for it. Also, @isaacs and @indexzero might have pro/con arguments based on their experiences.

@aseemk

This comment has been minimized.

Show comment
Hide comment
@aseemk

aseemk Jan 10, 2013

Thanks @domenic for the fantastic work.

I've run into cases where module A "works with" module B, but it's a side-by-side relationship, so optionalDependencies isn't right. Examples are that Streamline.js works well with CoffeeScript, but doesn't require it, doesn't depend on it, and uses its API, so specifying some version information seems nice.

Instead of adding "yet another" dependencies hash (e.g. optionalPeerDependencies), one idea is to expand npm's semver syntax to support this "optional" notion. E.g. a strawman is "foo": "[~0.2.x]" or "foo": "(~0.2.x)", to mean foo is an optional (regular, dev, or peer) dependency, but if it's there, it needs to be ~0.2.x.

aseemk commented Jan 10, 2013

Thanks @domenic for the fantastic work.

I've run into cases where module A "works with" module B, but it's a side-by-side relationship, so optionalDependencies isn't right. Examples are that Streamline.js works well with CoffeeScript, but doesn't require it, doesn't depend on it, and uses its API, so specifying some version information seems nice.

Instead of adding "yet another" dependencies hash (e.g. optionalPeerDependencies), one idea is to expand npm's semver syntax to support this "optional" notion. E.g. a strawman is "foo": "[~0.2.x]" or "foo": "(~0.2.x)", to mean foo is an optional (regular, dev, or peer) dependency, but if it's there, it needs to be ~0.2.x.

@naholyr

This comment has been minimized.

Show comment
Hide comment
@naholyr

naholyr Jan 10, 2013

I think optionalPeerDependencies will be far easier to understand than some cryptic characters appended to our standard-compliant version 😒
But maybe we should talk about that in a new issue, dedicated to "optional peer dependencies" to avoid wasting space here?

naholyr commented Jan 10, 2013

I think optionalPeerDependencies will be far easier to understand than some cryptic characters appended to our standard-compliant version 😒
But maybe we should talk about that in a new issue, dedicated to "optional peer dependencies" to avoid wasting space here?

@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Jan 11, 2013

Looks like there are bugs related to peerDependencies and npm ls: isaacs#3048

Looks like there are bugs related to peerDependencies and npm ls: isaacs#3048

@gregerolsson

This comment has been minimized.

Show comment
Hide comment
@gregerolsson

gregerolsson Feb 7, 2013

Is there any work ongoing with regards to the question @mfncooper raised; an npm linked package cannot find peer dependencies since it is symlinked from another part of the file system? I wouldn't mind giving it a stab unless someone is already working on it.

Is there any work ongoing with regards to the question @mfncooper raised; an npm linked package cannot find peer dependencies since it is symlinked from another part of the file system? I wouldn't mind giving it a stab unless someone is already working on it.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Feb 8, 2013

Member

@gregerolsson I think the point @mfncooper raised is not terribly relevant to our current peer dependencies implementation. The idea of a host package auto-discovering its plugins is not really npm's responsibility IMO. Although, if someone wanted to write something that automatically crawled siblings, inspected their package.json, and thus compiled a list of plugins, maybe host packages would start using it.

That said, if there are issues related to npm link that make it not work with peer dependencies when peer dependencies are used in the normal way, I'm all for fixing it. But I'd think that if a peer-depends on b, and x depends on a and b, this should work:

$ cd b; npm link; cd ..
$ cd a; npm link; cd ..
$ cd x; npm link a; npm link b
Member

domenic commented Feb 8, 2013

@gregerolsson I think the point @mfncooper raised is not terribly relevant to our current peer dependencies implementation. The idea of a host package auto-discovering its plugins is not really npm's responsibility IMO. Although, if someone wanted to write something that automatically crawled siblings, inspected their package.json, and thus compiled a list of plugins, maybe host packages would start using it.

That said, if there are issues related to npm link that make it not work with peer dependencies when peer dependencies are used in the normal way, I'm all for fixing it. But I'd think that if a peer-depends on b, and x depends on a and b, this should work:

$ cd b; npm link; cd ..
$ cd a; npm link; cd ..
$ cd x; npm link a; npm link b
@gregerolsson

This comment has been minimized.

Show comment
Hide comment
@gregerolsson

gregerolsson Feb 8, 2013

Right, the problem is actually with Node (module.js) when you only link a which peerDepends on b. I described it more thoroughly in this gist. We might just be using npm link in a way that was not intended, but we actually found us in this situation.

We have an Express-based host application that depends on a bunch of smaller "mountable" Express-apps that we develop ourselves as reusable packages. These apps peerDepend on Express (and a few others) that they expect the host application to provide. And if we need to make changes to an "app" we just npm link that particular app to a development version of it, but it still executes in the host for the final tests. Once linked, the app cannot find its peerDependencies.

Again, we might be doing things the wrong way here.

Right, the problem is actually with Node (module.js) when you only link a which peerDepends on b. I described it more thoroughly in this gist. We might just be using npm link in a way that was not intended, but we actually found us in this situation.

We have an Express-based host application that depends on a bunch of smaller "mountable" Express-apps that we develop ourselves as reusable packages. These apps peerDepend on Express (and a few others) that they expect the host application to provide. And if we need to make changes to an "app" we just npm link that particular app to a development version of it, but it still executes in the host for the final tests. Once linked, the app cannot find its peerDependencies.

Again, we might be doing things the wrong way here.

@mfncooper

This comment has been minimized.

Show comment
Hide comment
@mfncooper

mfncooper Feb 10, 2013

Contributor

I agree with @domenic that the auto-discovery code isn't part of npm's purview. I have working code that includes dealing with npm linked packages; I just need to pull it out of the app it's currently embedded in, and make it a reusable module. Will try to do that soon ...

BTW, @domenic, did that blog post happen? Link, please?

Contributor

mfncooper commented Feb 10, 2013

I agree with @domenic that the auto-discovery code isn't part of npm's purview. I have working code that includes dealing with npm linked packages; I just need to pull it out of the app it's currently embedded in, and make it a reusable module. Will try to do that soon ...

BTW, @domenic, did that blog post happen? Link, please?

@edef1c

This comment has been minimized.

Show comment
Hide comment
Member

edef1c commented Feb 10, 2013

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Feb 11, 2013

Member

@gregerolsson nice gist, that does explain the problem. It seems like a limitation of how npm link and Node's module system interact, sadly. Can't think of a nice solution, and can't think of anything you're doing wrong :(. Probably the dev-time copying is what you need to do.

Oh npm link, so close yet so far...

Member

domenic commented Feb 11, 2013

@gregerolsson nice gist, that does explain the problem. It seems like a limitation of how npm link and Node's module system interact, sadly. Can't think of a nice solution, and can't think of anything you're doing wrong :(. Probably the dev-time copying is what you need to do.

Oh npm link, so close yet so far...

@dandean dandean referenced this issue in dandean/express-form Feb 12, 2013

Closed

Express 3.0 Support #35

@danielmcormond danielmcormond referenced this issue in yeoman/yo Apr 9, 2013

Closed

Remove bundled generators #19

@jasonkarns jasonkarns referenced this issue in linemanjs/lineman Jun 6, 2013

Closed

Make `lineman new` even better #58

@balupton balupton referenced this issue in docpad/docpad Oct 30, 2013

Open

Semver Plugins #691

@yamadapc yamadapc referenced this issue in moveline/mongoose-pagination Dec 20, 2013

Open

Made pagination installable, converted to CoffeeScript #1

@briandipalma

This comment has been minimized.

Show comment
Hide comment
@briandipalma

briandipalma Jan 5, 2014

Is there a --save-peer command for installing a peerDependency? It doesn't seem to work atm.

Is there a --save-peer command for installing a peerDependency? It doesn't seem to work atm.

@KenanY

This comment has been minimized.

Show comment
Hide comment
@briandipalma

This comment has been minimized.

Show comment
Hide comment
@briandipalma

briandipalma Jan 5, 2014

@KenanY Thank you, I see why --save-peer may not make sense but it's still a handy piece of functionality...

I'm struggling to get versions to work as I wish when using a github URL as my installing version...I guess you can't have versioning unless you publish to npm...

@KenanY Thank you, I see why --save-peer may not make sense but it's still a handy piece of functionality...

I'm struggling to get versions to work as I wish when using a github URL as my installing version...I guess you can't have versioning unless you publish to npm...

@briandipalma

This comment has been minimized.

Show comment
Hide comment
@briandipalma

briandipalma Jan 13, 2014

If you want to make npm the best JS package manager out there (and I hope you can) this inability to easily save a peerDependency should be fixed. In the front end I would prefer to have a flat dependency tree so when I install using --save I open up my package.json and move the newly installed dependency (let's us say a Web Component) from dependencies to peerDependencies.

This is a unfortunate overhead..

Unless people would recommend npm dedupe instead? That again seems like an overhead...after every install you would need to remember to run it and all new devs would have to be taught this...and I don't think dedupe would pull up dependencies that aren't duplicates so still leaving me with a deep tree in many cases...

If you want to make npm the best JS package manager out there (and I hope you can) this inability to easily save a peerDependency should be fixed. In the front end I would prefer to have a flat dependency tree so when I install using --save I open up my package.json and move the newly installed dependency (let's us say a Web Component) from dependencies to peerDependencies.

This is a unfortunate overhead..

Unless people would recommend npm dedupe instead? That again seems like an overhead...after every install you would need to remember to run it and all new devs would have to be taught this...and I don't think dedupe would pull up dependencies that aren't duplicates so still leaving me with a deep tree in many cases...

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 13, 2014

Member

Yeah, you definitely should not use peerDependencies for a flat dependency tree; that is going to break a lot of things. They are meant for plugins, not artificially flattening your dependency tree. Use dedupe for that.

Member

domenic commented Jan 13, 2014

Yeah, you definitely should not use peerDependencies for a flat dependency tree; that is going to break a lot of things. They are meant for plugins, not artificially flattening your dependency tree. Use dedupe for that.

@rlidwka

This comment has been minimized.

Show comment
Hide comment
@rlidwka

rlidwka Jan 13, 2014

Contributor

Unless people would recommend npm dedupe instead?

Yes. I'd rather see it done automatically though.

Contributor

rlidwka commented Jan 13, 2014

Unless people would recommend npm dedupe instead?

Yes. I'd rather see it done automatically though.

@briandipalma

This comment has been minimized.

Show comment
Hide comment
@briandipalma

briandipalma Jan 13, 2014

I know it's not meant for that but it seems the only way to create a flat dependency tree, dedupe won't flatten unless there are two instances of a dependency...umm...maybe this is not an issue if using an ES6 module loader like I'm doing atm as I won't be creating script tags to import JS code any longer (Praise the Lord!).

I've added npm dedupe as a postinstall script, I'll see how that works out I guess.

I know it's not meant for that but it seems the only way to create a flat dependency tree, dedupe won't flatten unless there are two instances of a dependency...umm...maybe this is not an issue if using an ES6 module loader like I'm doing atm as I won't be creating script tags to import JS code any longer (Praise the Lord!).

I've added npm dedupe as a postinstall script, I'll see how that works out I guess.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 13, 2014

Member

Flat dependency trees are not inherently virtuous; npm doesn't want to support them just to keep the files flat. Non-duplicated dependencies do have independent virtues, though, so npm does put in work to make that possible. Peer dependencies are pretty unrelated to this whole thing though.

Member

domenic commented Jan 13, 2014

Flat dependency trees are not inherently virtuous; npm doesn't want to support them just to keep the files flat. Non-duplicated dependencies do have independent virtues, though, so npm does put in work to make that possible. Peer dependencies are pretty unrelated to this whole thing though.

@briandipalma

This comment has been minimized.

Show comment
Hide comment
@briandipalma

briandipalma Jan 13, 2014

Sorry @domenic, I meant non-duplicated dependencies, we don't want to send the same dependency version to the browser twice that would be a performance issue as far as I'm concerned but two versions of the same dependency might be OK, at first I was against it but I can see that it could have its uses.

It gives the developer time to verify the package that depends on the old version works on the new one and then upgrade or if not fix the issues and then upgrade in a new release.

You would need a custom ES6 module loader though that understood node_modules but any complex web app would probably end up having it's own module loader anyway.

Sorry @domenic, I meant non-duplicated dependencies, we don't want to send the same dependency version to the browser twice that would be a performance issue as far as I'm concerned but two versions of the same dependency might be OK, at first I was against it but I can see that it could have its uses.

It gives the developer time to verify the package that depends on the old version works on the new one and then upgrade or if not fix the issues and then upgrade in a new release.

You would need a custom ES6 module loader though that understood node_modules but any complex web app would probably end up having it's own module loader anyway.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 13, 2014

Member

I mean, any complex web app would probably end up concatenating all its modules into a single file anyway, at which point you're best off using browserify :)

Member

domenic commented Jan 13, 2014

I mean, any complex web app would probably end up concatenating all its modules into a single file anyway, at which point you're best off using browserify :)

@rlidwka

This comment has been minimized.

Show comment
Hide comment
@rlidwka

rlidwka Jan 13, 2014

Contributor

I mean, any complex web app would probably end up concatenating all its modules into a single file anyway

I wonder if this practice will go away with SPDY adoption...

Contributor

rlidwka commented Jan 13, 2014

I mean, any complex web app would probably end up concatenating all its modules into a single file anyway

I wonder if this practice will go away with SPDY adoption...

@briandipalma

This comment has been minimized.

Show comment
Hide comment
@briandipalma

briandipalma Jan 13, 2014

@domenic For development too? You'd want F5 reload capability so I guess you use beefy?

@rlidwka I would love to move away from these tools and rely on the platform instead, I've had some problems with my SPDY tests but I haven't spend much time on them. I had them working up to 100 files pushed but then the browsers hit their max push resources limit and so I tried to push 100 files per request but that seemed to totally fail as the browsers requested files that I had already pushed...

@domenic For development too? You'd want F5 reload capability so I guess you use beefy?

@rlidwka I would love to move away from these tools and rely on the platform instead, I've had some problems with my SPDY tests but I haven't spend much time on them. I had them working up to 100 files pushed but then the browsers hit their max push resources limit and so I tried to push 100 files per request but that seemed to totally fail as the browsers requested files that I had already pushed...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment