Install multiple versions of a dependency

[denis-sokolov]

Currently a package can have only a single version of a dependency defined.
I have a need to install multiple versions of a dependency at once.
This is a subset of aliasing features that were rejected in #2943.

In my use case I have a library (frozen-express) that can during runtime take an object produced by another library (Express app) and do useful things with it.
I support multiple versions of Express.

During development I want to be able to test my application with different versions of my dependency. I could fake both versions of Express, but I consider that using a real Express app is a better deal. The tests become more of integration tests as opposed to unit tests, but that only makes them more useful.

I could use npm install express@3 && npm test && npm install express@4 && npm test, but that is very significantly slower than running my entire test suite.
In addition, this wrecks any chance of receiving proper code coverage reports.
Obviously, I have code that conditionally applies to one or another version, and no code coverage tool understands that, if I need to run npm test twice (or more times).

The best workaround is, unfortunately, abusing the npm database. A single-line module express3 that depends on express@3 and exports it solves all my problems.

Perhaps this feature request is a tiny bit more viable than #2943?

[denis-sokolov]

Also consider the following use case: an application depends on a useful-util-lib@3. A version 4 of the library comes out and developers want to use it in their application. However, the application contains quite a bit of code that relies on a version 3 lib. Until the time comes to upgrade that all at once, new code continues to depends on version 3, increasing the upgrade burden.

If, however, the requested feature was implemented, the developers could begin using version 4 of the dependency, and refactor old code incrementally.

I believe this is an important step to allow for better, more up-to-date codebases.

[mgol]

Yes please. I've been wishing for this functionality for some time; it's hard to impossible to test with different versions of external packages otherwise.

[analog-nico]

+1

We could learn from Bower. A sample bower.json looks like:

{
    "name": "My Project",
    "version": "0.0.0",
    "dependencies": {
        "jquery": "~2.1.1"
    },
    "devDependencies": {
        "jquery-legacy": "jquery#1.11.1"
    }
}

Thus my bower_components folder contains two folders - jquery and jquery-legacy - each with a different version of jQuery.

npm could implement it identically - except using @ instead of #.

[othiym23]

I have a hard time believing that it would be possible to constrain a solution to this problem in such a way that it wouldn't be subject to the same complications that @isaacs was so wary of in #2943 and #798. In addition, I have written tests that depend on multiple versions of dependencies (I wrote a thing that ran a complex test suite against every published version of node-mongodb, which was an interesting experience I will, with luck, never be asked to repeat), so I know this isn't the only way to solve that particular problem.

Bringing Bower into the discussion is an interesting wrinkle, because it makes me wonder what impacts a change like this might have on the front end, both good and bad. Bower's ability to install the same files to the same place and provide an explicit mechanism for reconciling which file gets written is a strength, but the dependency hell this process can engender is a worrisome weakness. This is more or less the inverse of that situation, but I'd be interested in hearing from anyone who thinks this might be applicable for browserify / atomify / other front-end component workflows.

I'm uncertain that the benefits outweigh the risks, particularly with this solution. I don't feel as strongly about it as @isaacs, but I also am wary.

[analog-nico]

The above Bower example comes from a project of mine which is a webapp supporting only IE9 and up. Thus I use jQuery 2.x. However, I needed to write tests that check that my app degrades successfully on IE8. Thus my test code uses jQuery 1.x.

Today I got the same situation in another project where I needed again two different installations for a node module - one for my production code and one for my test code.

To make multiple versions of the same node module viable it would be important to install the additional versions through aliases like jquery-legacy in my Bower example. To use this version e.g. the test code would have to explicitly require('jquery-legacy'). This means that there is always a primary version of a module. (The one installed by its name and not through an alias.) npm could choose the primary over secondary versions in any dependency resolution algorithm that might need to be extended.

IMHO the cases of requiring multiple versions are rather rare. But sometimes this feature would be very helpful. Possibly we could say that someone who would use such an advanced feature is pretty knowledgeable and knows what he is doing.

[analog-nico]

Maybe @sindresorhus could quickly give his two cents on why Bower needed the "jquery-legacy": "jquery#1.11.1"-aliasing-of-a-different-package-version-feature. Sindre, plenty kudos if you jump in! Or point to a collaborator of yours. Thanks!

tl;dr: @othiym23 asks:

it makes me wonder what impacts a change like this might have on the front end, both good and bad.

[sindresorhus]

👎 I think this is a really bad idea. Fwiw I was against doing that in Bower.

[denis-sokolov]

In #2943, Isaac has argued a single point: require('underscore') returning lodash is confusing.
I couldn't agree more.

This feature request, however, would never introduce anything confusing like that.
Consider that semantically two versions of a package can be as different as two packages.
In fact, I'd say what really identifies a package is its name and version.
Thus in my code I'd want to write var legacyExpress = require('express@3'); (or require('express', '3') or whatever).
This is because to my code, express 3 and express 4 are two separate packages, and if it was express and slowpress, I wouldn't have a problem, but now I do.

Thus the main Isaac's criticism of #2943 does not apply here at all.

[othiym23]

Thus the main Isaac's criticism of #2943 does not apply here at all.

I don't believe this to be true, and the jQuery example @analog-nico sketches explains why. Anything that puts something in a folder on disk named differently from what's in package.json raises a variation of the problem @isaacs described.

Thus in my code I'd want to write var legacyExpress = require('express@3'); (or require('express', '3') or whatever).

require('express@3') will work with the Node module loader, but require('express', '3'), along with anything that requires changes to the Node module resolution algorithm, is a non-starter. That said, that is even more of a potential footgun than the original proposal, and therefore even less likely to get the support of the npm core team.

[denis-sokolov]

Anything that puts something in a folder on disk named differently from what's in package.json raises a variation of the problem @isaacs described.

But that's exactly it.Unlike #2943, I don't want to do that!
I want to require express and express@3 in my package.json.
I want those to be installed in directories express and express@3 on disk.
I then want to use those in my code using require('express') and require('express@3').

but require('express', '3'), along with anything that requires changes to the Node module resolution algorithm, is a non-starter.

What is the point of criticizing bikeshedding? If require('express@3') works, I'm more than happy.

[othiym23]

I want to require express and express@3 in my package.json.
I want those to be installed in directories express and express@3 on disk.
I then want to use those in my code using require('express') and require('express@3').

Where does express@3 come from? If you're aliasing it in package.json, there's a thing on disk that has a name different than its name on the registry, which is the exact thing that @isaacs was objecting to in #2943.

[othiym23]

What is the point of criticizing bikeshedding?

There are hard constraints on how far the bikeshedding can go, and it's useful to know where the boundaries of the design space are. People regularly make feature requests of npm that can't be satisfied without making dramatic modifications to frozen Node APIs.

[denis-sokolov]

there's a thing on disk that has a name different than its name on the registry

I must be missing something. The original problem of confusion between what's installed is clear - is it lodash or is it underscore. In my example it's clear to everybody - it's express version 3.
Consider @3 as part of syntax, same as node_modules/. I wouldn't imagine you saying that node_modules/express is not the same as express in the registry, "because it has node_modules/ in the front.
Or consider installing it into node_modules/express/_versions/3 and then require('express/_versions/3').

And I wouldn't even call it aliasing in package.json. Consider:

{
  "devDependencies": {
    "express": "^4.0.0",
    "express@3": "^3.0.0"
  }
}

It's like a different package, which it kind of is.

[othiym23]

OK. I understand what you're proposing now, and I understand why you want it, but my reservations remain. Thanks for taking the time to explain it, though.

[denis-sokolov]

Thank you for your work at clarifying my proposal.

If reservations still remain, perhaps limiting the feature only to devDependencies would make it easier to agree with? By limiting the feature so strongly we'd avoid unexpected consequences and abuse, but solve the primary use case mentioned in this topic.
In particular, devDependencies only really matter to the package developers anyway.
But I understand if you consider having different functionality between *dependencies even more awful than the alternative. :)

[analog-nico]

Consider @3 as part of syntax

This is a good point @denis-sokolov. In contrast to my Bower example we should find a solution like the one you suggested that does not allow renaming the package. Your suggested solution would cover all my use cases as well.

Let me just throw in a variation of your solution to see what options we might have.

We could also make the notation more compact:

{
  "devDependencies": {
    "express": "^4.0.0, ^3.0.0"
  }
}

The first version number would always be the preferred version to use. Then we could require them like this:

require("express"); // -> Version 4
require("express/_versions/4"); // -> Version 4
require("express/_versions/3"); // -> Version 3

[denis-sokolov]

I did consider such syntax, @analog-nico.
There's a risk associated with it, though, if you require more than a major version.
Consider:

{
  "devDependencies": {
    "express": "^4.0.0, ^3.1.0, 3.0.5"
  }
}

Now how do I require version ^3.1.0? If I require('express/_versions/3') it's unclear if I mean 3.0 or 3.1, but if I require('express/_versions/3.1'), we need to install the latest 3.x version 3.99.0 into a directory 3.1, which is that evil confusion Isaac wants to avoid.

If we use my proposed syntax all of that is avoided as I directly alias short names with requested versions:

{
  "devDependencies": {
    "express": "^4.0.0",
    "express@3": "^3.1.0",
    "express@3.0": "3.0.5"
  }
}

Installing 3.99 into a directory 3 is not controversial, as well as installing 3.0.5 into 3.0,
and my requires are explicit as well.

[denis-sokolov]

Of course, npm should avoid the following illegal syntax:

{
  "devDependencies": {
    "express@3.0.9": "^4.0.0"
  }
}

[analog-nico]

If reservations still remain, perhaps limiting the feature only to devDependencies would make it easier to agree with?

+1 although I understand that handling dependencies and devDependencies differently is not so nice. However, as I see it, for multiple versions in devDependencies there are perfectly valid use cases when writing tests. My guts say that multiple versions in dependencies -> production code shows that developers didn't properly design or maintain their code base.

[analog-nico]

@denis-sokolov I prefer your solution now.

[pderaaij]

Whats the status on this issue? We are in need for such a solution so we can use multiple versions of a dependency. Our use case is the following.

We have a dependency that contains schema object which are information contracts. Our application using these schema objects need to support multiple versions of these schemas so we can send out the correct information based on the version of the schemas used by the consumer.

So we want to define a dependency on tag V1 of schemas and tag V2 on schemas. Within the application we'll do the logic of requiring the correct version, but for now it isn't possible to define the different versions as a dependency in the package.json.

It seems to me that the proposed solution in this issue will help us out, so I'm curious what the state of this issue is.

[denis-sokolov]

Interesting use case, @pderaaij. I like it.

There's a possible cricitism: versions of your schemas should not necessarily be tied to versions of your schema package. Consider that if V1 and V2 are used in production, they're both still actual, so perhaps the versioning distinction should happen inside a single version of the schema package.

As for what the state of the issue is, the answer, I think, is the same as with most other feature requests: npm devs are flooded with work.

[medikoo]

I think this change will require major changes to node's modules resolution logic, which I think it's no go, and not really worth it.

If someone has really such rare use case, he/she should probably include needed modules versions directly in a project, and require it as e.g. require('./lib/express/v2'), require('./lib/express/v3')

@pderaaij how do you imagine require'ing different versions of same package in your module? e.g. if package is name schema, normally you require it with require('schema'). How do you see require'ing distinctively two different versions of schema within one module?

[silkentrance]

@skylize if so, they could have shut down comments on this thread a while ago, but they did not.

[scott113341]

@skylize your assertion that the maintainers of npm don't listen to its community of users is, frankly, disrespectful. There are dozens of comments from maintainers in this issue alone explaining very well why this functionality may be very difficult to implement properly.

If you think that there is a particular solution (either discussed above or not yet explored) that is robust enough to overcome these well-explained technical barriers, then I would kindly ask that you contribute to the technical discussion by explaining why the solution is adequate.

[skylize]

@scott113341

your assertion that the maintainers of npm don't listen to its community of users is, frankly, disrespectful.

I'm greatly appreciative of all the work that's gone into npm, so I apologize to anyone who might feel disrespected by my comments.

Unfortunately, I guess I let my real feelings spill out about this particular issue. The claim that you should only ever need one version of a dependency seems absurdly near-sighted to me in today's world of deeply nested graphs of rapidly updating dependencies with breaking changes.

I remember reading this post way back when from @isaacs on Jan 22, 2015, which permanently closed the issue:

I don't think that we need new fanciness for this. It's a rare edge case, and a confusing thing to implement in a general way.

This post did not raise any real technical concerns or explain what is so difficult. It was just a raw, unfiltered opinion from one person that shut everything down for everybody. From what I can tell, this seems to be not-even-on-the-radar for the people actually writing code for the project, because it is a "closed" issue. Put this on a roadmap and we might see some real progress and some practical solutions.

In a later comment @isaacs adds

If someone has a dozen or a hundred actual use cases today for this, then let's investigate further. But what I'm hearing so far is a lot of engineering solutions for a very small one-off problem that only a few people have.

The community has filled this thread to the brim with numerous use-cases, but the issue remains closed and seemingly ignored.

---

If you think that there is a particular solution (either discussed above or not yet explored) that is robust enough to overcome these well-explained technical barriers, then I would kindly ask that you contribute to the technical discussion by explaining why the solution is adequate.

I added a proposal in June 2015 on how to deal with the overall technical problems, which nobody ever raised any concerns against.

I propose something like npm install --save-common lodash@1 or npm install --save-common-dev lodash@1.
Npm would at that point:

1. Install the module into a folder with the version number appended to the name
2. Replace all matching modules in the dependency tree with symlinks to the versioned module.
3. Mark it in as a common dependency in package.json using token characters, so npm install would be able to replicate the process.

Looking back at that I see some flaws in my idea, namely the reliance on symlinks which are not easily dealt with cross-platform. But platform-specific symlinks are not impossible. And regardless, the basic premise of writing packages to version-numbered folders is still a good starting point, if someone was actually trying to build this.

[mnpenner]

Just wanted to remind you guys that yarn add lodash2@npm:lodash@2.x already works so we know this is possible, and there's already a precedent for the syntax for it. It looks like this in package.json:

  "dependencies": {
    "lodash2": "npm:lodash@2.x"
  },

I think this is a better solution than the one proposed by @skylize too because it lets you choose a name for the additional packages, opposed to just appending a number to the name which may or may not conflict with another package. Also, I don't think there's a need for the special cli-args, --save-common.

[skylize]

@mnpenner

I think this is a better solution than the one proposed by @skylize too because it lets you choose a name for the additional packages

In this comment to #2943, @isaacs made it pretty clear that blanket aliasing will never be considered. If we want a solution that will actually get built, we need to be thinking of ways to give the user only the bare minimum amount of control needed to reach the target, not allowing them to just name anything whatever they want.

[mnpenner]

@skylize I think what isaacs was getting at is that is if you require one module, it shouldn't load the module from an unrelated directory (I think). With yarn's solution, lodash2 is actually installed into node_modules/lodash2. So node's module resolution algorithm remains completely unchanged.

That said, yes, you could use this deceive the reader by aliasing one package to an unrelated package's name, but I don't understand why you would want to go to such great lengths to prevent developers from writing stupid code. I could just as easily clone jquery and publish to npm as lodash2 and then what? A few people get confused and the world chugs on.

[nyurik]

@skylize I don't think "never consider" is ever a good approach when working with a community. Opinions change, new arguments and usecases come forth, etc. The #2943 comment was made over 5 years ago - yarn didn't even exist back then. In just a year, yarn is a major player in this field - clearly showing that NPM has become the IE6 of the package management, and could use some love.

yarn clearly broke compatibility on this one. Yet, the community is clearly in favor of yarn's approach, so I think NPM team should reconsider. Otherwise it hurts all of us - we all would like both, the compatibility AND the version aliasing. If NPM introduces a new incompatible syntax just for version aliasing without any justification, the issue will get even worse.

[dead-claudia]

Could we start trying sending support emails or something, rather than blowing this issue up with borderline off-topic stuff? It kind of gets old getting an email + a notification each time one of you all comment here.

[DuBistKomisch]

In response to #5499 (comment):

No one yet has provided a real-world example of (2) in this thread, so I am somewhat suspicious that it's only hypothetical anyway.

We want to use socket.io-client to connect to separate v1 and v2 servers, and the protocols are mutually incompatible, so we'd need both clients installed simultaneously.

Eventually I'd like to switch to yarn for this project of course (sick of npm's shit, and yarn's only burned me once so far), but I don't have time to clean up all the tech debt required right now, so this is really annoying... guess I'll try out npm-install-version/node-multidep.

Edit: npm-install-version works I guess, can't actually include it in the package.json though so just need to install it manually everywhere...

Edit 2: npm isntall -g yarn gg ez

[palmerj3]

I have a real-world scenario for this.. I maintain jest-junit and I would like to setup travis ci to test it against multiple versions of jest. Doing that is fine but I would also like to be able to run my unit tests with whatever version of jest I choose.

[hinell]

@palmerj3 Well, there are a lot of real-world scenarios especially when you writing cross-platform software but looks like npm is unlikely to implement anything cause the issue is opened almost 3 years ago and nothing has changed since then. That's why it makes me sad. 😞

[silkentrance]

While it is fun seeing that this is not going to be resolved, whereas for example yarn provides a very good alternative, I am now signing off.

[trevorwhealy]

👋 We have a lot of legacy code written in the v1 release of React-Transition-Group (https://github.com/reactjs/react-transition-group), and it is working great.

As React evolved and the maintainers changed, the codebase also changed dramatically, causing them to release a v2 that had breaking changes. It would be a lot of work to retool legacy code that is actually working well and won't be changed for the foreseeable future.

We locked down to v1 so that our app would still work, but we want to use v2 for all future development. Allowing aliases would be pretty useful in this case.

This comment from earlier in the thread seems to be a simple solution to this common problem: #5499 (comment)

Ideally, we would really enjoy being able to alias like so:

"react-transition-group@v1": "^1.1.3",
"react-transition-group@v2": "https://github.com/reactjs/react-transition-group.git#master",
"react-transition-group@v3: "^3.0.0",

[jlpospisil]

I am working on migrating a large project from bootstrap 3 to 4. This feature would be nice. I would like to import both, and apply bootstrap 4 to specific pages/areas as the migration progresses.

[pauldraper]

Yarn supports this case via general-purpose aliasing. ("This isn't going to be reconsidered" #2943)`

"react-transition-group-v1": "npm:react-transition-group@^1.1.3",
"react-transition-group-v2": "https://github.com/reactjs/react-transition-group.git#master",
"react-transition-group-v3: "npm:react-transition-group@^3.0.0",

[VincentCharpentier]

So basically is this gonna be a feature ?
With the constant (breaking) changes we have to face in javascript projects this seems like an obviously needed feature.
Will it be possible soon ?
If not, that's not a big issue I could switch to yarn, but honestly I wish I could stay with npm
Thanks

[jamesmfriedman]

Same issue. I run a UI library for React and I need to test across 6 different versions since the react api is subtlety different between them.

[armanriazi]

According to the following links how match package.json for angular 5 and 6

In my use case I need to have both of them on "Dependency" section.It's not important how handle with every form.

key : "npm-install-version"
{ "name": "app-angular", "version": "1.0.0", "scripts": { "npm-install-version": "npm-install-version", "ng": "ng", "start": "ng serve", "build": "ng build", "test": "ng test", "lint": "ng lint", "e2e": "ng e2e" }, "private": true, "dependencies": { "core-js": "^2.5.4", "zone.js": "^0.8.26" }, "devDependencies": { "@angular-devkit/build-angular": "~0.6.1", "@angular/cli": "~6.0.1", "@angular/compiler-cli": "^6.0.0", "@angular/language-service": "^6.0.0", "@types/jasmine": "~2.8.6", "@types/jasminewd2": "~2.0.3", "@types/node": "~8.9.4", "codelyzer": "~4.2.1", "jasmine-core": "~2.99.1", "jasmine-spec-reporter": "~4.2.1", "karma": "~1.7.1", "karma-chrome-launcher": "~2.2.0", "karma-coverage-istanbul-reporter": "~1.4.2", "karma-jasmine": "~1.1.1", "karma-jasmine-html-reporter": "^0.2.2", "npm-install-version": "^6.0.2", "protractor": "~5.3.0", "ts-node": "~5.0.1", "tslint": "~5.9.1", "typescript": "~2.7.2" }, "npm-install-version": { "@angular/animations": [ "^6.0.0", "^5.1.0" ], "@angular/common": [ "^6.0.0", "^5.1.0" ], "@angular/compiler": [ "^6.0.0", "^5.1.0" ], "@angular/core": [ "^6.0.0", "^5.1.0" ], "@angular/forms": [ "^6.0.0", "^5.1.0" ], "@angular/http": [ "^6.0.0", "^5.1.0" ], "@angular/platform-browser": [ "^6.0.0", "^5.1.0" ], "@angular/platform-browser-dynamic": [ "^6.0.0", "^5.1.0" ], "@angular/router": [ "^6.0.0", "^5.1.0" ], "rxjs": [ "^6.0.0", "^5.5.2" ] } }

[armanriazi]

Because i need Angular 5 for GraphQl and Angular 6 for stable material UI !!?

[Mani-RRI]

hello every one,
I need both rxjs@^5.5.0 and rxjs@^6.0.0-beta.4 have to be installed. But if I try to install one of these, the other gets vanish. Do I have a solution?

[nyurik]

@Mani-RRI yes, we switched to yarn, and used their aliasing system. Works well.

[Mani-RRI]

@nyurik , thank you : )

[doesdev]

I really don't want to switch to Yarn. I hate that such a key part of the ecosystem is fragmenting. That being said, this thread is extremely discouraging when considering staying with NPM.
😟
The team has ignored all the clearly laid out and highly valid reasons why this is needed and pretended that this is a non-issue. If it were a non-issue it would, quite literally, not be an issue. It is, and with an eff ton off comments and support from the community for said feature.

[Mani-RRI]

@doesdev there is an easy solution, but it depends on you which versions you want to use. There are some particular sets of dependencies work fine together. And you need to install them together. After instaling check your JSON file to ensure that all the dependencies are available that you wanted to be installed. for example, one set of working dependencies I am providing here,...

[nyurik]

@Mani-RRI the whole idea of package manager is that you don't want to know what works well together. It should "just work" (TM). Even if two versions only differs in the readme, or if a totally new code was swapped in, I as a developer should not be aware of that. I should be able to alias two or more versions, and use them side by side.