Git urls don't allow semver ranges

[NickHeiner]

According to the documentation:

Git URLs as Dependencies
Git urls can be of the form:

git://github.com/user/project.git#commit-ish
git+ssh://user@hostname:project.git#commit-ish
git+ssh://user@hostname/project.git#commit-ish
git+http://user@hostname/project/blah.git#commit-ish
git+https://user@hostname/project/blah.git#commit-ish

The commit-ish can be any tag, sha, or branch which can be supplied as an argument to git checkout. The default is master.

I would like to be able use the full semver range syntax for the commit-ish part:

git://github.com/user/project.git#1.0.x
git://github.com/user/project.git#~2

I'm dependent on git urls because my company uses a number of repos on our internal github instance, and we don't have our own npm registry. (Based on digging through some old issues, it wasn't even clear that setting up our own npm registry would be possible.)

Could I submit a PR for this?

[isaacs]

If you want that to work, then tag or branch so that 1.0.x points to something in git. We're not going to walk through commits finding semver-ish-looking things and try to be clever about them.

[edef1c]

@isaacs We can walk through tags, though? I believe I proposed this to you in the past.

[isaacs]

Yeah, it's not a good idea, though. Very complicated and weird. If you have loose git deps, then point at a branch, rather than a commit.

[myrne]

Write a pre-procressor for package.json, call your sourcefile clever-package.json or so. It's a known fact that pre-processors are the solution to just about any gripe you have with the syntax for your platform of choice. ;)

But seriously, the pre-processor could fix up the package.json for you so that it points to a real commit. It'd be certainly a useful little tool, if open source.

Ideally, npm would have a "preinstallpackages" script hook for this. Then you can be sure that the generated package.json is always up to date before it gets evaluated.

[bcherny]

@isaacs It's certainly not a trivial implementation, but it would be very nice to have feature parity with bower here.

[rlidwka]

It's certainly not a trivial implementation

If we're talking about github only, it's trivial. I can't point out the exact commit, but it was done in visionmedia/npm and yapm a long time ago using github api and gh-lookup. Just run yapm install visionmedia/commander.js@^1.2.0 and see for yourself.

Actual git url's are more tricky than that, because you have to fetch the entire tree before you know what commit hash you need, and there are a few corner cases. But people usually look just for github repos anyway.

[bcherny]

For the record, here is bower's implementation: https://github.com/bower/bower/blob/master/lib/core/resolvers/GitResolver.js#L91

[NickHeiner]

Amusingly, although I opened this issue originally, I've now come to agree with @isaacs that this would be "very complicated and weird". I would advise people to just use an npm registry instead of conflating source control with published package management.

[jasonkuhrt]

@NickHeiner How's that working out with your private packages? Even today, years since npm started, private package hosting is only just in beta.

[rlidwka]

Even today, years since npm started, private package hosting is only just in beta.

npm private package hosting isn't the only one hosting available out there. I'm using private packages for 2 years now, and it works just fine.

But git urls are useful for public stuff, if people don't want to use npm registry for one reason or another.

[jasonkuhrt]

npm private package hosting isn't the only one hosting available out there. I'm using private packages for 2 years now, and it works just fine.

Link to service provider?

[rlidwka]

@jasonkuhrt , if you are looking for a npm-compatible private registry installed on your servers, look at sinopia. If you're looking for 3rd party service provider (SaaS), check cnpm.

[NickHeiner]

@jasonkuhrt, we are hosting our own copy of the npm registry on couchdb. It's painful to maintain at times but generally works.

[Aaike]

how come jspm is able to install git packages using a semver range ? can the same technique not be used ? i think it's just getting the tags using git ls-remote --tags and then do the semver match. no need to over complicate it ?
it would really be fantastic to be able to specify a semver range for github packages...

[ciekawy]

being impressed by the richness of node packages and the well working idea of reusable modules I am convinced that this is the very case where having an option of working semvers for git urls is worth extracting the job already done for bower.

[dehru]

bower.json has no problem with this syntax

"thingy": "http://thingy.lss.com/scm/cuc/emc-login.git#1.x.x",

I wish I could add a similar syntax to package.json.

[ciekawy]

@dehru Actually latest npm resolves your case correctly. The challenge is the semver handling ie. git://myrepo.com/mymodule.git#^1.0.5 I mean the caret and tilde chars as range selectors.

[artemsvirskyi]

@ciekawy, see https://github.com/npm/node-semver#x-ranges-12x-1x-12-

[raveclassic]

@dimfeld Thanks for the tip, but unfortunately this won't help with version ranging with ^ and ~ which is critical.

UPD: Moreover, we have force push restricted in repos

[filipesilva]

@dimfeld thanks for pitching in your strategy! The team I'm in is a small one and as is atm, I think moving the tags would be more complicated (and thus error prone) for everyone to grok correctly than just being thorough with updating platforms.

Experience might show me otherwise, and in that case your strategy is probably the soundest one. The only other sane alternative is branches for minor version ranges, as someone suggested earlier.

[dimfeld]

Yeah, I agree it's a high-maintenance strategy... It could maybe be automated using a post-commit hook that pulls the package version from package.json and updates the tags if needed. https://coderwall.com/p/mk18zq/automatic-git-version-tagging-for-npm-modules may be a good starting point, though I haven't tried it yet.

As far as ^ and ~ compatibility, my thinking was that using the vX tag is similar to using ^X.Y.Z, in that it get the latest version with a matching major version , and the vX.Y tag is like using ~X.Y.Z, as it gets the latest version matching the major and minor versions. When you update a library from 1.5.7 to 1.6.0, for example, the v1.5.7 and v1.5 tags would not be updated, so puling in the library at tag v1.5 would continue to be similar to semver matching on ~1.5. Of course, this is a simplified model compared to the full power of semver matching, and there may be some subtleties you're thinking of that I'm forgetting. And if you're unable to force push it's a moot point anyway.

Ultimately, I agree fully that having official support for semver ranges on git urls would be much better :)

[StephanBijzitter]

Honestly, although I highly support npm adding this possibility, why doesn't GitHub itself do this?
https://github.com/npm/npm/releases/tag/v3.8.x would be rather awesome. The only caveat would be that a tag could literally be v3.8.x and in that case, you'd have a pretty poor naming convention and you just don't deserve any better.

[arikon]

@StephanBijzitter This should be done in npm, not in Github.

[othiym23]

The CLI team still intends to ship this as part of giving hosted Git dependencies as much parity with registry dependencies as is practicable. If somebody wants to take a stab at implementing this, it's mostly in the caching logic (and fetchPackageMetadata) where things would need to change. Thanks to all for your patience!

[cellvia]

thank you @dimfeld i ended up implementing the same thing for my team as well, only difference is i opted to use v1.1.x and v1.x.x notation to help make obvious to them whats going on with the semver-ing ... works like a charm til semver on git urls is officially supported

as far as automation, i have the script that re-creates those tags run anytime i do a "npm run publish-{major,minor,patch}" along with testing, babel transpiling, npm version, etc

[dimfeld]

I'm glad it's working well for you @cellvia. Good idea with the v1.x.x style notation. I think I'll start using that too.

[sankethkatta]

@othiym23 I took a stab at implementing this over at #15308. Would love to get your feedback!

[ondratra]

If you really need this feature you can just switch to Yarn utility instead of NPM. It supports regular package.json and has this feature already.

Example

$ yarn add visionmedia/commander.js#^1.2.0
yarn add v0.20.3
info No lockfile found.
warning No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 2 new dependencies.
├─ commander@1.3.2
└─ keypress@0.1.0
warning No license field
Done in 1.98s.

[raveclassic]

@ondratra That's exactly what we've ended up with.

[npm-robot]

We're closing this issue as it has gone thirty days without activity. In our experience if an issue has gone thirty days without any activity then it's unlikely to be addressed. In the case of bug reports, often the underlying issue will be addressed but finding related issues is quite difficult and often incomplete.

If this was a bug report and it is still relevant then we encourage you to open it again as a new issue. If this was a feature request then you should feel free to open it again, or even better open a PR.

For more information about our new issue aging policies and why we've instituted them please see our blog post.

[benwiley4000]

Worth noting this is supported now: https://github.com/npm/npm/releases/tag/v5.0.0

[SkeLLLa]

It seems broken in npm 5 (at least 5.5.1):

"npm": "git+https://github.com/npm/npm#semver:^5.5.0"

Produces:

npm ERR! code ENOVERSIONS
npm ERR! No valid versions available for npm

[Yannik]

@SkeLLLa maybe try #semver:^v5.5.0?

[Yannik]

@SkeLLLa also, according to #17621, this should be fixed with 5.6.0

[SkeLLLa]

@Yannik yep, it supported actually in >=5.6.0.