Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Updating git dependency urls does not work #1727

Closed
felixge opened this Issue Nov 17, 2011 · 93 comments

Comments

Projects
None yet
Contributor

felixge commented Nov 17, 2011

Currently npm does not notice when the version reference of a previously installed package is upgrade to a git url.

So in order to get the right version one has to rm -rf node_modules/<package> first before running npm install again.

Example: Consider the following package which had some changes in master after version 0.0.2 was released. Among them the Readme was modified:

$ npm install -S utest
utest@0.0.2 ./node_modules/utest 
$ shasum node_modules/utest/Readme.md 
c0f676fb3ccd4aa405c92451e3a9de53f23c39bf  node_modules/utest/Readme.md
$ vim package.json
# Change utest version from "0.0.2" to "git://github.com/felixge/node-utest.git"
$ npm install .

$ shasum node_modules/utest/Readme.md 
c0f676fb3ccd4aa405c92451e3a9de53f23c39bf  node_modules/utest/Readme.md
# ^- Notice how the package has not been updated with the new code from git
$ rm -rf node_modules/utest
$ npm install .
utest@0.0.2 ./node_modules/utest 
$ shasum node_modules/utest/Readme.md 
0167009693fce6aa30e7ef6ea25406da8e1cc0b3  node_modules/utest/Readme.md
# ^- Notice how the package was now updated

Let me know if I can provide further details.

--fg

Owner

isaacs commented Nov 17, 2011

In general, npm install (without any args) doesn't update anything that's already present, and only installs missing deps. If you then want it to install again, you need to do npm install <whatever>.

@mikeal asked me about this last night. I think if you and he are getting confused by it, it's the wrong api. Wanna discuss on the npm-@googlegroups.com mailing list?

Contributor

felixge commented Nov 18, 2011

I've created a discussion on the mailing list, would love to get your input: https://groups.google.com/d/topic/npm-/KnSd2gpA6fw/discussion

dshaw commented Nov 18, 2011

+1 I've been rm -rf-ing to get around this too.

Contributor

felixge commented Nov 23, 2011

@isaacs: did you have a chance to look at this yet / read the post in the ML?

Owner

isaacs commented Nov 23, 2011

I saw the mailing list post, but haven't gotten to it yet. Been super busy trying to get 1.1 finished (with the built-in tar stuff.)

jperras commented Nov 25, 2011

To add my two cents: This becomes something of an issue when attempting to deploy to a PaaS, such as Heroku. And by issue, I mean the only way around it seems to be completely destroying the application and re-deploying it from scratch.

Contributor

felixge commented Nov 28, 2011

@isaacs np, I understand.

@jperras Good point. I'm rather suprised that heroku doesn't wipe the install on deployment so. It seems like that's what they are advocating:

http://www.12factor.net/
http://blog.heroku.com/archives/2011/6/28/the_new_heroku_4_erosion_resistance_explicit_contracts/

(Great articles btw.)

jperras commented Nov 28, 2011

@juggy Yes, but I have it pointed towards a tarball, e.g.

  "dependencies": {
    "blah": "https://github.com/jperras/repository_name/tarball/v1.2.0",
    ...
  },

I'm not sure this is the same issue, but I've seen that git updates treats the git dependencies as npm registry modules, here's an example:

alfred@ubuntu:~/apps/flight-office$ npm update couch-potato

npm WARN eventemitter2@0.4.1 package.json: 'contributers' should probably be 'contributors'
npm http GET https://registry.npmjs.org/couch-potato/1.0.2
npm http 404 https://registry.npmjs.org/couch-potato/1.0.2

npm ERR! 404 'couch-potato' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, or http url, or git url.
npm ERR! 
npm ERR! System Linux 3.0.0-16-generic
npm ERR! command "node" "/home/alfred/.nvm/v0.6.13/bin/npm" "update" "couch-potato"
npm ERR! cwd /home/alfred/apps/flight-office
npm ERR! node -v v0.6.13
npm ERR! npm -v 1.1.9
npm ERR! code E404
npm ERR! message 404 Not Found: couch-potato
npm ERR! errno {}
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/alfred/apps/flight-office/npm-debug.log
npm not ok

But it worked with npm install:


alfred@ubuntu:~/apps/flight-office$ npm install couch-potato
couch-potato@1.0.2 ./node_modules/couch-potato 
└── blanket@0.1.0

What is the current workaround for this? I tried pointing the dependency at a url like:

 https://github.com/name/repo/tarball/master

but it doesn't seem to update to latest when I run npm update (or npm install).

Owner

isaacs commented Apr 14, 2012

@jesseditson Just install it again. That's all update does, but there's no way for it to know that it's out of date. But, the original need for this should be fixed.

Well there is, if it detects a change in the url(changing the tag), it should be able to detect that and re-install it.

Owner

isaacs commented Apr 14, 2012

Right, so if the package.json's _from attribute doesn't match the url dep, then it should update it, or treat it as an invalid dep. That's why this issue is still open :)

This is a major gotcha for me as well, and I'm sure many others. We depend on git and npm to update our production code, now there is no point to running npm update ever. rm -rf node_modules && npm install is the work around. Kinda lame. :-/

Can't npm update just always reinstall the deps where it's not (easily) determinable whether they need updating? This would help a bunch.

Owner

isaacs commented Nov 29, 2012

@ypocat +1. Patch welcome.

Iristyle commented Jan 9, 2013

Yeah, running into this as well... deploying Node with Capistrano, and pulled a trick to share node_modules from one release to the next to save time on deploys. Some of the packages have to compile, etc, etc.

However, I'm noticing dependencies are not properly updating.

So back to the slower deploys until theres a good method for doing this.

@Iristyle Iristyle referenced this issue in loopj/capistrano-node-deploy Jan 19, 2013

Open

FYI - Npm install won't update git style packages #6

asgrim commented Feb 27, 2013

+1 to fix this.

I currently work around by doing:

$ npm update
$ npm install mygitdep

where mygitdep is something like:

{
    "dependencies": {
        "mygitdep": "git+ssh://git@our.gitlab.server/mygitdep.git"
    },
}
Contributor

ralt commented Mar 9, 2013

As far as I understand, it updates if it's not the same version.

So it seems this is filtered out earlier. Can't see where though. Gut-wise, I think the game is over there.

+1 I've been doing rm -rf node_modules/ as well as a workaround.

Owner

isaacs commented Apr 8, 2013

+1

FWIW, rm -rf isn't necessary. Just npm install <module> will get it updated.

+1

Contributor

luk- commented Apr 9, 2013

I'll have some time this weekend for npm bugs (probably just this one heh). If someone wants to do it before then, feel free.

Contributor

Raynos commented Apr 15, 2013

pro tip: alias ninstall="rm -rf ./node_modules && npm install"

@luk- luk- added a commit to luk-/npm that referenced this issue Apr 15, 2013

@luk- luk- npm install of git dependency always update
This addresses issue #1727. The change ensures that
git dependencies will always update by checking for
_resolved.
8388f0c

preface: @isaacs I almost have the perfect dependency linking solution, but this issue is in the way!

It would be nice if everything could work without requiring a version number, but even with a version number, npm gets confused.

Say in a project I do this:

npm install git://github.com/devinrhode2/historicalConsole.js.git --save

Then I have:

"dependencies": {
    "historical-console": "git://github.com/devinrhode2/historicalConsole.js.git",

Now I make changes to historicalConsole, and npm update - I expect it to see there's a new changes on the git repo and update it. Without a version number or a different version number, nothing happens. When I update the version number in historicalConsole/package.json, this happens:

$ npm update
npm http GET https://registry.npmjs.org/historical-console/0.0.3
npm http 404 https://registry.npmjs.org/historical-console/0.0.3
npm ERR! Error: version not found: 0.0.3 : historical-console/0.0.3
npm ERR!     at RegClient.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-registry-client/lib/request.js:268:14)
npm ERR!     at Request.self.callback (/usr/local/lib/node_modules/npm/node_modules/request/main.js:120:22)
npm ERR!     at Request.EventEmitter.emit (events.js:98:17)
npm ERR!     at Request.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/request/main.js:649:16)
npm ERR!     at Request.EventEmitter.emit (events.js:117:20)
npm ERR!     at IncomingMessage.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/request/main.js:611:14)
npm ERR!     at IncomingMessage.EventEmitter.emit (events.js:117:20)
npm ERR!     at _stream_readable.js:883:14
npm ERR!     at process._tickCallback (node.js:415:13)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

npm ERR! System Darwin 11.4.2
npm ERR! command "node" "/usr/local/bin/npm" "update"
npm ERR! cwd /Users/drhode/repos/shield.js
npm ERR! node -v v0.10.3
npm ERR! npm -v 1.2.17
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/drhode/repos/shield.js/npm-debug.log
npm ERR! not ok code 0

So npm finds the new version number from my git repo.. and then requests it from the npm registry..

Contributor

luk- commented Apr 16, 2013

IMO installing from a git package should always update with whatever’s there.

It looks like there might be two issues getting mixed up in this, I don't think #2419 is the same issue, though they are similar.

  1. npm update will not pull changes from a git repo unless the version number changed, as the OP @felixge described
  2. npm update will look at the registry for modules from a git repo if the version number has changed (which it oddly enough determines by fetching the git repo, checking the version number, then ignoring the rest of the git repo), as @devinrhode2, and @alFReD-NSH describe as well as #2419

nleush commented May 22, 2013

+1

So has the issue with npm update with a git+ssh url trying to fetch the repo from the npm repository been fixed? Its biting me now and if it hasn't then I wouldn't mind diving in.

still occurring on the latest node with npm, and latest npm from the repo (v1.2.27 at this time)

Didn't realize it but I'm kind of waiting for this to get fixed before I keep working on another open source project

@devinrhode2

Fix is here: https://github.com/mattlunn/npm (his latest commit)

Just waiting on review/feedback from izs I think. Until then, just git clone the repo then run make uninstall install (you might need sudo depending on how your normally use npm)

When you need to switch back to the official npm, just update as usual with sudo npm install -g npm

thesmart commented Jun 7, 2013

Amazing. :)

On Thu, Jun 6, 2013 at 5:47 PM, Zeus notifications@github.com wrote:

@devinrhode2 https://github.com/devinrhode2

Fix is here: https://github.com/mattlunn/npm (his latest commit)

Just waiting on review/feedback from izs I think. Until then, just git
clone the repo then run make uninstall install (you might need sudo
depending on how your normally use npm)


Reply to this email directly or view it on GitHubhttps://github.com/isaacs/npm/issues/1727#issuecomment-19083298
.

@distracteddev has your fix been merged yet?

Latest node v0.10.15 seems to work fine.

This should be closed then?

Not working for me with the latest node. To be clear @nakosung you're able to add a dependency like

"dependencies": {
    "thing": "git://github.com/repo/thing.git#56477cb",
}

"npm install" it, then change it like

"dependencies": {
    "thing": "git://github.com/repo/thing.git#67f90b5",
}

npm install again, and it the dependency at the new commit ref is downloaded and appears in your project?

Because I just tried it and that doesn't work for me.

Contributor

mattlunn commented Jul 28, 2013

@pharcosyle: That is fixed in my unmerged PR #3578 (open for a month... I've prodded isaacs to no avail).

Until its merged, just git clone the repo then run make uninstall install (you might need sudo depending on how your normally use npm)

When you need to switch back to the official npm, just update as usual with sudo npm install -g npm

(instructions stolen from isaacs#1727 (comment))


I think @nakosung was talking about another bug which was discussed within this issue thread (#2374), which was fixed in PR #3509

Ahh ok, thanks @mattlunn . Out of curiosity does your PR #3578 also make so when a git dependency targets a branch (like the default master) an npm install will check if there are any new commits on the branch and update it if so?

Contributor

mattlunn commented Jul 28, 2013

@pharcosyle: No, #3578 doesn't cover it.

I couldn't even find an npm issue which describes whether the not-updating behaviour is "by design" (since npm will update the branch if the version within package.json has changed), or whether it is indeed a bug.

If you can point me in the direction of an issue which marks it as a bug, I'd be more than happy to lend a hand submitting a separate PR for it.

I also have not found it described separately. Seems like there are a lot of interrelated issues overlapping here. Hopefully I'll have time to pursue this further soon, I'll let you know

+1 this just bit me today. I thought leaving the version number off would mean it always grabbed master's latest on install/update.

@domenic domenic added a commit that referenced this issue Aug 14, 2013

@mattlunn @domenic mattlunn + domenic Treat dep. as outdated if its _from changes
This is per Isaacs comment in #1727
dcef57f
Member

domenic commented Aug 14, 2013

So. 2 years later. This should be fixed now. By the brave and persistent @mattlunn. w00t!

@domenic domenic closed this Aug 14, 2013

Woot! Thanks guys!

Awesome! Possibly dumb question: what version of npm will have the fix?

Member

domenic commented Aug 15, 2013

The one after the current one, so 1.3.8.

What about if you've got something like

"dependencies": { "package": "git://github.com/username/repo.git#master" }

Does the patch in 1.3.8 cover the situation where master has changed to a different commit than it was last pointed to? Asking cause I just rm -red for this reason.

Contributor

mattlunn commented Aug 23, 2013

Thanks very much @mattlunn. Maybe we can find out whether this is by design in a subsequent thread.

I'm also having the problem where dependencies like the following are not updated properly:

"git+https://git@github.com/fresheneesz/trimArguments.git#578afe0fa6ce96797c36e018bf5bae31b508a02f"

Updating the git commit hash and npm installing still does not change the installed package in version 1.3.8. Should I open another ticket? This is obviously not by design. It prevents any kind of sane package management for git urls.

I think that's one of the points of this issue, stuff like that should work. If any url changes, npm should try to update package.

Member

domenic commented Aug 26, 2013

Ugh, so not only did this fix break everything, but it also doesn't even work? We gotta revert.

@domenic domenic reopened this Aug 26, 2013

strk commented Sep 4, 2013

Same problem here. I reference by commit as an attempt to workaround, but still no dice.

Member

domenic commented Sep 5, 2013

If someone could re-test this with 1.3.10, that would be swell.

potmo commented Sep 8, 2013

+1

potmo commented Sep 8, 2013

EDIT: I was a bit too fast so I removed the last post. Here is an edit.

I'm thinking about fixing this since I really need this for my project. I should say that I'm quite new to npm so all this could be obvious for you.

While investigating this I realize that its quite hard to know how one would like it to be.

This is what I would have thought would happen when have a git dependency:

On a tag:
"dependency-a": "git://github.com/potmo/dependency-a.git#v0.0.7"
I think a tag is always bound to a commit so it's easy to know what to checkout when you do npm install.
When you do npm update I think that the most reasonable thing would to be to go to the last tag rather than the last commit. You stated a tag in the package.json so that is what I would expect to be updated.
Should it even checkout all tags and then see when the package.json version changes?
There should be some kind of connect to the package.json version number shouldn't it?

On a branch:
"dependency-a": "git://github.com/potmo/dependency-a.git#master"
When having a branch as a dependency I guess npm install should always install the HEAD of that branch.
npm update should keep the dependency unchanged but checkout the HEAD commit of the branch.

On a commit:
"dependency-a": "git://github.com/potmo/dependency-a.git#bd9d154bb7261471635776e3e634efdce0736cf8"
npm install is a no-brainer.
When you do npm update what should you update to? We'll it's impossible to know.
You are on detached HEAD so you can't really know what the next step in the history is. I think it would be nice if npm update warned about that to avoid false positives.

Is this the intended behavior?

I can't get npm outdated to work at all on git dependencies but the man page states that "This command will check the registry to see if any (or, specific) installed packages are currently outdated." so that might be intended. I guess the outdated command is not for this discussion anyway.

Member

domenic commented Sep 8, 2013

@potmo thanks for getting the discussion rolling. Here are my thoughts, which I believe align with how npm generally works internally:

  • On a tag: update should not do anything, since updates are just points to commits. You are already as up-to-date as you can be on the referenced version.
  • On a branch: HEAD of that branch is indeed correct.
  • On a commit: just like a tag, update should not do anything.

npm outdated is actually key to making npm update work, as it returns the info npm update needs to do its job.

It would be good to familiarize yourself with the output of npm outdated and npm update on master, as they've recently been upgraded to give a bit more info.

potmo commented Sep 8, 2013

@domenic thanks for your quick reply.
I'm reading though the code right now.

But isn't it a bit strange that update shouldn't up the version when using a tag reference?
Usually when you have a regular registry dependency and you do update, the version changes in your dependencies in package.json to the last available version. Shouldn't the same behavior be reflected when using git version references? That is at least what I would expect.

Tags are attached to commits rather than branches (that makes them kind of "global"). And you can get a list of tags from git with git tag so it would be possible to search for semver compatible tags and pick the latest.

Of course this approach could be a bit messy when there are another version in the actual content of the commits package.json than the tag name. That could really be confusing.
Also, an argument against this idea would be that if tags should be treated just like any other semver string then you should be able to define version ranges and then they are no longer tags... hmmm.

Member

domenic commented Sep 8, 2013

Yes, @isaacs has repeatedly rejected the idea of giving semver-like meaning to tags. Tags in the form v1.2.3 or 1.2.3 should be treated the same as tags in the form abcd or this-is-not-a-version.

potmo commented Sep 8, 2013

Okay. Makes sense.

npm outdated seems to work just as @domenic said three posts ago as long as the new commit has another version in the package.json. The cache updates to the new commit but not the files in node_modules.

So the only change would be to always copy the files from the cache to node_modules when installing git dependencies right? That would cover all the cases.

Member

domenic commented Sep 8, 2013

npm outdated seems to work just as @domenic said three posts ago as long as the new commit has another version in the package.json. The cache updates to the new commit but not the files in node_modules.

Hmm, I think we want something a tiny bit more powerful than that (but I am not sure). In particular, I think if you have no hash, or if your hash points to a branch, we want update to go get the latest commit in the branch. I agree this is theoretically problematic if the actual version number in the newer commit is lower than the older commit, but in practice I think the semantics of #branch should be "latest commit on this branch."

As for why the cache is updated but the files in node_modules are not, that sounds weird. At this point we're getting into implementation details that I'm not sure I understand, and it might be time to bring in @isaacs.

algesten commented Sep 9, 2013

  • On a tag: update should not do anything, since updates are just points to commits. You are already as up-to-date as you can be on the referenced version.

Agree on the behaviour of branch and commit, however a tag can be moved. Whilst I could live with npm not supporting that, I feel making them behave just like branches would be better.

strk commented Sep 9, 2013

+1, tags should behave like branches.

Contributor

guybrush commented Sep 9, 2013

+1, tags should behave like branches since tags can be moved. if you want to pinpoint a git-dependency, use the sha1-hash. it seems very odd in my opinion, when any npm-command which is supposed to keep dependencies up to date does not check for moving git-refs.

but then, modules in the registry also can be published with -f and so their version-numbers could move? so npm should check against checksums of registry-modules too? haha

Member

domenic commented Sep 9, 2013

Thanks for the corrections on tags guys, agree, we should check to see if the tag has moved.

@strk strk referenced this issue in CartoDB/Windshaft-cartodb Sep 11, 2013

Closed

Cannot call method 'indexOf' of undefined #81

@robertkowalski robertkowalski added a commit to robertkowalski/npm that referenced this issue Nov 7, 2013

@robertkowalski robertkowalski Always update git urls
Do directly an update if a git url is specified in the package.json
For outdated, print wanted=git and latest=git

Additionally, fixing a small, still unknown  bug in update,
regarding the positions of where, latest and req, introduced
by 2f7fd62

Fixes #1727
805be15

@robertkowalski robertkowalski added a commit to robertkowalski/npm that referenced this issue Nov 7, 2013

@robertkowalski robertkowalski Always update git urls
Do directly an update if a git url is specified in the package.json
For outdated, print wanted=git and latest=git

Additionally, fixing a small, still unknown  bug in update,
regarding the positions of where, latest and req, introduced
by 2f7fd62

Fixes #1727
39ef02f

@robertkowalski robertkowalski added a commit to robertkowalski/npm that referenced this issue Nov 8, 2013

@robertkowalski robertkowalski Always update git urls
Do directly an update if a git url is specified in the package.json
For outdated, print wanted=git and latest=git

Additionally, fixing a small, still unknown  bug in update,
regarding the positions of where, latest and req, introduced
by 2f7fd62

Fixes #1727
05d14e1

@robertkowalski robertkowalski added a commit to robertkowalski/npm that referenced this issue Nov 15, 2013

@robertkowalski robertkowalski Always update git urls
Do directly an update if a git url is specified in the package.json
For outdated, print wanted=git and latest=git

Additionally, fixing a small, still unknown  bug in update,
regarding the positions of where, latest and req, introduced
by 2f7fd62

Fixes #1727
89381ab

@robertkowalski robertkowalski added a commit to robertkowalski/npm that referenced this issue Nov 15, 2013

@robertkowalski robertkowalski Always update git urls
Do directly an update if a git url is specified in the package.json
For outdated, print wanted=git and latest=git

Additionally, fixing a small, still unknown  bug in update,
regarding the positions of where, latest and req, introduced
by 2f7fd62

Fixes #1727
46c7ebe
Member

robertkowalski commented Nov 25, 2013

should work now. landed as 5829ecf

glukki commented Nov 30, 2013

Awesome, thx @robertkowalski ! :)
But, it may be an issue for someone, that npm update replaces links in node_modules made with npm link <pkgname>.

@andreypopp andreypopp pushed a commit to andreypopp/jsxx that referenced this issue Feb 2, 2014

@zpao zpao Use github tarball link for esprima dependency
It turns out that (at least for local development) npm has a long
standing bug where it doesn't recognize changing dependencies stored as
git urls (see npm/npm#1727). Luckily npm
understand tarballs and GitHub provides tarballs for every commit, so
the workaround is easy, though unfortunate.
bd044fc

@swalkinshaw swalkinshaw referenced this issue in capistrano/npm Mar 8, 2014

Closed

Put node_modules in linked_dirs? #7

vmkcom commented Mar 12, 2014

thx for updating git repos!

But, how about mercurial? I have a mercurial repo stored at bitbucket.org
In package.json used link like that:
https://{user}:{pass}@bitbucket.org/{user}/{package}/get/default.tar.gz

aearly commented Mar 12, 2014

Bitbucket supports git. Have you considered converting your repo?

@aearly @vmkcom This is not about mercurial or git, that is a gziped tarbal url, which npm supports it. I think you shouldn't have any problem with those.

vmkcom commented Mar 13, 2014

@aearly @alFReD-NSH I have problems with update my custom library from gziped tarball url. This library not updated by npm update in 1.4.4
"mylib": "https://mylib.com/mylib.tar.gz" in package.json

if I try to npm update or npm update mylib I see npm http 404 https://registry.npmjs.org/mylib
npm doesn't request url from package.json and try to search at npm registry

in 5829ecf added isGitUrl function, and these urls always updated from given git url

can we have something like isGitUrl for all tarball urls? Say, special query or hash parameter

@vmkcom Either open a new issue, or wait for one of the maintainers re-open this issue.

qzaidi commented Jun 4, 2014

This change breaks something for me. I have multiple dependent packages having a common dependency.

A
|---B
| |---D
|---C
| |---D
|---D

D comes from a git repo (isGit is true). Prior to this change (5829ecf), I would end up with a single copy of git. I was sort of depending on this behavior, and it says something like that in npm help install.

npm help install
....
 To avoid this situation, npm flat-out refuses to install any name@version that is already present  anywhere  in  the  tree  of  package folder  ancestors.  A more correct, but more complex, solution would be to symlink the existing version into the new location.  If this ever affects a real use-case, it will be investigated.

My packages are private, so I can't publish them to public npm repo. Tarballs or private npm repo are an option for now, but is there any other alternative?

So you probably shouldn't be depending on that behavior. Other packages that require different versions of D could eff things up. If you're depending on, for example, instanceof to work on a member of the D module, you will likely mess stuff up when other packages require a different version of D.

Whats the reason you're depending on that?

qzaidi commented Jun 5, 2014

B and C can be installed stand alone, or they can be installed as a dependency (of A). When they are installed as a dependency of A, I want all of them to share the same instance of D. Due to require caching, I have then one exported object shared between all packages, and changes to that object are reflected everywhere.

While this behavior persists on npm and tarball packages, this breaks if the package is installed from a git repo. Isn't this going to be a confusing side effect of the above change. The situation described in help for npm install (Limitations of npm Install algorithm) can potentially happen and the doc needs to be updated too. For me, a work around is to run npm dedupe, or to not install from git (setup tarballs or a private repo).

The question is: why do you want them to share the same instance of D? This sounds an awful lot like a global variable with the usual problems.

qzaidi commented Jun 5, 2014

D has a pool of connections to the DB. Rather than have A B and C have
different pools (each with a certain number of db connections), I prefer to
have a single pool - scales better in our scenario.

On Thu, Jun 5, 2014 at 2:01 PM, fresheneesz notifications@github.com
wrote:

The question is: why do you want them to share the same instance of D?
This sounds an awful lot like a global variable with the usual problems.


Reply to this email directly or view it on GitHub
#1727 (comment).

I would recommend creating a way to pass a connection pool to D. That way you can share the same connection pool, not only with D, but with any module that needs it.

qzaidi commented Jun 6, 2014

I already do that, but I will have to do it at multiple places now (in A, B
and C), since now there are multiple instances of D and a pool has to be
passed to all 3.

On Fri, Jun 6, 2014 at 1:39 AM, fresheneesz notifications@github.com
wrote:

I would recommend creating a way to pass a connection pool to D. That way
you can share the same connection pool, not only with D, but with any
module that needs it.


Reply to this email directly or view it on GitHub
#1727 (comment).

True, but doing it this way is almost definitely the best way to ensure that other people can use your modules effectively and efficiently. In the cases where modules are using different versions of D, or where a module is using A, B, or C but doesn't require D, it will allow them to still have a single connection pool. Relying on the require cache to handle this for you doesn't allow that. Anyways, I'm sure you'll figure out how to best handle this for your modules.

the temp fix from @asgrim works for me.

roddik commented Jan 13, 2016

Any plans to finally fix this?

@sagalbot sagalbot referenced this issue in sagalbot/vue-select Apr 5, 2016

Closed

How can I install the lastest version ? #25

Why is this closed, it still does not work as expected.

dgreensp commented May 26, 2016 edited

npm install seemingly won't update a package specified by git URL, ever; I changed the semver from 0.1.0 to 10.0.0 and specified a new commit hash, and nothing, with npm 3.9.3.

Edit: Correction, changing the major version seems to have worked. So I think the current behavior is npm install will update if the git URL changes, and it points to a different semver version, AND that semver version is incompatible with the current one; but I'm not positive.

yorch commented Jul 22, 2016 edited

Same boat here. Using a tarball URL to install a package that gets updated frequently, but NPM does not take the newest version, even thought the new version is already in the local cache (~/.npm/{package-name}/) unless I run npm install {package}.

I'm currently using 3.10.5, but I can swear some older NPM version (maybe 2.x) was actually updating it automatically. In order to make that old NPM version work like that, I had to send a dynamic Last-Modified header when serving the tarball.

hayesmaker commented Aug 4, 2016 edited

same here.. can't get npm install to install anything newer than a 13 day old commit.. we've had 20 commits since then.. we have to specify the commit hash each time we change the dependency for this to work properly?

ahauser31 commented Sep 13, 2016 edited

I have the same issue... And in my case, it's quite confusing: I have a github open source public dep and a bitbucket private dep, both managed by me and with the package.json set up exactly the same way. I npm install, it fetches both repos. I make changes to both repos, push them to github and bitbucket and run npm update. It will only find the changed github repo. If I run npm update multiple time, it will each time "update" the github dep, even tho there is no change and the new version info / commit hash is stored in the tags that npm creates in the package.json of my dep in the node_modules.

I guess my question is: Why is there a difference between github and bitbucket (both git repos, forgot to say... no tarball or mercurial etc.)?
And why is the github dep "updated" each time I run npm update even tho there is no change?

Edit:
To clarify, when I added both dependencies, I did not specify any version numbers, commit hashes, etc. that might cause npm to only use this specific version, i.e. I ran e.g npm i git+ssh://git@bitbucket.org:XYZ/XXYYZZ.git -S and npm i git+ssh://git@github.com:ABC/AABBCC.git -S

Edit2:
While it's a bit weird that is always re-downloads the github dep, I rather this would be default for bitbucket as well. As it stands, this inconsistency is making it hard to predict how and when things are updated.

I think I have the same issue. I have a shared library dependency which project X uses.

package.json
...
"dependencies": {
    "my-project": "bitbucket:my-username/my-project"
  }
...

npm update doesn't get the newest version.

I have to manually write into the command line:

npm install bitbucket:my-username/my-project

So right now, my build tools are kind of dead, however, this seems like an old problem and I am wondering if I am just approaching it wrong.

@islemaster islemaster referenced this issue in code-dot-org/code-dot-org Oct 24, 2016

Merged

Use code-dot-org/p5.play #11305

In #3328 (and also somewhere above in this massive thread) there is a suggestion to use branch references such as #master as follows:

package.json
...
  "dependencies": {
    "my-project": "bitbucket:my-username/my-project#master"
  }
...

In my usage with v3.10.8, npm update appears to correctly install the latest commit pushed to the specified branch when the local copy falls behind the remote. Perhaps that will help the situations seen by @clark-stevenson and @ahauser31 ?

There doesn’t appear to be any built in support for versioning of git based dependencies, however a manual strategy such as maintaining and referencing semver style branches such as [n].x.x and ensuring backwards compatibility within those branches could help avoid exposing consumers to breaking changes. Similarly, tagging each specific release allows consistent referencing / locking of minor versions.

If you follow the thread further in #3328 there is the suggestion that support for #1.x.x tag based semver style versioning similar to bower's may be coming for git dependencies. Registry support for private packages and namespaces may make this even less of a priority but… watch that space I guess.

aneilbaboo commented Feb 1, 2017 edited

This is slightly off topic (please chime in if you think I should create a separate issue):

It would be nice to specify the semversion you want from a github url. It would need a new bit of syntax. I'm thinking something like:

"somepackage": "git@github.com:developer/somepackage.git|0.2.x"

The idea is that npm would fetch only package.json from the repo, check and then automatically download and install whatever it finds there if the version number has changed.

This inherits the useful features of the existing versioning mechanism, and some:

  1. Package author controls when new functionality is ready to be used
  2. Package user can accept bug fixes and avoid large / breaking changes to the API
  3. From an efficiency standpoint, it will have the same network overhead as public packages
  4. No new data (e.g., commit hashes as in some of the proposals above) need to be stored
  5. It should be easy to implement, since the mechanisms are already in place

I haven't spelunked NPM, but isn't the change as simple as:

  1. Add/modify code to parse this style of URI
  2. Pass the github url and semversion to existing npm functions

?

kirill-konshin commented Feb 1, 2017 edited

@aneilbaboo you can use the following:

"package": "git+http://gtihub.com/user/package.git#0.1.11"

After slash there could be a tag, branch, or even SHA of the commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment