New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seems like Meteor doesn't detect dependencies in linked modules. #7154

Closed
trusktr opened this Issue Jun 2, 2016 · 13 comments

Comments

Projects
None yet
4 participants
@trusktr
Contributor

trusktr commented Jun 2, 2016

All works fine until I npm link a package and run npm prune. I'm using NPM v3. So, before linking, dependencies are flat. After running npm link <package-name> then npm prune, the flat dependencies are removed because they exist inside the linked package. In my case, a linked package depends on babel-runtime, and although that linked package contains babel-runtime in it's node_modules folder, Meteor throws this error:

W20160601-22:33:53.007(-7)? (STDERR)
W20160601-22:33:53.023(-7)? (STDERR) /Users/trusktr/.meteor/packages/meteor-tool/.1.3.2_4.1pv1bhm++os.osx.x86_64+web.browser+web.cordova/mt-os.osx.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:267
W20160601-22:33:53.023(-7)? (STDERR)                                            throw(ex);
W20160601-22:33:53.024(-7)? (STDERR)                                                  ^
W20160601-22:33:53.024(-7)? (STDERR) Error: Cannot find module 'babel-runtime/core-js/promise'
W20160601-22:33:53.024(-7)? (STDERR)     at Function.Module._resolveFilename (module.js:338:15)
W20160601-22:33:53.024(-7)? (STDERR)     at Function.Module._load (module.js:280:25)
W20160601-22:33:53.024(-7)? (STDERR)     at Module.require (module.js:364:17)
W20160601-22:33:53.024(-7)? (STDERR)     at require (module.js:380:17)
W20160601-22:33:53.025(-7)? (STDERR)     at Object.<anonymous> (/Users/trusktr/src/MightyDevs+website/meteor-app/node_modules/awaitbox/meteor/startup.js:7:16)
W20160601-22:33:53.025(-7)? (STDERR)     at Module._compile (module.js:456:26)
W20160601-22:33:53.025(-7)? (STDERR)     at Object.Module._extensions..js (module.js:474:10)
W20160601-22:33:53.025(-7)? (STDERR)     at Module.load (module.js:356:32)
W20160601-22:33:53.025(-7)? (STDERR)     at Function.Module._load (module.js:312:12)
W20160601-22:33:53.025(-7)? (STDERR)     at Module.require (module.js:364:17)

I'm in Meteor 1.3.2.4.

@trusktr

This comment has been minimized.

Contributor

trusktr commented Jun 2, 2016

Will post a repro when I can.

@abernix

This comment has been minimized.

Member

abernix commented Jun 2, 2016

First of all, Meteor 1.3.3 uses npm 2.15.1 and will not expect its node_modules to be setup as v3 would have them – I could imagine some incompatibility here.

Can you at least post a command play-by-play and explain what you are trying to accomplish? Why are you running npm prune? Do you have the npm link'd package in your package.json? Can you include your package.json and the output of meteor npm ls? Separately npm ls too, might be useful. What happens if you use try using npm v2 to accomplish what you're trying to do, does it work?

@abernix

This comment has been minimized.

Member

abernix commented Jun 2, 2016

(edited previous message to add a missing "not") 😉

@trusktr

This comment has been minimized.

Contributor

trusktr commented Jun 6, 2016

Running npm prune is useful to clear package that may have been temporary (for example, we don't commit a change, or we switch branches). In theory, npm prune removes unused packages. In my case, it removes packages that were direct children in node_modules because the linked package contains the dependencies. So, in NPM's mind, all the dependencies exists (they are nested in a linked package, so they do). Meteor simply would need to follow this case so that NPM users don't run into these hitches (since Meteor supports NPM and npm link is a prime tool that many will inevitably be using at some point in their Meteor apps).

@clayne11

This comment has been minimized.

clayne11 commented Jun 8, 2016

npm link doesn't always work though in general IME. I've found that I sometimes end up with duplicate packages - some in the linked node_modules folder and some in the main app's node_modules folder.

@trusktr

This comment has been minimized.

Contributor

trusktr commented Jun 9, 2016

@clayne11 That's true. Usually I run npm prune after npm link to remove the app's duplicate dependencies. npm prune will notice this and clean it up.

@trusktr

This comment has been minimized.

Contributor

trusktr commented Jun 9, 2016

When dependencies get cleaned up by npm prune, that's what makes this Meteor bug show itself because Meteor isn't looking in the linked module for dependencies.

@clayne11

This comment has been minimized.

clayne11 commented Jun 9, 2016

@trusktr that's good to know - thanks for the tip!

@abernix

This comment has been minimized.

Member

abernix commented Jun 27, 2016

meteor npm link should work fine, however as I previously stated, the NPM version inconsistencies are high likelihood for issue here since NPM v2 and v3 manage/store/discover dependencies in entirely different ways – though you didn't respond with anything that I asked for in order to further debug this.

Regardless, I'm curious if your problem still exists if you bring your project up to Meteor 1.3.4.1 as Meteor's internal NPM has been upgraded from v2.15.1 to v3.9.6.

@trusktr

This comment has been minimized.

Contributor

trusktr commented Jun 27, 2016

the NPM version inconsistencies are high likelihood for issue here since NPM v2 and v3 manage/store/discover dependencies in entirely different ways

That may be true, but once dependencies are in the filesystem inside node_modules, it is Node.js that searches for the dependencies, and the algorithm (I believe) is the same no matter which version on NPM was used to get the modules onto the filesystem. NPM 3 just happens to make things more flat, but that flatness doesn't (as far as I know) affect Node.js' module-loading algorithm, which checks leaf modules first, then traverses up the tree (in the case of flat modules from NPM3, Node.js just ends up traversing all the way up to the root of node_modules).

If Meteor can also find modules the same way as Node.js (i.e. use basically the same algo to discover modules, including whatever Node.js does with links) then the problem should go away.

@abernix

This comment has been minimized.

Member

abernix commented Jun 28, 2016

You're talking about running npm commands using different major versions of npm and that's what I'm referring to as the problem. I don't believe the problem is with Node finding your modules, I just don't believe they are actually where they are expected to be after you run npm prune (since I'm assuming it works fine before you run npm prune) but unfortunately you still haven't provided anything I've asked for to help you in debugging this so it's still hard to say.

Meteor's code to resolve packages is pretty simple. It uses Node's require.resolve which has obeyed the same rules for a long while.

As previously requested, please post a reproduction of your problem or explicit reproduction steps. I'm not asking for explanations on how NPM or Node works.

@abernix

This comment has been minimized.

Member

abernix commented Jul 4, 2016

Closing due to lack of reproduction. Happy to re-open when a reproduction is ready.

@abernix abernix closed this Jul 4, 2016

@worldsayshi

This comment has been minimized.

worldsayshi commented Nov 2, 2016

I've created a way to reproduce a similar or perhaps identical problem:
https://github.com/worldsayshi/meteorRequireProblems

benjamn added a commit that referenced this issue Nov 4, 2016

Use absolute paths for external symlinks in Builder#copyDirectory.
Thanks to @worldsayshi for reporting this issue with a reproduction.

Fixes #8005.
Fixes #2876.
Fixes #7154.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment