Look for node_modules in any recursive parent directory #44

ariofrio · 2018-09-01T08:21:39Z

Resolves #43.

~~This needs tests at the moment.~~ Tests added.

raphinesse · 2018-09-02T01:11:38Z

It turns out that when using resolve-from, Node's Module._pathCache is causing more trouble than it's worth. So I've added an implementation using resolve. Thanks for suggesting the package!

Do you think you could write a few tests for this, @ariofrio? That would be awesome! I think a few unit tests for pathToInstalledPackage with some local fixtures should be enough.

raphinesse

Needs test as described in my previous comment.

ariofrio · 2018-09-05T20:18:34Z

Thanks for working on this, @raphinesse! I'm pretty swamped right now; hopefully I can find some time for in a week or so to write a few tests for this.

raphinesse · 2018-09-12T20:14:17Z

@ariofrio FYI we're in the process of preparing the next major release. If we want this change in it, we should start work on it rather sooner than later. I'm sorry to say I can give no specific time frame though.

ariofrio · 2018-09-20T17:41:51Z

Working on this now. Not sure what a unit test for pathToInstalledPackage that mocks resolve would accomplish, since most of the functionality that needs to be tested depends on the way resolve works.

I have in mind an "integration" test that creates this directory structure:

tmpdir/
  node_modules/
    dummy-local-plugin/
  nested-directory/

And then tries to run pathToInstalledPackage to find dummy-local-plugin from 'tmpdir' and then from 'tmpdir/nested-directory'.

What do you think?

Update: Committed a version of this. Verified locally that the new tests fail on master. The first test succeeds because it tests finding the package in the current directory, which already worked.

raphinesse · 2018-09-20T21:43:50Z

The tests look good to me on a first glance 👍

I never expected resolve to be mocked. How you laid out the tests was exactly how I imagined it.

One last thing: it would be nice to add a test that shows that globally installed modules will be taken into account too. Not sure how to best test this, but maybe you have an idea?

raphinesse · 2018-09-20T22:30:04Z

Or it might have been the linting errors that show up in the CI logs 😅

ariofrio · 2018-09-20T22:35:14Z

Huh, is taking into account globally installed modules expected behavior?

Seems like it would be error-prone, since running the same command on a different environment without that globally installed module would fail.

Not sure if resolve provides that functionality, anyway, reading the docs.

raphinesse · 2018-09-20T22:40:58Z

While I agree that taking global modules into account might be problematic, I would have thought resolve covers that functionality since it's part of the node module resolution algorithm.

Any way: all the more reason to assert the presence/absence of this behavior in the current implementation.

ariofrio · 2018-09-20T22:53:43Z

Good point about needing to test it either way.

Research and links about globally-installed modules

At least on my environment, requiring a module that's installed globally doesn't work.

It seems that it's not supposed to work, unless NODE_PATH is set or the package is installed to a fixed set of three GLOBAL_FOLDERS (on Ubuntu using the nodesource distribution, globally installed modules don't go into any of them, but into /usr/lib/node_modules).

A grep through the source code seems to indicate that resolve doesn't read NODE_PATH or check the GLOBAL_FOLDERS, but expects them to be passed as the paths argument.

So to achieve full fidelity, we'd probably need to pass NODE_PATH and those three directories to resolve. Even then, in common cases globally installed modules won't be accessible.

Certainly doable. Looking for use-cases to work off of, it seems RisingStack and other use NODE_PATH to avoid relative paths in require(), but they don't use it with whole packages.

Bottom line, globally installed modules probably shouldn't work on node, but it depends on whether the node environment is setup to install global packages to one of GLOBAL_FOLDERS. I don't know if we can assume that modern distributions avoid that, though this thread suggests we can.

Supporting NODE_PATH would make cordova-fetch fully compatible with npm resolution algorithm AFAIK, and the test for it seems straightforward.

ariofrio · 2018-09-20T23:37:12Z

Added NODE_PATH support and tests.

This is the code that generates those GLOBAL_FOLDERS. It's straightforward to replicate this functionality in cordova-fetch, but the way it finds the node_prefix has changed because the node binary moved, and I don't know if it could in the future.

Unfortunately, globalPaths seems to be undocumented, though it can be accessed through require('module').globalPaths in all node versions thus far, as far as I can tell. It can also be accessed by looking at the last three items of module.path.

All of these options have drawbacks, and supporting GLOBAL_FOLDERS would be "mostly for historic reasons". @raphinesse, up to you whether to support them and which option has the best trade-offs for the cordova project.

As far as testing global modules, I only see two ways of testing it:

actually install a global module and uninstall it afterwards, which could be disruptive and require sudo on some systems
use NODE_PATH to simulate global modules, but this is indistinguishable from testing NODE_PATH support

raphinesse · 2018-09-24T16:17:53Z

Great research @ariofrio! 🏅

I think supporting local lookup plus NODE_PATH lookup is just fine. No need for the hard-coded GLOBAL_FOLDERS. I will do a code review and then we need to run some tests with cordova-lib using this version.

Tests have been added as requested

raphinesse · 2018-09-25T10:25:41Z

Alright! I just ran the automated tests of cordova-lib using this version of cordova-fetch and everything was fine.

raphinesse

All good IMHO. Commits should be squashed on merge.

raphinesse · 2018-09-25T16:00:17Z

So after really putting this to work to do dependency prefetching for the cordova-lib test suite, one problem popped up. I added a regression test and a fix for it.

erisu · 2018-09-28T04:44:45Z

index.js

@@ -22,7 +22,7 @@ var events = require('cordova-common').events;
 var path = require('path');
 var fs = require('fs-extra');
 var CordovaError = require('cordova-common').CordovaError;
-const { getInstalledPath } = require('get-installed-path');
+const resolve = Q.denodeify(require('resolve'));


Not an issue but I know there was an initiative to remove the Q dependency and use Native promises. But this is no problem for now.

apache/cordova#7

Everything looks good IMO.

ariofrio · 2018-09-28T19:14:50Z

Great! Thanks @raphinesse and @erisu for taking the time to work with me on this!

raphinesse · 2018-09-28T19:56:29Z

Thanks for your contribution @ariofrio!

ariofrio added 2 commits September 1, 2018 01:20

Look for node_modules in any recursive parent directory

f7fb64a

Fix typos

acb1c6e

raphinesse force-pushed the patch-1 branch from 3d53c2a to acb1c6e Compare September 2, 2018 00:47

Use resolve package to implement module lookup

a5d29c5

janpio added this to 🐣 New PR / Untriaged in Apache Cordova: Tooling Pull Requests Sep 5, 2018

janpio moved this from 🐣 New PR / Untriaged to ⏳ Waiting for Review in Apache Cordova: Tooling Pull Requests Sep 5, 2018

Apache Cordova: Tooling Pull Requests automation moved this from ⏳ Waiting for Review to 🙅 Pending Approval Sep 5, 2018

raphinesse previously requested changes Sep 5, 2018

View reviewed changes

add pathToInstalledPackage tests for finding packages in ancestor dirs

b5f9425

oops, don't only run the pathToInstalledPackage tests

9769cb8

fix lint issue

995577b

add support for NODE_PATH + tests

d9fa631

raphinesse added 7 commits September 24, 2018 18:46

Move pathToInstalledPackage tests to fetch-unit.spec

1ebbbf9

Fix catching of failed expectation in test

733752a

Cosmetic changes

2e8b3d5

Factor out package path resolving

01a4d64

Change tests to only test resolvePathToPackage

90fccbd

Check returned package.json in tests

77328b5

Improve test helper name

f675216

Apache Cordova: Tooling Pull Requests automation moved this from 🙅 Pending Approval to ✅ Approved, waiting for Merge Sep 25, 2018

raphinesse approved these changes Sep 25, 2018

View reviewed changes

raphinesse requested review from erisu and dpogue September 25, 2018 10:28

raphinesse added 2 commits September 25, 2018 17:48

Add test to ensure proper installation destination

4c0ce92

Ensure that we install to dest and not any of its ancestors

3e850e0

raphinesse self-assigned this Sep 25, 2018

This was referenced Sep 26, 2018

How to handle save: true when requested package is already installed #47

Open

Cordova 9 release plan apache/cordova#10

Closed

erisu approved these changes Sep 28, 2018

View reviewed changes

Clarify comment

553abaa

raphinesse merged commit f37c7ba into apache:master Sep 28, 2018

Apache Cordova: Tooling Pull Requests automation moved this from ✅ Approved, waiting for Merge to 🏆 Merged, waiting for Release Sep 28, 2018

raphinesse mentioned this pull request Sep 29, 2018

Handle broken NODE_PATH setups gracefully #50

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Look for node_modules in any recursive parent directory #44

Look for node_modules in any recursive parent directory #44

ariofrio commented Sep 1, 2018 •

edited

raphinesse commented Sep 2, 2018 •

edited

raphinesse left a comment

ariofrio commented Sep 5, 2018

raphinesse commented Sep 12, 2018

ariofrio commented Sep 20, 2018 •

edited

raphinesse commented Sep 20, 2018

raphinesse commented Sep 20, 2018

ariofrio commented Sep 20, 2018 •

edited

raphinesse commented Sep 20, 2018

ariofrio commented Sep 20, 2018 •

edited

ariofrio commented Sep 20, 2018

raphinesse commented Sep 24, 2018

raphinesse commented Sep 25, 2018

raphinesse left a comment

raphinesse commented Sep 25, 2018

erisu Sep 28, 2018

erisu Sep 28, 2018

ariofrio commented Sep 28, 2018

raphinesse commented Sep 28, 2018

Look for node_modules in any recursive parent directory #44

Look for node_modules in any recursive parent directory #44

Conversation

ariofrio commented Sep 1, 2018 • edited

raphinesse commented Sep 2, 2018 • edited

raphinesse left a comment

Choose a reason for hiding this comment

ariofrio commented Sep 5, 2018

raphinesse commented Sep 12, 2018

ariofrio commented Sep 20, 2018 • edited

raphinesse commented Sep 20, 2018

raphinesse commented Sep 20, 2018

ariofrio commented Sep 20, 2018 • edited

raphinesse commented Sep 20, 2018

ariofrio commented Sep 20, 2018 • edited

ariofrio commented Sep 20, 2018

raphinesse commented Sep 24, 2018

raphinesse commented Sep 25, 2018

raphinesse left a comment

Choose a reason for hiding this comment

raphinesse commented Sep 25, 2018

erisu Sep 28, 2018

Choose a reason for hiding this comment

erisu Sep 28, 2018

Choose a reason for hiding this comment

ariofrio commented Sep 28, 2018

raphinesse commented Sep 28, 2018

ariofrio commented Sep 1, 2018 •

edited

raphinesse commented Sep 2, 2018 •

edited

ariofrio commented Sep 20, 2018 •

edited

ariofrio commented Sep 20, 2018 •

edited

ariofrio commented Sep 20, 2018 •

edited