Require files outside of the project's node_modules folder

[simov]

👋 I have a bunch of modules required in my project that are outside of my project's node_modules folder, I know that's highly unusual, but that also works perfectly fine with Node because it looks for files in various places up to the file systems's root. However when I run my app through the devtool I get this error message:

Error: Cannot find module '@xsql/fixtures'
    at Function.Module._resolveFilename (module.js:336)
    at Function.Module._load (module.js:286)
    at Module.require (module.js:365)
    at require (module.js:384)
    at Object.<anonymous> (index.js:18)
    at Module._compile (module.js:434)
    at Object.devtoolCompileModule [as .js] (/media/SSD/github/devtool/lib/require-hook.js:36)
    at Module.load (module.js:355)
    at Function.Module._load (module.js:310)
    at Module.require (module.js:365)
    at require (module.js:384)
    at EventEmitter.<anonymous> (/media/SSD/github/devtool/lib/preload.js:60)
    at emitOne (events.js:77)
    at EventEmitter.emit (events.js:169)

That's a module that's not directly in my project's node_modules folder, it's a few levels up in the file system. So I'm suspecting that this is something Electron specific, but I'm not that familiar with it and have no clue.

[mattdesl]

Thanks for the bug report.

Just so I'm clear, your folder structure looks like this?

~/
  my-app/
    index.js
    node_modules/
      foo/
  node_modules/
    bar/

So I guess foo is getting resolved, but bar is not found?

[simov]

I have modules in:

~/modules/my-module/node_modules
~/root/node_modules
~.node_modules

And I'm trying to run my-module devtool is somewhere in ~/.nvm/....

Modules outside of the ~/modules/my-module/node_modules result in the above error.

[simov]

Here is how Node load modules from node_modules

It searches in the .node_modules as well.

This is my development setup and it won't be easy for me to change it like it's suggested in the docs.

[mattdesl]

I don't think that should work. As it says in the docs, it just looks up folders recursively until it sees a node_modules folder. If it reaches the top (~/) before that, then it fails.

In your case, it should only look inside ~/node_modules. So you should not be able to resolve my-module like that.

[simov]

Here is how I run my module:

mighty@sky:~/modules/sql-admin$ node index.js

Here is how I run it using devtool:

mighty@sky:~/modules/sql-admin$ devtool index.js -s

I have modules located UP in the file system tree:

mighty@sky:~/modules/sql-admin/node_modules$
mighty@sky:~/root/node_modules$
mighty@sky:~/.node_modules$

Node find modules in all three places, devtool does not. Node searches in the nearest node_modules folder first. Then it moves one level above and tries again. And this isn't something new.

[mattdesl]

I just ran a test with node v4 and node isn't finding modules in ~/root/node_modules/. Is that your $HOME or something?

And yeah, it seems like an Electron bug since this happens even without the devtool require hooks.

[simov]

It's all in my $HOME ~, it points to my user's home folder. All of my files are in my home folder.

From the docs:

For example, if the file at '/home/ry/projects/foo.js' called require('bar.js'), then Node.js would look in the following locations, in this order:

/home/ry/projects/node_modules/bar.js
/home/ry/node_modules/bar.js
/home/node_modules/bar.js
/node_modules/bar.js

So this is exactly what is happening, Node searches up to the root of the directory tree.

I'll take a further look at your code and I'll try to fix it. Otherwise devtool seems useful to me, but I can't use it that way. It might be something Electron related as well, because node-webkit for example requires all modules to be in the project's node_modules if I remember correctly.

[simov]

Hmm wait. All my paths are actually symbolic links, so the actual paths are as follows:

mighty@sky:/media/SSD/Dropbox/code/modules/sql-admin$
mighty@sky:/media/SSD/node_modules$

So they are in fact under the same directory. Anyway, the point being is that Node searches recursively up to the top until it finds the module.

[simov]

Ok, so it seems that only the ~.node_modules path isn't working.

It seems that the entry point (my example file) 1 (which is also located in a symbolic link location) is being resolved before generating all search paths in 2 that are the real paths.

So my test file located in /media/SSD/tmp/devtool.js have no problem finding the /media/SSD/node_modules/extend/index.js which is UP one level, but finding the /home/mighty/.node_modules is a problem because the path has already been resolved.

Great job anyway, the tool is looking really good!

[mattdesl]

Thanks for testing. 😄

Hopefully we can figure out a fix for the ~/.node_modules path.