Too many dependencies break the Windows file system

[mcartoixa]

On Windows, the file system itself cannot seem to cope very well with a too deeply nested node_modules directory...

We just added bower to our dependencies, and now:

• the node_modules directory cannot be deleted from a batch command: DEL /S /Q /F node_modules fails.
• CLOC fails, even when told to ignore the node_modules directory completely.
• Git clean fails C:\PROGRA~2\Git\bin\git.exe clean -d -f -x.
• MSBuild item wildcards are not replaced anymore.

We have tried to run npm dedupe to flatten our hierarchy but this is not enough.

Effectively our Continuous Integration and delivery platform is broken for node.js/bower projects. Other people seem to have encountered the same kinds of problems on Windows.

Any heads up? A more efficient dedupe command, perhaps?

Using npm 1.3.2 on node.js 0.10.13. Reproduced on Windows 8 (x64) and Windows Server 2008 R2 (x64).

[dominictarr]

oh, I've been experimenting with a new npm client, https://github.com/dominictarr/npmd
you can do npmd install X --greedy and it installs everything directly into node_modules,
only going deeper if there is a version conflict.

This is all pretty experimental, and npmd is very young, so no promises, but it might fix your problem.

[ralt]

This sounds like a huge issue. Is it a problem that npm cannot solve? Can we determine exactly what corrupts the filesystem? Is there no workaround? (Like switching an option on or off in ntfs.)

The issue seems serious enough that time should be spent on it. Having no windows, I can't help, but from an external point of view, this looks serious.

[domenic]

The file system is not corrupted. You just need to manipulate the files in question using the GUI or using UNC paths instead of the command line and traditional paths.

[mcartoixa]

Maybe my wording is wrong, as could be my understanding of the problem.
But the symptoms remain:

• the GUI is not a very practical solution in continuous integration scenarii.
• node.js tools like npm or grunt can fail too on our systems due to the mere presence of these folders (even on unrelated projects in different directories).
• I am not sure I could rewrite all our scripts to use UNC paths instead of traditional paths.

@domenic Do you have more insight (maybe links) as to why UNC paths would work? I have once had to manually rename a whole hierarchy of folders to single letter names so that they could be deleted, even using the GUI...

[domenic]

@mcartoixa http://msdn.microsoft.com/en-us/library/Aa365247

My impression was that libuv does this prefixing of "\\?\" transparently for you and so node programs should never run in to this problem. I have not verified however.

[mcartoixa]

Here's an excerpt of a failing npm install output, due to filesystem related issues:

unbuild isogeo-id@0.0.1
error : error rolling back Error: ENOTEMPTY, rmdir 'D:\ccnet\projects\Isogeo.App\ContinuousBuild\work\node_modules\isogeo-id\node_modules\express-hbs\node_modules\glob'
error : error rolling back express-hbs@0.2.0 { [Error: ENOTEMPTY, rmdir 'D:\ccnet\projects\Isogeo.App\ContinuousBuild\work\node_modules\isogeo-id\node_modules\express-hbs\node_modules\glob']
error : error rolling back errno: 53,
error : error rolling back code: 'ENOTEMPTY',
error : error rolling back path: 'D:\\ccnet\\projects\\Isogeo.App\\ContinuousBuild\\work\\node_modules\\isogeo-id\\node_modules\\express-hbs\\node_modules\\glob' }
npm http GET https://registry.npmjs.org/i/-/i-0.3.2.tgz
error : Error: ENOENT, lstat 'D:\ccnet\projects\Isogeo.App\ContinuousBuild\work\node_modules\isogeo-id\node_modules\express-hbs\node_modules\glob\README.md'
error : If you need help, you may report this log at:
error : <http://github.com/isaacs/npm/issues>
error : or email it to:
error : <npm-@googlegroups.com>
error : System Windows_NT 6.1.7601
error : command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install"
error : cwd D:\ccnet\projects\Isogeo.App\ContinuousBuild\work\build
error : node -v v0.10.13
error : npm -v 1.3.2
error : path D:\ccnet\projects\Isogeo.App\ContinuousBuild\work\node_modules\isogeo-id\node_modules\express-hbs\node_modules\glob\README.md
error : fstream_path D:\ccnet\projects\Isogeo.App\ContinuousBuild\work\node_modules\isogeo-id\node_modules\express-hbs\node_modules\glob\README.md
error : fstream_type File
error : fstream_class FileWriter
error : code ENOENT
error : errno 34
error : fstream_stack C:\Program Files\nodejs\node_modules\npm\node_modules\fstream\lib\writer.js:284:26
error : fstream_stack Object.oncomplete (fs.js:107:15)
npm http GET https://registry.npmjs.org/esprima/-/esprima-1.0.3.tgz
npm http 304 https://registry.npmjs.org/argparse

The same command would work on other OSs (this is Windows Server 2008 x64). A build on a Windows 7 x64 machine would fail at the grunt stage. Note that the folders described in the output are unlikely to be the cause of the trouble, but at this stage other deeply nested dependencies have already been fetched (based on bower on our case) that trigger all the kinds of strange behaviour described above.

[JoseOneTime]

Is there a resolution for this problem yet? I'm running into it on Windows 7.

[ghost23]

I am running into this, too. I can install almost no packages, always get errors like shown above. If i browse through the folder structure, i see huge nested 'node_modules' structures.

[clca]

to repro the problem I did the following:
create a temp directory
ran npm install bower
cd node_modules/bower
npm install
(at that point I could see the huge node_modules structure)

I ran rmdir /s /q node_modules (from the temp directory) to delete the whole directory structure. It worked just fine on my Win8 machine.

[isaacs]

It looks like this is related to a bug that's been fixed in later versions of rimraf and npm. Please update, and it should be fixed.

While the Windows cmd program may have issues with long folder names, Node (and thus npm) always uses the UNC paths internally for fs operations, so it should work fine in npm.

If you want to remove a folder, you can do npm install -g rimraf and then use the rimraf bin to do this. rimraf path\to\folder

[calbertts]

rimraf command works for me, thanks.

Shit Windows!.

[mcartoixa]

But what if I want to use CLOC or MSBuild item wildcards while the node_modules folder is still around?

[domenic]

I suppose you can only hope those programs are as well designed as node and npm are, and can cope with paths longer than 256 characters!

[neenhouse]

I just wrote a node module "npm-flatten" that can do this. Check it out here: https://www.npmjs.org/package/npm-flatten. Source is here: https://github.com/neenhouse/npm-flatten

[ghost]

An easy solution would be have an option to symlink dependencies directly into the top level node_modules folder. Say you had something like bower. The top level node_modules directory would have a folder structure looking something like:

node_modules
├───abbrev
│   └───1.0.5
├───amdefine
│   └───0.1.0
├───ansi-styles
│   └───1.0.0
├───ansicolors
│   └───0.2.1
├───archy
│   └───0.0.2
├───argparse
│   └───0.1.15
├───array-filter
│   └───0.0.1
├───array-map
│   └───0.0.0
├───array-reduce
│   └───0.0.0
├───asn1
│   └───0.1.11
├───assert-plus
│   └───0.1.2
├───async
│   ├───0.2.10
│   └───0.9.0
├───aws-sign
│   └───0.3.0
├───aws-sign2
│   └───0.5.0
├───binary
│   └───0.3.0
├───block-stream
│   └───0.0.7
├───boom
│   └───0.4.2
├───bower
│   └───1.3.4
├───bower-config
│   └───0.5.1
├───bower-endpoint-parser
│   └───0.2.1
├───bower-json
│   └───0.4.0
├───bower-logger
│   └───0.2.2
├───bower-registry-client
│   └───0.2.1
├───buffers
│   └───0.1.1
├───cardinal
│   └───0.4.4
├───chainsaw
│   └───0.1.0
├───chalk
│   └───0.4.0
├───chmodr
│   └───0.1.0
├───cli-color
│   └───0.2.3
├───configstore
│   ├───0.2.3
│   └───0.3.1
├───cookie-jar
│   └───0.3.0
├───core-util-is
│   └───1.0.1
├───cryptiles
│   └───0.2.2
├───ctype
│   └───0.5.2
├───decompress-zip
│   └───0.0.8
├───deep-extend
│   └───0.2.10
├───es5-ext
│   └───0.9.2
├───esprima
│   └───1.0.4
├───event-emitter
│   └───0.2.2
├───forever-agent
│   └───0.5.2
├───form-data
│   └───0.1.3
├───fstream
│   └───0.1.27
├───fstream-ignore
│   └───0.0.8
├───glob
│   └───3.2.11
├───graceful-fs
│   ├───2.0.3
│   └───3.0.2
├───handlebars
│   └───1.3.0
├───has-color
│   └───0.1.7
├───hawk
│   └───1.0.0
├───hoek
│   └───0.9.1
├───http-signature
│   └───0.10.0
├───inherits
│   └───2.0.1
├───inquirer
│   └───0.4.1
├───insight
│   └───0.3.1
├───intersect
│   └───0.0.3
├───is-root
│   └───0.1.0
├───isarray
│   └───0.0.1
├───js-yaml
│   └───3.0.2
├───json-stringify-safe
│   └───5.0.0
├───jsonify
│   └───0.0.0
├───junk
│   └───0.3.0
├───lockfile
│   └───0.4.2
├───lodash
│   └───2.4.1
├───lodash.debounce
│   └───2.4.1
├───lodash.isfunction
│   └───2.4.1
├───lodash.isnative
│   └───2.4.1
├───lodash.isobject
│   └───2.4.1
├───lodash.now
│   └───2.4.1
├───lodash.objecttypes
│   └───2.4.1
├───lru-cache
│   ├───2.3.1
│   └───2.5.0
├───memoizee
│   └───0.2.6
├───minimatch
│   └───0.3.0
├───minimist
│   ├───0.0.10
│   └───0.0.8
├───mkdirp
│   ├───0.3.5
│   └───0.5.0
├───mkpath
│   └───0.1.0
├───mout
│   └───0.9.1
├───mute-stream
│   └───0.0.4
├───next-tick
│   └───0.1.0
├───nopt
│   ├───1.0.10
│   └───2.2.1
├───oauth-sign
│   └───0.3.0
├───object-assign
│   ├───0.1.2
│   └───0.3.1
├───opn
│   └───0.1.2
├───optimist
│   ├───0.3.7
│   └───0.6.1
├───osenv
│   ├───0.0.3
│   └───0.1.0
├───p-throttler
│   └───0.0.1
├───promptly
│   └───0.2.0
├───punycode
│   └───1.2.4
├───q
│   ├───0.9.7
│   └───1.0.1
├───qs
│   └───0.6.6
├───read
│   └───1.0.5
├───readable-stream
│   └───1.1.13-1
├───readline2
│   └───0.1.0
├───redeyed
│   └───0.4.4
├───request
│   ├───2.27.0
│   ├───2.34.0
│   └───2.36.0
├───request-progress
│   └───0.3.1
├───request-replay
│   └───0.2.0
├───retry
│   └───0.6.0
├───semver
│   ├───2.2.1
│   └───2.3.1
├───shell-quote
│   └───1.4.1
├───sigmund
│   └───1.0.0
├───sntp
│   └───0.2.4
├───source-map
│   └───0.1.34
├───stringdecoder
│   └───0.10.25-1
├───stringify-object
│   └───0.2.1
├───strip-ansi
│   └───0.1.1
├───tar
│   └───0.1.19
├───throttleit
│   └───0.0.2
├───through
│   └───2.3.4
├───tmp
│   └───0.0.23
├───touch
│   └───0.0.2
├───tough-cookie
│   └───0.12.1
├───traverse
│   └───0.3.9
├───tunnel-agent
│   ├───0.3.0
│   └───0.4.0
├───uglify-js
│   └───2.3.6
├───underscore
│   └───1.4.4
├───underscore.string
│   └───2.3.3
├───update-notifier
│   └───0.1.10
├───uuid
│   └───1.4.1
├───which
│   └───1.0.5
└───wordwrap
    └───0.0.2

Then each of the dependencies under node_modules for bower would be symlinked to these versions. Same would happen for dependencies of dependencies. At most you would only ever have a folder depth of 3 under node_modules (node_modules/bower/1.3.4/node_modules/<symlink>)

├─┬ bower
│ ├── abbrev (Symlink to ../../../abbrev/1.0.5)
│ ├── archy (Symlink to ../../../archy/0.0.2)
│ ├── bower-config (Symlink to ../../../bower-config/0.5.1)
│ ├── bower-endpoint-parser (Symlink to ../../../bower-endpoint-parser/0.2.1)
│ ├── bower-json (Symlink to ../../../bower-json/0.4.0)
│ ├── bower-logger (Symlink to ../../../bower-logger/0.2.2)
│ ├── bower-registry-client (Symlink to ../../../bower-registry-client/0.2.1)
│ ├── cardinal (Symlink to ../../../cardinal/0.4.4)
│ ├── chalk (Symlink to ../../../chalk/0.4.0)
│ ├── chmodr (Symlink to ../../../chmodr/0.1.0)
│ ├── decompress-zip (Symlink to ../../../decompress-zip/0.0.8)
│ ├── fstream (Symlink to ../../../fstream/0.1.27)
│ ├── fstream-ignore (Symlink to ../../../fstream-ignore/0.0.8)
│ ├── glob (Symlink to ../../../glob/3.2.11)
│ ├── graceful-fs (Symlink to ../../../graceful-fs/2.0.3)
│ ├── handlebars (Symlink to ../../../handlebars/1.3.0)
│ ├── inquirer (Symlink to ../../../inquirer/0.4.1)
│ ├── insight (Symlink to ../../../insight/0.3.1)
│ ├── is-root (Symlink to ../../../is-root/0.1.0)
│ ├── junk (Symlink to ../../../junk/0.3.0)
│ ├── lockfile (Symlink to ../../../lockfile/0.4.2)
│ ├── lru-cache (Symlink to ../../../lru-cache/2.5.0)
│ ├── mkdirp (Symlink to ../../../mkdirp/0.3.5)
│ ├── mout (Symlink to ../../../mout/0.9.1)
│ ├── nopt (Symlink to ../../../nopt/2.2.1)
│ ├── opn (Symlink to ../../../opn/0.1.2)
│ ├── osenv (Symlink to ../../../osenv/0.0.3)
│ ├── p-throttler (Symlink to ../../../p-throttler/0.0.1)
│ ├── promptly (Symlink to ../../../promptly/0.2.0)
│ ├── q (Symlink to ../../../q/1.0.1)
│ ├── request (Symlink to ../../../request/2.34.0)
│ ├── request-progress (Symlink to ../../../request-progress/0.3.1)
│ ├── retry (Symlink to ../../../retry/0.6.0)
│ ├── semver (Symlink to ../../../semver/2.2.1)
│ ├── shell-quote (Symlink to ../../../shell-quote/1.4.1)
│ ├── stringify-object (Symlink to ../../../stringify-object/0.2.1)
│ ├── tar (Symlink to ../../../tar/0.1.19)
│ ├── tmp (Symlink to ../../../tmp/0.0.23)
│ ├── update-notifier (Symlink to ../../../update-notifier/0.1.10)
│ └── which (Symlink to ../../../which/1.0.5)

This would also help with deduplicating because you could easily just repoint the symlink to a module.

This, from the programming point of view, wouldn't be difficult to implement. I'd happily submit a pull request, however it would be a question of whether or not it was considered the correct direction for the project manager as a whole.

[isaacs]

So, like i said in my comment on joyent/node, the issue isn't that we haven't invented any flat reification strategies. There are plenty of those, and a lot of them work, even using symlinks. (One single global folder where everything lives, with symlinks all over the damn place, and swapping out the require.paths value, is not an ideal strategy, but there are many others.)

We could, for example, have node_modules/.store/<pkgname>/<version> much like @xeoncore is proposing. A problem you quickly run into is people doing require('.store/abbrev/1.2.3'), which, if you think people won't do, you have a short memory ;)

We can avoid that by having the name be non-predictable, but then that kinda sucks for a lot of other reasons.

One of the really nice things about nesting node_modules is that it will literally break half the time if you rely on it being in ./node_modules, so people don't do that for long

So, let's say you have:

node_modules
+-- .store
|   +-- a
|   |   +-- 1.2.3
|   |   +-- 2.5.4
|   |
|   +-- b
|       +-- 1.2.3
|       +-- 5.4.3
|
+-- a -> .store/a/1.2.3
    +-- node_modules
        +-- b -> ../.store/b/5.4.3

I guess you'd have to put the a/node_modules/b in .store/a/node_modules/b, and it's a symlink to a dir, so that's how it'd work anyway.

But now you end up with a rats' nest pretty fast, because actually it's this:

node_modules
+-- .store
|   +-- a
|   |   +-- 1.2.3
|   |   |   +-- node_modules
|   |   |       +-- b -> ../../../.store/b/5.4.3
|   |   +-- 2.5.4
|   |
|   +-- b
|       +-- 1.2.3
|       +-- 5.4.3
|
+-- a -> .store/a/1.2.3

Let's say b->c, and c->a@2.5.4, and a@2.5.4 depends on the other version of b.

Updating a dep, then, is not a simple matter of replacing a single dir. you still have to build the tree, analyze the tree, reify the tree. We still have to figure out who needs what, and dedupe it as much as possible, and then analyze the changes, or else we're just trading deep trees for a giant .store dir.

What's more, we MUST maintain the symlinks in each .store/*/node_modules dir, because otherwise they'll fall up to the top level node_modules folder, and potentially get an incompatible version!

My point here is not to shoot down @xeoncore's proposal, but merely to point out that coming up with the reification strategy is only a small part of the solution (albeit the creative fun part.) The other parts are much less fun:

1. Figuring out the ways that people will abuse it, and how to make sure that they feel the pain of doing so as early as possible, when it is easy to correct.
2. Working out all the ways that the strategy will fall over when there are deeply nested dep trees with lots of conflict management.
3. Implementing the code that predictably guarantees dep version satisfaction in the craziest desert-island scenarios that we can imagine.

[ghost]

@isaacs You make a good point with the user abuse situation. I wasn't suggesting a solution, more so trying to encourage discussion and development because in my opinion this is a legitimate and prevalent issue.

To solve the issue with people directly using modules from the .store directory, is prohibit requiring of modules from it. That wouldn't be hard to do would it?

[pushpan999]

@isaacs Thanks a lot. I tried every possible option that I found in google, none of them worked. Finally rimraf worked :)

[StoneCypher]

Fixing this in NPM isn't enough, because normal tools don't do things like this, so when you try to use node with other things, like the Windows Git package, these paths break them by proxy.

Please stop pursuing the existing bad install strategy.

[ghost]

@StoneCypher You shouldn't be committing your node_modules folder anyway.

[justrhysism]

@xeoncore You're right, we "shouldn't". But unfortunately, in the real-world, we need to. I've been bitten one too many times relying on package versions to handle this for me, and when a client calls for an update to an old project with old dependencies - and for whichever reason these dependencies aren't available, the project cannot be built and many, many hours are wasted trying to get a project to build just so a simply update can be done.

[ryanm2000]

@xeoncore Irrelevant.

[graphnode]

Maybe I'm just showing my ignorance but couldn't we have a simple structure like:

node_modules
+-- a@1.2.3
|   +-- a.js
|   +-- package.json
+-- a@2.5.4
|   +-- a.js
|   +-- package.json
+-- b@1.2.3
|   +-- b.js
|   +-- package.json
+-- b@5.4.3
|   +-- b.js
|   +-- package.json
main.js
package.json

Where, when a module requires another module, node.js tries to:

• Get the module from the local node_module folder; if not found:
• Get the module from the main node_module where the application started (like above); if not found:
• Get the module from the global node_module folder.

Note: All of this with the specific version requested (from package.json) in mind.

Any attempt to directly require a versioned module should throw a error (ex: require('a@2.5.4') or require('node_modules/a@2.5.4'): the versions used are in the package.json.

No symbolic links are used and you can still use deep nesting if you wish (for module development?).

What do you guys think?

[StoneCypher]

Should work.

[brad-sf]

Sounds good @graphnode

[StoneCypher]

@hopeseekr well, all other operating systems should conform to their expectations, you see, and if they've found an api that allows abuse, that means that all software globally should update to the thing they expect, and damn the consequences. this is especially easy since all software worldwide is actively being maintained, and nobody anywhere needs unsupported legacy software for anything.

also what's a red book cd

clearly that's much more straightforward than just having a sensible module deploy system in a single product that claims first class windows support, and didn't bother to learn how the filesystem worked

[StoneCypher]

anything that doesn't work the way they expect is emphasis-bold broken, and should immediately conform to them

[othiym23]

@StoneCypher I know you're passionate about this, but I'd just like to point out that I've been doing all of my non-release process work on npm so far this year running Windows 8.1 as my sole OS (i.e. not in a vm), and I've already addressed several issues with Windows, as well as working with the io.js team to improve the Windows experience. In addition, npm@3 will be beta soon, and it builds a flatter dependency tree by default. I don't think the npm team can make any more plain that we do care about Windows and are committed to improving the experience of npm on it.

[StoneCypher]

The commentary mostly regards community positions and tone.

As before, I'm engaging in someone else's pre-existing commentary theretowards.

I like node. A lot. I continue to use it. I am slowly becoming productive in it.

However, every time Windows people point out where Node is doing severely broken things unnecessarily, like abusing the internationalization api's widechar support to stuff bytes where the API explicitly states they're not permitted to be, we are met with claims that because we do things differently, we suck and are broken, and that 40 years of other software's stability should be violated so that a marginally more sophisticated layout algorithm need not be deployed.

We're fine with "this is pretty hard to do because it turns out that it involves rewriting a bunch of the core." ljharb told me about that the other day. I didn't get a clear context on the problem, and to be frank it seems super weird, but if that's the case, then fine.

But that's not what we got. We got mocking and taunting and sophistry that didn't make sense.

This community has an admirable focus on choice of wording to prevent subtle hostility from excluding people. The gender pronoun debate, though I think handled poorly, was on the right topic.

You do realize that more than ninety percent of developers are Windows goons, right?

Have you ever wondered why you have so few of them?

I am glad that you choose to stamp out the exclusory behaviors that you can see, so as to make a more inclusive environment where you can get the greatest participation.

I just wish that you saw more of them. People hang lanterns on them all the time.

Hopeseeker's a fun dude. I met him in the PHP community. Don't ignore him just because you see my name.

[am11]

Guys what I have learned from IRC (freenode -> nodejs@freenode channel) is that this issue will be fixed in npm v3 as @othiym23 mentioned. Also @iarna indicated above that she was working on it. TBVH, this is no straightforward workaround for this 256 char limitation of NTFS (not very new after all). Not sure what is the situation with ReFS. Nonetheless, I also sent a feedback for this on Windows 10 preview, _hopefully others will do the same_ in order to make them realize that this limitation needs to be revoked and/or reworked. IMO, as a whole npm team has done much better job compared to other package managers (Nuget Packages, Ruby Gems etc.).

Well the reluctance to Windows in any non-MSFT technology community is very common. Although, some people are quite transparent in delivering their concern (for instance the guy who wrote dietlibc (which I was using to make node-sass binary cross-Linux-distro compatible) has to say something in his no windows passage), but on that stance they forget that others (MacWhack) are doing similar stuff (making revenue -- limiting freedom). Since WindOS being non-pure-Unix-like, it will keep getting this special treatment which is not something really new / surprising. :)

On the flipside, see the work from neovim and clang folks, how hard they had to struggle to make their software compatible with Windows, when VC team was not making it easier for masses up until very recently (C99, C11, C++11, C++14 STL and lang features and since so forth).

[iarna]

@StoneCypher When you complain here about "you all" not taking windows seriously, and "here" is an issue on npm, it's hard to not take that as "npm is not taking windows issues seriously." We're aware that we've got some work to build back trust for Windows users, but I'm hopeful that 2015 will be a year that will change that perception. As far as fixing issues with nodejs, you might try raising the issues again over on the iojs issue board (if you haven't already), as it's possible they'll be more receptive.

@am11 That's right: npm@3's dependency resolution algorithm makes the node_modules folder as flat as possible (that is, if there are no conflicts, it puts all requirements at the top level. This results in a implicitly deduped and very shallow tree. Outside of pathological cases, I anticipate that this will solve the "256 char limit" problem by never producing folders nested deeply enough to hit it.

[hopeseekr]

@iarna That's great news! And it's a complete reversal from the first comments about this issue, that went something like, "Haha! You run Windows? You Plebe! Suck it!"

[StoneCypher]

@iarna The io.js people have been great; I'm a semi-regular in their channel.

[Albert-IV]

@StoneCypher As a non-npm or node/io developer just lurking the issue trackers, you've always seemed very combative on the subject of Windows integration, which is why I assume most people aren't taking you seriously on it.

Not trying to poke the bear here; I understand you're frustrated. But I do think the developres are moving towards a solution here.

[StoneCypher]

@droppedoncaprica I understand how telling people how it feels to be treated this way must seem combative, yes.

[johnnyreilly]

@iarna - great to hear that npm@3 is likely to improve life in Windows. Is there any thought as to when that might ship?

[nurbek-ab]

@StoneCypher thanks a lot! subst solution provided by you works like a charm. All other solutions suggested here didn't work for me but by simply mounting node_modules folder to Z drive I was able to remove the whole folder structure.

[DullReferenceException]

Is this still being tracked? I see it's been closed but not resolved. My perception when a non-resolved issue is closed is that it's being ignored, which I really hope is not the case.

[kevva]

It's being fixed in npm@3.

[deostroll]

You can uninstall node modules in that folder serially to solve the problem. http://stackoverflow.com/a/29915329/145682

[chrispappas]

Found this suggestion for using robocopy with an empty directory and node_modules, using the purge switch:
http://stackoverflow.com/a/3320317

Paraphrased TL;DR:
robocopy empty_dir node_modules /purge

I wasn't able to install rimraf because of a borked up npm directory... hoping I can get everything back working again. Hope this helps someone!

[crertel]

So, what is the solution in npm@3 going to be? And how far away is that, exactly?

I'm rather blocked on this at work, and honestly I'm really getting a bit sick of dealing with the Node ecosystem's quirks. I don't want to clutter up this thread with continued kvetching, but I'm really a bit puzzled that a problem well-solved by, say, Bundler and Hex is somehow still causing grief.

(And unfortunately, yes, some of us are stuck with Windows deployments--and no, we'd rather not be, but them's the breaks.)

[iarna]

npm@3 makes the install "maximally flat" by hoisting everything it can to the top level node_modules. This means nesting only occurs on conflicts and as such, trees should never get very deep. As such, the windows path length limitation shouldn't be run into.

Timeline-wise, we're looking to do the first public release this week.

[hopeseekr]

Thank God[dess]!! After the release, NodeJS can announce it once again supports Windows operating systems! I've been battling this issue for... get this!! 2 years and 4 months! For such a critical issue for so many of my clients, I am very happy something is finally being done. Because, frankly, I've had to create a pure PHP equivalent of npm, Gulp and Grunt that doesn't have the damn nested problem. And it works a lot better, too.

[iarna]

I'm locking comments here as they're no longer constructive. Anything new issues around the things discussed here should be opened as new issues.