Slow run with large (unused) node_modules folder

[ToshB]

I have an issue where t2 run spends a long time to start up, it might be related to indexing of node_modules folder before build.

I started with the t2 init project, and ran t2 run index.js. It used around 14 secs to build, deploy and run the project.

I then added the following dev-dependencies:

"babel-cli": "^6.9.0",    
"babel-preset-es2015": "^6.9.0",
"babelify": "^7.3.0",
"eslint": "^2.11.1",

Without any other changes, the command t2 run index.js now used 40 secs to complete. I checked the files on the device, and the only files transferred was the index.js and package.json, as expected.

[Frijol]

This is odd because I would expect these to be taken out by the default --slim during bundling. @HipsterBrown do you know why this might be happening?

[ToshB]

I might also add that this was slow even if I added node_modules or node_modules/babel* and node_modules/eslint* to .tesselignore.

Adding --slim (which should be default anyway) changes nothing.

I can also note that I get a warning during build about a missing pre-compiled module fsevents, which is used by babel-cli. So it is doing something unnecessary with my node_modules folder.

[HipsterBrown]

That is interesting. Ideally, if none of those dev dependencies are required your program, then t2-cli should not parse them. It might be a bug within t2-project. Pinging @rwaldron for feedback. I will also investigate this afternoon while I'm working on a few issues.

[rwaldron]

None of those will be included, or even looked at (ie. file system traversal, etc), unless they are being required from the "entry point" file, eg. t2 run <entry point file>. That said, I'm going to try to repro this and see what I can learn.

[johnnyman727]

@rwaldron were you able to repro?

[rwaldron]

I just attempted a repro right now and I can immediately tell where the issues are:

1. In the binary module discovery phase, which is globbing for *.node files to process. Obviously we can't limit that process in any way, because we can never know where the compiled module files will be.
   • Possible optimization: apply the project's own ignore rules to the patterns used to find the *.node files
2. Looking for ./**/.tesselignore and ./**/.tesselinclude
   • Possible optimization: apply the project's own ignore rules to those lookups, so that project-ignored paths are never visited.

...

Note that all of our assumptions above were completely correct—the issue was elsewhere and previously unforeseen in the build flow.

[rwaldron]

This is turning out to be a lot more difficult than I made it sound. I have an optimization based on my own plans above, but the complexity cost is growing, so I want to stop and come back to it later after I've had some time to think about it more. In the meantime, can I suggest not using Babel? The transpiled result will be obscene. Any ES6 features supported in Node.js LTS 4.x.x are natively available to you.