Stop scanning node_modules directories in PackageSource#_findSources. #9825
Conversation
This functionality was originally intended to allow importing compiled-to-JS modules from `node_modules`, by precompiling any modules found in top-level npm packages, as long as a Meteor compiler plugin was registered for the module's file extension. As discussed in #9800, this extra compilation burden can be non-trivial, especially if you happen to install an npm package such as `less`, which contains hundreds of `.less` files in the `node_modules/less/test/` directory. More generally, this functionality was an early attempt to enable selective compilation of `node_modules` directories, but it was not a good solution to that problem, because in almost all cases the extra compilation was unwanted. Meteor 1.7 (formerly known as 1.6.2) will give full control over selective compilation of `node_modules` back to the application developer (#9771), which should afford a much better solution to this problem. If you've installed some `.less` or `.scss` or `.ts` files from npm into your `node_modules` directory, just create a symlink to the package directory within your application, and those modules will be compiled and become importable by other JS modules, as if they were part of the application.
benjamn
added a commit
that referenced
this pull request
May 29, 2018
…Sources. (#9825)" This reverts commit c5302bb. Based on conversation with @KoenLav, it seems this optimization was more of a breaking change than anticipated, and the workaround of creating a symbolic link to induce compilation is not just annoying on Windows but also does not satisfy the goal of being able to import from node_modules (as before) rather than importing via the symbolic link. We will need to revisit this pitfall in Meteor 1.7.1, but it's important to unbreak it ASAP in Meteor 1.7.0.1 (since 1.7 is unfortunately final). #9826 (comment) mozfet/meteor-autoform-materialize#43 For apps that are accidentally compiling too many LESS or SCSS files, the recommended workaround (for now) is to add the offending directories and/or files to a .meteorignore file.
benjamn
added a commit
that referenced
this pull request
May 29, 2018
Merged
benjamn
added a commit
that referenced
this pull request
Jun 7, 2018
During the Meteor 1.7 beta testing process, I noticed that npm packages installed in `node_modules` that have a lot of `.less` files in them (such as the `less` npm package or `antd`) can slow down rebuilds significantly, because every `.less` file gets compiled, even if it is never used. With #9825, I attempted to solve this problem by completely removing the logic for scanning `node_modules` for files to process with compiler plugins, but that turned out to be a breaking change for existing apps that relied on that behavior, so we reverted that optimization in Meteor 1.7.0.1: #9917 Of course, this means the performance problem still occurs in Meteor 1.7.0.1, as demonstrated by #9957. After investigating the problem a bit further, I discovered that the `LessCompiler#isRoot` method was simply ignoring whether a `.less` file comes from `node_modules` or not. More specifically, Meteor sets `fileOptions.lazy = true` by default for any file found in `node_modules` [here](https://github.com/meteor/meteor/blob/0a336175c4a47cd264060891b5f66893b945afe4/tools/isobuild/package-source.js#L1020), so `LessCompiler#isRoot` should return `false` from `isRoot` when `fileOptions.lazy` is truthy. Thanks to this much more precise solution, `.less` files in `node_modules` can still be imported by other `.less` files; they just won't be eagerly compiled by the `less` package. Fixes #9957.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
This functionality was originally intended to allow importing compiled-to-JS modules from
node_modules, by precompiling any modules found in top-level npm packages, as long as a Meteor compiler plugin was registered for the module's file extension.As discussed in #9800, this extra compilation burden can be non-trivial, especially if you happen to install an npm package such as
less, which contains hundreds of.lessfiles in thenode_modules/less/test/directory.More generally, this functionality was an early attempt to enable selective compilation of
node_modulesdirectories, but it was not a good solution to that problem, because in almost all cases the extra compilation was unwanted.Meteor 1.7 (formerly known as 1.6.2) will give full control over selective compilation of
node_modulesback to the application developer (#9771), which should afford a much better solution to this problem. If you've installed some.lessor.scssor.tsfiles from npm into yournode_modulesdirectory, just create a symlink to the package directory within your application, and those modules will be compiled and become importable by other JS modules, as if they were part of the application.