Webpack unexpectedly watches unrelated files even the ones not imported anywhere

[ChristianIvicevic]

Bug report

What is the current behavior?
Webpack in watch mode is recursively scanning unrelated files in the current working directory and the src folder even though none of them should appear in the dependency graph and they don't happen to be imported in any of the files defined by the entry config.

The result is that even though those files don't end up in the bundle, webpack still attempts to process them and (re)builds take an enormous amount of time since it aparently works on the dependency graph on the "94% after seal" step.

I noticed this issue due to immense build times and because I had some TypeScript errors in certain files in the src folder which all of a sudden appeared in the webpack output even though those files are unrelated to the entry file!

If the current behavior is a bug, please provide the steps to reproduce.
Create a simple foo.ts file in your root folder and have a src folder with an arbitrary amount of source files. In our case this was our entire codebase. The foo.ts file should only contain console.log("Foo");. Run webpack in watch mode with the following config file:

const path = require('path');

module.exports = {
    entry: './foo.ts',
    devtool: 'inline-source-map',
    module: {
        rules: [
            {
                test: /\.tsx?$/,
                use: 'ts-loader',
                exclude: [
                    /node_modules/
                ]
            }
        ]
    },
    resolve: {
        extensions: [ '.tsx', '.ts', '.js' ]
    },
    output: {
        filename: 'bundle.js',
        path: path.resolve(__dirname, 'dist')
    }
};

What is the expected behavior?
The bundle will only contain the foo.ts output and be built in merely milliseconds instead of 10+ seconds with our huge codebase in the mentioned src folder.

Other relevant information:
webpack version: 4.29.6
Node.js version: 8.10.0
Operating System: macOS 10.14.4
Additional tools: n/a

[sokra]

cc @johnnyreilly

[johnnyreilly]

Do you experience the same if you're using the fork-ts-checker-webpack-plugin? Example setup here: https://github.com/TypeStrong/ts-loader/tree/master/examples/fork-ts-checker-webpack-plugin

[ChristianIvicevic]

@johnnyreilly I just tried the sample you linked and compilation and bundling is of course nearly instant with the single foo.ts file, but nonetheless the entire src tree is being linted in the background. So basically webpack is still attempting to go over virtually every single file.

I might attempt to integrate ForkTsCheckerWebpackPlugin in our codebase nonetheless to figure out whether bundling is as fast as hoped.

[arpowers]

I'm having a similar issue, I have a node_modules folder nested in a project which is unrelated to the webpack build. But it's getting scanned somehow and takes the build process from 15s to 100s (or webpack just crashes)

@sokra I can verify that this is happening without Typescript. Is a Webpack issue. Also using the ignored 'watchOptions' config doesn't help.

[sokra]

Thanks for your report. It would be great if you reduce your issue to a small reproducible example. Best put this example into a github repository together with instructions how to get to the problem.

[arpowers]

Ended up figuring out that it was related to a globbing feature on one of the webpack loaders.

Was ridiculously hard to debug.

[ljcrapo]

@arpowers what plugin was it?

[arpowers]

Style resources loader.

[alexander-akait]

Can you create minimum reproducible test repo?

[marcusjwhelan]

Ended up figuring out that it was related to a globbing feature on one of the webpack loaders.

Was ridiculously hard to debug.

How did you fix the loader? @arpowers

[arpowers]

In my case, i just switched the approach. However Im sure it would have been improved if i'd used specific paths instead of a glob.

…

On Wed, May 29, 2019 at 9:47 AM Marcus ***@***.***> wrote: Ended up figuring out that it was related to a globbing feature on one of the webpack loaders. Was ridiculously hard to debug. How did you fix the loader? @arpowers <https://github.com/arpowers> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9020?email_source=notifications&email_token=AACLHJQ3SQLYVADQ6O44JITPX2CPVA5CNFSM4HELUQF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWPMCEA#issuecomment-496943376>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACLHJTFLHRYLPHYGFJBRQDPX2CPVANCNFSM4HELUQFQ> .

[alexander-akait]

Somebody can create minimum reproducible test repo? Looks like problem in ts-loader

[marcusjwhelan]

@evilebottnawi Here is a project I I have that uses electron-webpack that has definite issues with TS checking https://github.com/marcusjwhelan/testingproj

run npm run dev-v1 to run development usually will crash after a bit of running
run npm run compile1 will normally crash just trying to compile the code
let the application just sit there in IntelliJ and it will slowly creep up memory and crash.

[marcusjwhelan]

Any Update on this. I still have a project that is about 10 times the size of that testing project and it constantly crashes my computer. No increasing node memory available does not help because slowly memory is filled and crashes. Is even worse when running another project with vagrant as the back end.
While in intelliJ. I found if I turn of Javascript Support plugin the issues disappear.

electron-userland/electron-webpack#280

[deser]

Also, seems watchOptions.ignored doesn't work

[kalbert312]

Running into same problem as @marcusjwhelan. Spent quite a bit of time upgrading packages, optimizing imports, adjusting memory limits, compiler options etc to no avail. TS 3.5.2, CRA, Electron, pretty much all latest packages. Haven't been able to pin point a cause.

[webpack-bot]

This issue had no activity for at least three months.

It's subject to automatic issue closing if there is no activity in the next 15 days.

[alexander-akait]

Somebody can create minimum reproducible test repo?

[larsnystrom]

@evilebottnawi I have a theory of what is happening here. ts-loader will compile all files that are matched by the include configuration in tsconfig.json, even if they are not actually imported in the code you are compiling.

To reproduce, run:

git clone git@github.com:larsnystrom/webpack-9020-repro.git
cd webpack-9020-repro
npm ci
npm run build

The problem is the include config in tsconfig.json. Since the glob there matches both app.ts and app.test.ts both will be compiled by ts-loader, even though the webpack.config.js only uses app.ts as an entrypoint.

Maybe this is documented behavior but it was not very obvious to me and personally I think ts-loader should only deal with the files that are either specified as entry points in webpack.config.js, or are imported somewhere in the tree of dependencies starting at the entrypoint.

In the repro case this can be fixed by also adding exclude: ['*.test.ts'] to tsconfig.json.

[alowdon]

@larsnystrom You can use the onlyCompileBundledFiles option to only compile those TypeScript files which are referenced from the code you are compiling, e.g.

use: [{
    loader: 'ts-loader',
    options: {
        onlyCompileBundledFiles: true
    }
}]

Not sure if this will also fix the issue that @ChristianIvicevic has with watching.

[webpack-bot]

Issue was closed because of inactivity.

If you think this is still a valid issue, please file a new issue with additional information.