Process out of memory - Webpack

[andrewhathaway]

After a couple of builds running in a Watch, Webpack crashes due to being out of memory.

<--- Last few GCs --->

 9223585 ms: Scavenge 1390.0 (1454.7) -> 1390.0 (1454.7) MB, 9.0 / 0 ms (+ 0.7 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
 9224727 ms: Mark-sweep 1390.0 (1454.7) -> 1388.3 (1452.7) MB, 1142.7 / 0 ms (+ 9.8 ms in 89 steps since start of marking, biggest step 2.5 ms) [last resort gc].
 9225694 ms: Mark-sweep 1388.3 (1452.7) -> 1388.8 (1454.7) MB, 966.6 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x35903c44a49 <JS Object>
    1: walkFunctionDeclaration [<PROJECT_DIR>/node_modules/webpack/lib/Parser.js:~443] [pc=0xa07a14ec8ee] (this=0x59f67991119 <a Parser with map 0x71f2d115d49>,statement=0x3507a80af661 <a Node with map 0x71f2d1157c9>)
    2: walkStatement [<PROJECT_DIR>/node_modules/webpack/lib/Parser.js:~348] [pc=0xa07a06dfc10] (this=0x59f6799111...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6

[SpaceK33z]

I'm having this too with webpack-dev-server on Debian; but I haven't had time to debug this further.
Do you have a minimal webpack config where you can reproduce this? It could also be an external loader or plugin.

[sokra]

Don't use hashes when watching. i. e. [chunkhash] causes all assets to be stored.

[SimenB]

@sokra In the documentation for css-loader it says to use hash for easier debugging wrt css modules. Does that have the same problem?

You can configure the generated ident with the localIdentName query parameter (default [hash:base64]). Example: css-loader?localIdentName=[path][name]---[local]---[hash:base64:5] for easier debugging.

We've also started to see this problem in dev (out of memory), but our only hash in dev mode is the css one.

EDIT: Of course it uses hash by default, so I'm guessing it's not a problem.

[sokra]

Does that have the same problem?

No. Only filenames.

You can compare it with webpack --watch. If this generates many files in to your output path, the webpack-dev-server generates many files in the memory-fs.

If this is not the issue, you can increase the node.js memory (it defaults to 1.7 GB, which can be too few for big builds).

[SimenB]

I'll test at work on Monday! If the outputted file is of the same name as the old one, then it's overwritten, right?

[sokra]

jup.

You can navigate to http://localhost:8080/webpack-dev-server to see a list of all files in memory.

[SimenB]

Ah, didn't know that! Using a small project I have on my own computer, I notice that all of the hot-update files gets added.

Could they be a problem? Any way of clearing them out after they've been there for x amount of time? If we develop for a few hours there's probably quite a few of them in the end

[sokra]

Could they be a problem?

They could, but they are usually pretty small.

[SpaceK33z]

I think clearing them out like @SimenB said would be a nice option, or maybe there is some way to delete all in-memory files that are no longer in use?

[sokra]

jup. Write a plugin...

[sokra]

But deleting them too early can break stuff...

[deepakmani]

We have the exact same memory leak problem after using webpack -w on MacOS X. Can you please suggest what is a the best way to fix / workaround this issue. The ngOfficeUIFabric seems to deal with a different problem.

Thanks,
Deepak

[luchillo17]

Have same issue of @deepakmani in OSX El Capitan, however i'm using Ionic 2 which in it's config doesn't use css-loader, instead it just uses ionic-gulp-sass-build, so ikd if the error happens because of webpack or that gulp plugin, also i'm not using HRM:

const path = require('path')
const PrefetchPlugin = require('webpack/lib/PrefetchPlugin')
const ProvidePlugin = require('webpack/lib/ProvidePlugin')
const DedupePlugin = require('webpack/lib/optimize/DedupePlugin')
const UglifyJsPlugin = require('webpack/lib/optimize/UglifyJsPlugin')
const WebpackNotifierPlugin = require('webpack-notifier')


module.exports = {
  entry: {
    polyfills: './app/polyfills.ts',
    vendor: './app/vendor.ts',
    app: path.resolve('app/app')
  },
  output: {
    path: path.resolve('www/build/js'),
    filename: '[name].bundle.js',
    publicPath: "/",
    chunkFilename: "[id].bundle.js",
    pathinfo: false, // show module paths in the bundle, handy for debugging
  },
  devtool: 'eval',
  module: {
    loaders: [
      {
        test: /\.ts$/,
        loader: 'awesome-typescript',
        query: {
          doTypeCheck: true,
          resolveGlobs: false,
          externals: ['typings/main.d.ts']
        },
        include: path.resolve('app'),
        exclude: /node_modules/
      }
      ,{
        test: /\.js$/,
        include: path.resolve('node_modules/angular2'),
        loader: 'strip-sourcemap'
      }
    ],
    noParse: [
      /es6-shim/,
      /reflect-metadata/,
      /zone\.js(\/|\\)dist(\/|\\)zone/
    ]
  },
  plugins: [
    new PrefetchPlugin('rxjs'),
    new ProvidePlugin({
      Chart: 'chart.js/Chart.min.js'
    }),
    new DedupePlugin()
    // ,new UglifyJsPlugin({
    //    // to debug prod builds uncomment //debug lines and comment //prod lines
    //   // beautify: true,//debug
    //   // mangle: false,//debug
    //   // dead_code: false,//debug
    //   // unused: false,//debug
    //   // deadCode: false,//debug
    //   // compress : { screw_ie8 : true, keep_fnames: true, drop_debugger: false, dead_code: false, unused: false, }, // debug
    //   // comments: true,//debug

    //   beautify: false,//prod
    //   // disable mangling because of a bug in angular2 beta.1, beta.2 and beta.3
    //   // TODO(mastertinner): enable mangling as soon as angular2 beta.4 is out
    //   // mangle: { screw_ie8 : true },//prod
    //   mangle: false,
    //   // dead_code : true,
    //   compress : { 
    //     dead_code : true,
    //     screw_ie8 : true
    //   },//prod
    //   comments: false//prod
    // })
    ,new WebpackNotifierPlugin({
      title: 'tao_app_ionic',
      alwaysNotify: true
      // contentImage: path.join(appDir, 'images/notifier.png')
    })
  ],
  resolve: {
    alias: {
      'angular2': path.resolve('node_modules/angular2')
    },
    extensions: ['', '.js', '.ts']
  }
};

[guruprasanth5]

I am having the same issue, any hack that can fix this for now.

[thunderkid]

Having the same problem running webpack via webpack-stream in gulp. I use gulp.watch to rerun on code changes, which then just runs

    return gulp.src(tsStartFile)
              .pipe(webpackStream(webPackConfig))
              .on('error', logError) 
              .pipe(gulp.dest(outputDir))

I'd have thought this would run webpack from scratch each time, but after 5 or so builds I get the out of memory error and have to rerun my gulp file.

[Pines-Cheng]

Did someone resolved this problem？

[tdsmithATabc]

Another data point here; I have filename hashes disabled and my build isn't terribly large (it uses Angular 2, but few other dependencies), but after about an hour of dev work the watch dies with Abort trap: 6.

I've temporarily increased the memory limit, which gives me a lot more time, but I'd rather not be consuming gigabytes just to run a watch command.

FYI if anyone is curious about upping the memory limit, I did it like so in my package.json (obviously the additional flags are dependent on your needs):

{
  "scripts": {
    "start": "node --max_old_space_size=4096 node_modules/webpack-dev-server/bin/webpack-dev-server.js --inline --progress --port 3000 --open"
  }
}

[mikiest]

Having the same issue as @thunderkid here. Any update?

[rochdev]

Same issue for a very small project

[idegree]

export NODE_OPTIONS=--max_old_space_size=4096

Works for me.

[Delagen]

@idegree I used this for a while, but this broke some programs in Windows. I took for about hour to detect that Geforce Experience does not start because of this environment variable.
I hope in Linux all ok )

[awacode21]

Having the same problem running webpack via webpack-stream in gulp. I use gulp.watch to rerun on code changes, which then just runs

    return gulp.src(tsStartFile)
              .pipe(webpackStream(webPackConfig))
              .on('error', logError) 
              .pipe(gulp.dest(outputDir))

I'd have thought this would run webpack from scratch each time, but after 5 or so builds I get the out of memory error and have to rerun my gulp file.

Same with grunt

[ingava]

Not sure if this is going to help anyone but we had a problem after updating to webpack 4 with the out of memory error. The build itself would often get stuck or become very slow. What very unexpectedly helped us was removing the BundleAnalyzerPlugin.

[kevinSuttle]

I got this in production mode, but not development 🤷‍♂️

[vishalraj82]

@kevinSuttle +1 I also encounter error in production mode, but not in development

[migueljoserivera]

I had this problem with the version 10.. of node but, when I upgraded to the v12.4.0 the error gone.

[FuDesign2008]

@buildbreakdo A better option is to do this:

"scripts": {
    "build": "cross-env NODE_OPTIONS=--max_old_space_size=8192 webpack"
}

(It uses the cross-env package.)

The nice thing about using NODE_OPTIONS is that it will affect everything, including any sub-processes that Webpack might spawn.

(It's useful for non-Webpack things as well: we're using it for TypeDoc, which consumes a lot of RAM.)

This solution did resolve the problem in my project.

[kamlesh-cis]

@buildbreakdo A better option is to do this:

"scripts": {
    "build": "cross-env NODE_OPTIONS=--max_old_space_size=8192 webpack"
}

(It uses the cross-env package.)
The nice thing about using NODE_OPTIONS is that it will affect everything, including any sub-processes that Webpack might spawn.
(It's useful for non-Webpack things as well: we're using it for TypeDoc, which consumes a lot of RAM.)

This solution did resolve the problem in my project.

It solved my problem as well I just had to install "cross-env" pkg and add the node option to the webpack command!

Thanks a lot!

[junibrosas]

upping the memory limit will only solve the problem temporarily but the RAM being used up the more you rebuild the application. You will notice your machine gets slower overtime.

[odykyi]

remove max_old_space_size

just add to ts-loader

           loader: 'ts-loader',
           options: {
             transpileOnly: true // this works for me
           }

[NextTrick]

Update Node versión from v8.16.0 to v12.6.0 and that issue is gone!

[sontd-0882]

Update Node versión from v8.16.0 to v12.6.0 and that issue is gone!

Me too, thanks! @NextTrick

[alexander-akait]

Memory leak found suguru03/neo-async#83

[satheler]

remove max_old_space_size

just add to ts-loader

           loader: 'ts-loader',
           options: {
             transpileOnly: true // this works for me
           }

Works for me.