Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Javascript heap out of memory when doing build-storybook #6408

Open
yuanstanley opened this issue Apr 3, 2019 · 65 comments
Open

Javascript heap out of memory when doing build-storybook #6408

yuanstanley opened this issue Apr 3, 2019 · 65 comments

Comments

@yuanstanley
Copy link

yuanstanley commented Apr 3, 2019

Describe the bug
When trying to do build-storybook it gets to 92% and then fails with the following out of memory error:

92% chunk asset optimization TerserPlugin
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF7CF93EEE5 
 2: 00007FF7CF918CD6 
 3: 00007FF7CF9196E0 
 4: 00007FF7CFD80D3E 
 5: 00007FF7CFD80C6F 
 6: 00007FF7CFCCC594 
 7: 00007FF7CFCC2B67 
 8: 00007FF7CFCC10DC 
 9: 00007FF7CFCCA0B7 
10: 00007FF7CFCCA136 
11: 00007FF7CFDEF7B7 
12: 00007FF7CFEC87FA 
13: 000000D1798DC6C1 
error Command failed with exit code 134.

The entirety of my source code is less than 1 MB so not sure why I would be OOM.

To Reproduce
Steps to reproduce the behavior:

Run yarn build-storybook

Expected behavior
The build shouldn't fail.

Screenshots
If applicable, add screenshots to help explain your problem.

Code snippets

My current webpack.config.js

const TerserPlugin = require('terser-webpack-plugin');

module.exports = {
    optimization: {
        // This line added as an earlier solution to this OOM error
        minimizer: [new TerserPlugin({ parallel: false })],
      },
    resolve: {
        extensions: ['.ts', '.tsx']
    },
    module: {
        rules: [{
            test: /\.js$/,
            use: 'source-map-loader',
            exclude: /node_modules/,
            enforce: 'pre'
          },
          {
            test: /\.tsx?$/,
            use: 'ts-loader',
            exclude: /node_modules/
          },
          {
            test: /\.scss$/,
            use: [{
                loader: "style-loader"
            }, {
                loader: "css-loader", options: {
                    sourceMap: true
                }
            }, {
                loader: "sass-loader", options: {
                    sourceMap: true
                }
            }]
          },
          {
            test: /\.stories\.tsx?$/,
            loaders: [{
                loader: require.resolve('@storybook/addon-storysource/loader'),
                options: {
                    parser: 'typescript'
                }
            }],
            enforce: 'pre',
        }
        ]
      }
}

My current .storybook config.js

import { addDecorator, configure } from '@storybook/react';
import { withOptions } from '@storybook/addon-options';
import { withKnobs } from '@storybook/addon-knobs';

// add withKnobs
addDecorator(withKnobs);

// Option defaults:
// Full settings here: https://github.com/storybooks/storybook/tree/master/addons/options
addDecorator(
  withOptions({
    name: 'some-package Storybook',
    showAddonPanel: false,
    addonPanelInRight: true,
    showStoriesPanel: false,
    sortStoriesByKind: true
  })
);

function loadStories() {
  const req = require.context('../src/', true, /\.stories\.tsx?$/)
  req.keys().forEach((filename) => req(filename))
}
configure(loadStories, module);

I also tried changing the build-storybook command in my package.json to force increase the memory, but that doesn't seem to work either.
"build-storybook": "npx --max_old_space_size=16384 build-storybook"

My full package.json

{
  "name": "some-package",
  "version": "0.0.1",
  "description": "PackageDescription",
  "registry": "https://myurl.com/",
  "main": "dist/charts.js",
  "types": "dist/charts.d.ts",
  "license": "ISC",
  "engines": {
    "node": ">=8.*",
    "yarn": ">=1.*"
  },
  "scripts": {
    "auth": "npx vsts-npm-auth -config .npmrc",
    "dev": "webpack --mode=development",
    "dev:watch": "webpack --mode=development --watch",
    "build": "webpack --mode=production",
    "lint": "tslint ./src/**/*.{ts,tsx}",
    "fix_coverage": "node ./fix_coverage.js",
    "test": "yarn lint && jest --coverage",
    "test:watch": "yarn lint && jest --watch",
    "test:watchAll": "yarn lint && jest --watchAll",
    "storybook": "start-storybook -p 6006",
    "build-storybook": "npx --max_old_space_size=16384 build-storybook"
  },
  "dependencies": {
    "json5": "^2.1.0",
    "lodash": "4.17.11",
    "moment": "^2.24.0",
    "react-localization": "^1.0.13",
    "react-select": "^2.3.0",
    "react-split-pane": "0.1.77",
    "sinon": "^7.2.3",
    "ts-mock-imports": "^1.2.2"
  },
  "peerDependencies": {
    "office-ui-fabric-react": "6.*",
    "plotly.js": "1.*",
    "react": "16.*",
    "react-dom": "16.*",
    "react-localization": "^1.0.13",
    "react-plotly.js": "2.*",
    "uuid": "^3.3.2"
  },
  "devDependencies": {
    "@babel/core": "7.1.6",
    "@storybook/addon-knobs": "4.1.16",
    "@storybook/addon-options": "4.1.16",
    "@storybook/addon-storysource": "4.1.16",
    "@storybook/react": "4.1.16",
    "@types/enzyme": "^3.1.15",
    "@types/jest": "^23.3.10",
    "@types/lodash": "4.14.118",
    "@types/plotly.js": "1.41.0",
    "@types/react": "16.7.9",
    "@types/react-dom": "16.0.11",
    "@types/react-plotly.js": "2.2.2",
    "@types/react-select": "^2.0.11",
    "@types/storybook__addon-knobs": "4.0.4",
    "@types/storybook__react": "4.0.1",
    "@types/uuid": "^3.4.4",
    "babel-loader": "8.0.4",
    "clean-webpack-plugin": "1.0.0",
    "css-loader": "^2.1.0",
    "dts-bundle-webpack": "1.0.1",
    "enzyme": "^3.8.0",
    "enzyme-adapter-react-16": "^1.7.1",
    "enzyme-to-json": "^3.3.5",
    "fork-ts-checker-webpack-plugin": "0.5.0",
    "fs-jetpack": "2.2.0",
    "gulp-jest": "^4.0.2",
    "identity-obj-proxy": "^3.0.0",
    "inline-css": "2.4.1",
    "jest": "^23.6.0",
    "jest-canvas-mock": "^1.1.0",
    "jest-cli": "^24.1.0",
    "jest-junit": "5.2.0",
    "net": "^1.0.2",
    "node-sass": "^4.11.0",
    "office-ui-fabric-react": "6.158.0",
    "plotly.js": "^1.44.4",
    "react": "16.6.3",
    "react-dom": "16.6.3",
    "react-plotly.js": "2.2.0",
    "sass-loader": "^7.1.0",
    "source-map-loader": "0.2.4",
    "style-loader": "^0.23.1",
    "terser-webpack-plugin": "^1.2.3",
    "tls": "^0.0.1",
    "ts-jest": "23.10.5",
    "ts-loader": "5.3.1",
    "tslint": "5.11.0",
    "tslint-config-prettier": "1.17.0",
    "tslint-react": "3.6.0",
    "typescript": "3.2.1",
    "webpack": "4.29.6",
    "webpack-cli": "3.3.0"
  }
}

System:

  • OS: Windows 10
  • Device: HP Desktop
  • Browser: Chrome
  • Framework: React
  • Addons: @storybook/addon-knobs, @storybook/addon-options, @storybook/addon-storysource
  • Version: 4.1.16
@yuanstanley
Copy link
Author

This issue ended up being resolved when I upgraded to the latest storybook version, 5.0.6.

@shilman
Copy link
Member

shilman commented Apr 4, 2019

Thanks for following up @yuanstanley -- glad it's working for you!

@Tobi-mmt
Copy link

Tobi-mmt commented May 27, 2020

I want to reopen this issue

Describe the bug
I got the exact same issue like @yuanstanley in version 5.3.13 of storybook. Updateing to the newest version does not help. So it's really the same issue. The build-storybook command stops at 92% while doing stuff in the TerserPlugin.

thre strange part for me is, that the build run previous without problems. But from one day to the next it does not work anymore. My collegues run on the same OS, the exact same version of storybook (same package lockfile) but do not run into the issue.

Here the full error:

yarn run v1.22.4
$ build-storybook
info @storybook/react v5.3.13
info 
info clean outputDir..
info => Copying prebuild dll's..
info => Building manager..
info => Loading manager config..
info => Loading presets
info => Loading custom manager config.
info => Compiling manager..
info => manager built (6.35 s)
info => Building preview..
info => Loading preview config..
info => Loading presets
info => Loading config/preview file in ".storybook".
info => Loading custom Webpack config (full-control mode).
info => Using base config because react-scripts is not installed.
info => Compiling preview..
92% chunk asset optimization TerserPluginFATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100d77bad node::Abort() (.cold.1) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 2: 0x10003ae83 node_module_register [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 3: 0x10003b044 node::FatalTryCatch::~FatalTryCatch() [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 4: 0x1001aa117 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 5: 0x1001aa0b1 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 6: 0x1005994c2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 7: 0x10059bbf3 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 8: 0x100597b6e v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 9: 0x1005959ed v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
10: 0x1005a2c15 v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
11: 0x1005a2c7f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
12: 0x100571e63 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
13: 0x100805ccc v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
14: 0x2959cc1dbe3d 
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100d77bad node::Abort() (.cold.1) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 2: 0x10003ae83 node_module_register [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 3: 0x10003b044 node::FatalTryCatch::~FatalTryCatch() [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 4: 0x1001aa117 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 5: 0x1001aa0b1 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 6: 0x1005994c2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 7: 0x10059bbf3 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 8: 0x100597b6e v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 9: 0x1005959ed v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
10: 0x1005a2c15 v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
11: 0x1005a2c7f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
12: 0x100571e63 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
13: 0x100805ccc v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
14: 0xca95225be3d 
15: 0xca95225d1ff 
^[[1;2DFATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100d77bad node::Abort() (.cold.1) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 2: 0x10003ae83 node_module_register [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 3: 0x10003b044 node::FatalTryCatch::~FatalTryCatch() [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 4: 0x1001aa117 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 5: 0x1001aa0b1 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 6: 0x1005994c2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 7: 0x10059bbf3 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 8: 0x100597b6e v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 9: 0x1005959ed v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
10: 0x1005a2c15 v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
11: 0x1005a2c7f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
12: 0x100571e63 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
13: 0x100805ccc v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
14: 0x1e9b8b8dbe3d 
15: 0x1e9b8b8dd1ff 
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100d77bad node::Abort() (.cold.1) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 2: 0x10003ae83 node_module_register [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 3: 0x10003b044 node::FatalTryCatch::~FatalTryCatch() [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 4: 0x1001aa117 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 5: 0x1001aa0b1 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 6: 0x1005994c2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 7: 0x10059bbf3 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 8: 0x100597b6e v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
 9: 0x1005959ed v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
10: 0x1005a2c15 v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
11: 0x1005a2c7f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
12: 0x100571e63 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
13: 0x100805ccc v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/xyz/.nvm/versions/node/v10.20.1/bin/node]
14: 0x3ea875f5be3d 
15: 0x3ea875f5d1ff 
ERR! => Failed to build the preview
ERR! vendors~main.32009e1438c5ef1ca75d.bundle.js from Terser
ERR! Error: Call retries were exceeded
ERR!     at ChildProcessWorker.initialize (/Users/xyz/Projects/a-dir/node_modules/@storybook/core/node_modules/jest-worker/build/workers/ChildProcessWorker.js:230:21)
ERR!     at ChildProcessWorker._onExit (/Users/xyz/Projects/a-dir/node_modules/@storybook/core/node_modules/jest-worker/build/workers/ChildProcessWorker.js:307:12)
ERR!     at ChildProcess.emit (events.js:198:13)
ERR!     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
(node:14471) UnhandledPromiseRejectionWarning: [object Object]
(node:14471) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:14471) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /Users/xyz/.nvm/versions/node/v10.20.1/bin/node
Arguments: /usr/local/Cellar/yarn/1.22.4/libexec/lib/cli.js run build-storybook
Directory: /Users/xyz/Projects/a-dir/bla_de
Output:

To Reproduce
run yarn build-storybook

Expected behavior
The build shouldn't fail.

Screenshots
nothing

Code snippets
on request

System:
Environment Info:

System:
OS: macOS 10.15.4
CPU: (8) x64 Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
Binaries:
Node: 10.20.1 - ~/.nvm/versions/node/v10.20.1/bin/node
Yarn: 1.22.4 - /usr/local/bin/yarn
npm: 6.11.3 - ~/Projects/a-dir/node_modules/.bin/npm
Browsers:
Chrome: 83.0.4103.61
Firefox: 76.0.1
Safari: 13.1

Framework: React

Additional context
I tried the following to solve the problem:

  1. delete all node_modules folders and reinstall all packages
  2. upgrade storybook to latest version 5.3.19
  3. upgrade storybook to version 6 beta 12
  4. set the RAM usage of node with npx --max_old_space_size=16384 build-storybook

All of that does not solve the error.

@shilman shilman reopened this May 27, 2020
@shilman
Copy link
Member

shilman commented May 27, 2020

@ndelangen what's the current thinking on terser plugin?

@Tobi-mmt
Copy link

Tobi-mmt commented May 28, 2020

The quickfix for me was to use the node version 12.x instead of 10.x . I don't know why, but this solves the problem for me at the moment.

If I switch back to 10, it is still failing 🤷‍♀️

@ndelangen
Copy link
Member

@ndelangen what's the current thinking on terser plugin?

It's the best in class javascript minifier AFAIK.

@Tobi-mmt
I'm glad you found a work-around.

I'm not sure what to do.. Well code splitting will possibly help.

@wkeese
Copy link

wkeese commented Jun 7, 2020

Hmm, it's failing for me even on Node 12. Getting that same error:

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

@ndelangen
Copy link
Member

ndelangen commented Jun 8, 2020

Could you try:

NODE_OPTIONS=--max_old_space_size=4096 build-storybook

or increase the number to more?

@wkeese
Copy link

wkeese commented Jun 8, 2020

I did try 4096 and it still failed. I tried 8192 and it just never finished. I suppose I should try spliiting things into layers, maybe the polyfills for IE.

@IshratPahalwi
Copy link

I too am facing this issue. Any updates ?

@jfoliveira
Copy link

Same error here with Storybook v5.3.18 and node v12.18.0. Running on an almost empty 4 GB RAM AWS EC2 instance (only nginx running) dedicated to build Storybook for a ReactJs web app with ~30 components and 5 pages. :-(

Using NODE_OPTIONS as suggested in other comments, the task seems to never finish (I waited for only one hour):

NODE_OPTIONS=--max_old_space_size=4096 yarn build-storybook

Question:
Could anyone please guide me on how to tell Storybook to not optimize, not minify, not use source maps, etc?

I went to the Storybook docs on customizing webpack but couldn't even find the words optimize, minify, source maps, and all other keywords that come to my mind.

As I really don't want to become a Webpack expert and I don't need any fancy bundling or any other optimization, I just need to get Storybook to work and be served as a static HTML site. If it's not minified, if it's not bundled but it's working that would be cool and perhaps all I'll ever need.

Please let me know what should I change to serve Storybook the simplest possible way and no longer be stuck at the 92% chunk asset optimization TerserPlugin building stage.

Thanks in advance for any advice!

@IshratPahalwi
Copy link

For me this worked : NODE_OPTIONS=--max_old_space_size=4096 build-storybook .
However, my travis build seems to give the error : The job exceeded the maximum log length, and has been terminated during the storybook build. I am assuming the storybook build is generating too much logs which is causing the travis build to terminate . Can anyone please suggest how to supress storybook from generating too much unnecessary logs ?

@jfoliveira
Copy link

jfoliveira commented Jun 17, 2020

Storybook as a component gallery is an amazing project and then we're here: developers struggling to take the most of this beautiful tool due to bundling size optimization issues - bundling that is not even needed in many scenarios.

Why is bundling such an important thing for a project like Storybook that mainly targets internal audience?
A very successful web app with 1 million daily users will share their components in Storybook with all their users?
I don't think so.

It's very likely only devs, product owners, managers, designers, UX/UI teams will need to interact with a Storybook instance. How big can an internal team like that be? 500 (five hundred) people in a really big enterprise?
And how many of them will be interacting with Storybook from a slow internet connection? I can't say, but it seems to me that will actually be an edge case.

In a scenario with up to 500 Storybook users is all this out-of-the-box optimization really worth it?

Sorry, I'm completely new to this project and don't mean to be critical, just putting this as food for thought.
I hope it's taken as some opportunity for improvement rather than as a negative feedback.

@ndelangen
Copy link
Member

@jfoliveira would you be interested in voicing your opinion on this matter in our next roadmap meeting?:
https://github.com/storybookjs/community

@jfoliveira
Copy link

jfoliveira commented Jun 23, 2020

Hey @ndelangen ! Thanks for the invite! Not sure if l'll be able to make it to the roadmap meeting but I'll try to attend.

In the meantime, for those still facing this same issue when building Storybook, I share below the minimal config changes that I came up with on my .storybook/main.js to disable bundling:

module.exports = {
  webpackFinal: async (config, { configType }) => {
    config.optimization = {
      minimize: false,
      minimizer: [],
    };

    return config;
  },
  stories: ['../**/*.stories.js'],
  addons: [
...

After applying these changes the error message 92% chunk asset optimization TerserPlugin no longer appeared. I've built/deployed on the same remote servers a few dozens of times in the past week, successfully.

Some numbers that might be helpful:

  1. My web server:
    I downgraded to an AWS EC2 t2.micro instance as my build server + web server:
    1 GB RAM + 1 GB swap disk space.

  2. Build time - with bundling off:
    Build is taking 2 to 3 minutes to complete, which for my team is totally acceptable.

  3. Final assets size:
    main*.bundle.js ~ 11 MB
    vendor*.bundle.js ~ 2 MB
    Total page size: 14 MB (according to Firefox 77)

  4. Page load time:
    About 5 minutes to load Storybook on the browser.
    I know, it's not great - but it's working fine for my team. Also, as I'm serving Storybook as a Node.js app from a very small/cheap cloud server instance, I think it's ok.
    Load it, go grab a coffee and then it'll be available offline and cached in the browser. :-)

Hope this helps other Storybook users on deciding whether disabling bundling might be an option for them or not.

@ghost
Copy link

ghost commented Jun 25, 2020

Maybe to add a little more information. I have the same memory leak issue when running start-storybook.

Storybook Version: 5.3.18
Angular Version: 8.1.2
Node Version: 12.16.1

95% emitting fork-ts-checker-webpack-plugin

@xseman
Copy link

xseman commented Jul 17, 2020

start-storybook I can confirm memory leak, storybook consumed about 6GB of RAM.

storybook/react:: v5.3.18
Node: v14.5.0
OS: Linux (Debian 10)

@neil-morrison44
Copy link

We've got two (large ish) projects that build storybooks as part of their CI, both started failing with JavaScript heap out of memory errors after @babel/core auto-updated from 7.10.5 to 7.11.0, a 3rd project with a much smaller JS footprint (no react) is fine on v7.11.0 though.

storybook/react:: v5.3.19
Node: v12
OS: Linux (Travis CI Container)

@shilman shilman added the tracked label Aug 4, 2020
labkey-nicka added a commit to LabKey/labkey-ui-components that referenced this issue Aug 7, 2020
@ndelangen
Copy link
Member

I don't think this is strictly a problem with storybook.
Likely the problem lies upstream: TerserPlugin possibly, or simply the problem arises because of too large files.

Here's the known work-around:
#6408 (comment)

@IceDevelop74
Copy link

I don't think this is strictly a problem with storybook. Likely the problem lies upstream: TerserPlugin possibly, or simply the problem arises because of too large files.

Here's the known work-around: #6408 (comment)

I've tried this workaround but nothing changes :/

@pcambra
Copy link

pcambra commented Apr 25, 2023

I've tried the workaround above and disabling the source maps and I have this issue in 7.0.7

@dizzyjaguar
Copy link

Currently also having this issue in 7.0.7

@mayerraphael
Copy link

Same here. 7.0.7

@chad-levesque
Copy link

Same error, 7.0.10. Running out of memory on Gitlab runner with source maps disabled and NODE_OPTIONS: '--max-old-space-size=4096'
Not sure what to try next 🤔

@catnuko
Copy link

catnuko commented May 30, 2023

Same 7.0.8

@ndelangen
Copy link
Member

Does someone have a repro I could use for an experiment?

I wonder if replacing Terser with esbuild would fix this problem.

@maxphillipsdev
Copy link

same result in 7.0.22. Tried running with 4GB but no luck

@rajpawar02
Copy link

same result in 7.0.22. Tried running with 4GB on aws amplify but no luck

@ndelangen
Copy link
Member

@maxphillipsdev @rajpawar02

Would you mind giving this a try:?

```js
// .storybook/main.js
export default {
// Replace your-framework with the webpack-based framework you are using (e.g., react-webpack5)
framework: {
name: '@storybook/your-framework',
options: {
builder: {
useSWC: true,
},
},
},
stories: ['../src/**/*.mdx', '../src/**/*.stories.@(js|jsx|mjs|ts|tsx)'],
};
```

@vui611
Copy link

vui611 commented Jul 3, 2023

@ndelangen give it a go but still got the out-of-memory problem.
VM has 4Gb memory and the storybook is v7.0.24

@sachin-walunj
Copy link

@maxphillipsdev @rajpawar02

Would you mind giving this a try:?

```js
// .storybook/main.js
export default {
// Replace your-framework with the webpack-based framework you are using (e.g., react-webpack5)
framework: {
name: '@storybook/your-framework',
options: {
builder: {
useSWC: true,
},
},
},
stories: ['../src/**/*.mdx', '../src/**/*.stories.@(js|jsx|mjs|ts|tsx)'],
};
```

Tried this solution but still facing the same issue

@sachin-walunj
Copy link

Screenshot 2023-07-07 at 12 58 26 PM

@dimafirsov
Copy link

dimafirsov commented Jul 7, 2023

@maxphillipsdev @rajpawar02

Would you mind giving this a try:?

```js
// .storybook/main.js
export default {
// Replace your-framework with the webpack-based framework you are using (e.g., react-webpack5)
framework: {
name: '@storybook/your-framework',
options: {
builder: {
useSWC: true,
},
},
},
stories: ['../src/**/*.mdx', '../src/**/*.stories.@(js|jsx|mjs|ts|tsx)'],
};
```

Thanks @ndelangen, looks like this fixed the memory leak issue for the vite builder with the mentioned above memory boost NODE_OPTIONS=--max_old_space_size=9126.

storybook 7.0.25

@juanigalan91
Copy link

👋 I can reproduce with our repo using storybook 7.1.0.

In my end, removing the feature storyStoreV7 fixed the issue and allowed the build to finish. I've narrowed the issue to TerserPlugin not being able to handle so many files, specifically files that weight a lot (around 10MB).

    features: {
        storyStoreV7: false,
    },

I've tried to tweak TerserPlugin by removing either parallel building, or just removing comments without minification but I had no success.

My best guess is that storyStoreV7, which enables code splitting (per storybook's documentation), and then splits stories into multiple bundles, therefore generating a main.bundle with all the common code used across all stories. This file has so much code (at least in our scenario) that TerserPlugin is not able to handle the size and runs out of memory.

@ndelangen
Copy link
Member

@juanigalan91 maybe this is something you might want to try:

```js
// .storybook/main.js
export default {
// Replace your-framework with the webpack-based framework you are using (e.g., react-webpack5)
framework: {
name: '@storybook/your-framework',
options: {
builder: {
useSWC: true,
},
},
},
stories: ['../src/**/*.mdx', '../src/**/*.stories.@(js|jsx|mjs|ts|tsx)'],
};
```

It's an experimental option to switch from babel to swc, and will also switch to TerserWebpackPlugin.swcMinify.

@juanigalan91
Copy link

hi @ndelangen thanks for the tip!

Unfortunately I am still running out of memory. I've also tried using TerserWebpackPlugin .esbuildMinify and TerserWebpackPlugin .uglifyJsMinify by modifying the webpack configuration manually, but I had unfortunately the exact same result.

The only workaround that I've managed to find while keeping storyStoreV7: true is to add NODE_OPTIONS=--max-old-space-size=4096 to the build script. But I must admit, it feels like cheating 😅

@ndelangen
Copy link
Member

FYI storyStoreV7: false will be removed/unsupported in the future (8.0 likely)

@juanigalan91
Copy link

Hope this gets fixed or at least we have a workaround for it by then. I feel like the code splitting might be a bit too agressive with storyStoreV7: false and it is not providing us the best results when it comes to building storybook and deploying it. At least that is what I understand from what is going on.

Maybe we would need to split our monorepo's storybook into sub storybooks and unify all of them with storybook composition, but it feels like a regression.

@lordrip
Copy link

lordrip commented Oct 2, 2023

In my case, I had a few big files.

storybook-static/assets/Catalog.stories-9eab6292.js                  9,958.00 kB │ gzip: 1,228.47 kB │ map:   160.12 kB

Locally, it worked for me, so the failure only happened on GitHub actions nodes. To replicate the issue, I modified the command as follows:

    "build:storybook": "NODE_OPTIONS=--max_old_space_size=2048 storybook build",

The fix for me was to reduce a JSON file used for providing demo data for one of the stories and then the files got smaller.

storybook-static/assets/Catalog.stories-ac6ca80f.js                  2,538.06 kB │ gzip: 313.20 kB │ map:   159.61 kB

In any case, @ndelangen , here's our repository if you want a reproduction example: https://github.com/KaotoIO/kaoto-next/

@Silinator
Copy link

Same issue now on 7.5.3 on github actions and only some times also in CLI
With "build-storybook": "NODE_OPTIONS=--max_old_space_size=4096 storybook build"
It's worked for me.

But kind of strange why i have to this < 7.5.3 worked for me with out any problems and i only have 4 Stories at the moment.

@williamhaley
Copy link

williamhaley commented Jan 25, 2024

Same issue here on a 4GB self-hosted runner on Github Actions. Killed with status 137 (OOM) seemingly at random on some jobs. Using the @storybook/nextjs framework as this is a Next.js project. Using version 7.6.10 for storybook and related @storybook/... addons.

<s> [webpack.Progress] 92% sealing asset processing TerserPlugin
Killed
error Command failed with exit code 137.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 137
Command: /opt/hostedtoolcache/node/18.19.0/x64/bin/node
Arguments: /opt/hostedtoolcache/node/18.19.0/x64/lib/node_modules/yarn/lib/cli.js build
Directory: /runner/_work/mugen/mugen/packages/storybook
Output:

Other jobs in the same workflow like tests, docker builds, deployments, compilation, etc., are all fine on that same runner so I'd rather not bump up to a larger one just for the sake of getting Storybook to build.

Changing the framework as suggested here seemed to yield reliable builds for the time being at least 🤞

edit: unfortunately, swc means I no longer use babel, which means I can't use idInterpolationPattern for format js, and so I had to turn off useSwc again 🥀 #25772

HowardBraham added a commit to MetaMask/metamask-extension that referenced this issue Feb 15, 2024
)

Building storybook in CI seems to fail often -- probably due to running out of memory. Changing the compiler to swc might help (according to storybookjs/storybook#6408 (comment)).

I don't know what the ramifications of this change may be (I don't even know what storybook is for 😅). So someone with knowledge here should review this PR.

---------

Co-authored-by: Howard Braham <howrad@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests