Storybook manager dev-server leaks memory during hot refresh.

[smurrayatwork]

Describe the bug
When using storybook in development mode, I've noticed that the amount of memory the node process uses increases with every change made to my react components. After doing some memory profiling, it appears to be that webpack-dev-middleware is writing to memory all of the data of the files made by storybook's temporary cache (located under node_modules/.cache after every hot reload, without clearing out previously made entries.

Storybook is using a hash for its filenames in this cache instead of a static file name, causing webpack-dev-middleware to write the newly generated files' data to memory after each hot refresh. This should be changed to a static filename. See the following:
https://github.com/storybookjs/storybook/blob/next/lib/core/src/server/manager/manager-webpack.config.ts#L56
webpack/webpack#10796 (comment)

While the manager does have writeToDisk: true set, this does not prevent webpack-dev-middleware from writing to memory: https://github.com/webpack/webpack-dev-middleware#writetodisk

To Reproduce

1. Start storybook up in dev mode while using node --inspect.
2. Make a change to any react components, then check node's memory usage.

Expected behavior
The main process for running storybook in dev mode doesn't leak memory.

[smurrayatwork]

I think upgrading core's webpack-dev-middleware to the latest 4.x version may fix this issue, as their changelog mentions respecting the use of file hashes.

[stale]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[smurrayatwork]

Just commenting to make not stale

[jdelStrother]

I keep seeing Storybook crash with OOM errors after working in it for a while. I tried attaching the Chrome profiler to the server process - here's successive heap snapshots after making a single change to a file on disk:

Seems like an awful lot of stuff in Webpack's in-memory cache.

[shilman]

Yowza!! I just released https://github.com/storybookjs/storybook/releases/tag/v6.2.0-rc.5 containing PR #14264 that references this issue. Upgrade today to the @next NPM tag to try it out!

npx sb upgrade --prerelease

Closing this issue. Please re-open if you think there's still more to do.

[jdelStrother]

That PR was reverted in #14283, so could we reopen for now?

In the meantime, I'm working around this by overriding Storybook's webpack config to set output.filename = "[name].bundle.js";, which seems to be working well. Memory usage stays constant and I'm not experiencing the crashes mentioned in #14283

[shilman]

Fixed in #14284 released in 6.2

npx sb ugprade