Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yarn serve - JavaScript heap out of memory crash #1453

Closed
xrei opened this issue Jun 3, 2018 · 38 comments
Closed

yarn serve - JavaScript heap out of memory crash #1453

xrei opened this issue Jun 3, 2018 · 38 comments

Comments

@xrei
Copy link

@xrei xrei commented Jun 3, 2018

Version

3.0.0-beta.15

Reproduction link

https://github.com/xrei/vuecli-bug

Steps to reproduce

Well.. it's not hard to reproduce but takes time.
Run yarn serve
and develop for some hours :)

What is expected?

Stable working dev server

What is actually happening?

After some hours (~1-2) dev server will crash with an error:

 95% emitting CopyPlugin
<--- Last few GCs --->

[2032:000001F314F3C8F0]  5471846 ms: Mark-sweep 1381.6 (1414.7) -> 1381.6 (1414.7) MB, 250.8 / 0.0 m
s  allocation failure GC in old space requested
[2032:000001F314F3C8F0]  5472120 ms: Mark-sweep 1381.6 (1414.7) -> 1381.6 (1413.7) MB, 273.5 / 0.0 m
s  last resort GC in old space requested
[2032:000001F314F3C8F0]  5472433 ms: Mark-sweep 1381.6 (1413.7) -> 1381.6 (1413.7) MB, 313.7 / 0.0 m
s  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 000001CC26384281]
Security context: 000002F26A1206A9 <JSObject>
    1: fromString(aka fromString) [buffer.js:349] [bytecode=0000006AEF71E081 offset=148](this=000001
3071A022E1 <undefined>,string=0000031511A4D141 <Very long string[3793765]>,encoding=000002F26A131A29
 <String[4]: utf8>)
    2: from [buffer.js:201] [bytecode=0000006AEF71DC01 offset=11](this=0000023CE59638A9 <JSFunction
Buffer (sfi = 0000037B38A8B959)>,val...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::DecodeWrite
 2: node_module_register
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::FatalProcessOutOfMemory
 5: v8::internal::Heap::MaxHeapGrowingFactor
 6: v8::internal::Factory::NewRawTwoByteString
 7: v8::internal::Smi::SmiPrint
 8: unibrow::Utf8DecoderBase::WriteUtf16Slow
 9: v8::String::WriteUtf8
10: std::basic_ostream<char,std::char_traits<char> >::basic_ostream<char,std::char_traits<char> >
11: node::Buffer::New
12: node::Buffer::New
13: v8::internal::interpreter::BytecodeDecoder::Decode
14: v8::internal::RegExpImpl::Exec
15: v8::internal::RegExpImpl::Exec
16: v8::internal::RegExpImpl::Exec
17: 000001CC26384281
error Command failed with exit code 134.
@ryouaki

This comment has been minimized.

Copy link

@ryouaki ryouaki commented Jun 4, 2018

change the scripts at package.json as below
node --max_old_space_size=4096 node_modules/@vue/cli-service/bin/vue-cli-service.js serve --open

@ryouaki

This comment has been minimized.

Copy link

@ryouaki ryouaki commented Jun 4, 2018

this is not an issue from vue-cli ,this is the limit from nodejs.

@yyx990803

This comment has been minimized.

Copy link
Member

@yyx990803 yyx990803 commented Jun 4, 2018

There's nothing vue-cli can do about this.

@yyx990803 yyx990803 closed this Jun 4, 2018
@atinybeardedman

This comment has been minimized.

Copy link

@atinybeardedman atinybeardedman commented Jun 6, 2018

Just in case anyone else finds this issue I had to change the package.json serve script to the following:

"serve" : "node --max_old_space_size=4096 node_modules/.bin/vue-cli-service serve --open"

@ryouaki your path didn't work for me

@xrei

This comment has been minimized.

Copy link
Author

@xrei xrei commented Jun 6, 2018

"dev": "npx --max_old_space_size=4096 vue-cli-service serve"

@atinybeardedman this worked for me

but just wanna say, that my own webpack config doesn't have this issue with memory
also i'm sure i have enough memory. just strange behaviour.
maybe it's because of webpack 4. I don't know.

@ryouaki

This comment has been minimized.

Copy link

@ryouaki ryouaki commented Jun 7, 2018

@xrei this is because of your code. for me , in my project , i have more than 500 pages and more 100 base components , and with a large assets and css with cssmodule.
some bad practice with cssmodule from junior developer , this will take large memory. ( ´▽`)

@ryouaki

This comment has been minimized.

Copy link

@ryouaki ryouaki commented Jun 7, 2018

@atinybeardedman you should change the path to you own path which you saved the vue-cli scripts.

@octref

This comment has been minimized.

Copy link
Member

@octref octref commented Jun 13, 2018

@yyx990803 You can fix this by adding --max_old_space_size=4096 to the hashbang of cli-service.

I only had a small project that I developed for 3 hours, and this happened for at least 5 times. I feel a lot of people might run into this issue.

Root cause is here: webpack/webpack#1914 (comment). From what I read in the thread, it seems to be that the memory usage from sourcemap generation & file watching hits V8's limit (1400MB). Bumping the limit to 4GB should make the ceiling much harder to hit.

@yyx990803

This comment has been minimized.

Copy link
Member

@yyx990803 yyx990803 commented Jun 13, 2018

@octref we can do that for now. Although in the long run we probably want to see if there's anything we can do to reduce the heap usage. I think it's because a single Vue file actually involves multiple source maps being passed and merged.

@chokn

This comment has been minimized.

Copy link

@chokn chokn commented Jun 21, 2018

Adding a directory of large library files to my project's .eslintignore file fixed the issue for me.

@vanderlin

This comment has been minimized.

Copy link

@vanderlin vanderlin commented Jun 23, 2018

I get this crash after about 10 saves and webpack tries to compile.

@DRoet

This comment has been minimized.

Copy link
Contributor

@DRoet DRoet commented Jun 29, 2018

I'm also having this problem a few times / day after a few dozen of saves even when increasing the max_old_space_size=4096. I never had this problem using the old vue cli 2.9.x + webpack template and the default max_old_space_size. perhaps an upstream bug somewhere that is increasing the heap ?

@vanderlin

This comment has been minimized.

Copy link

@vanderlin vanderlin commented Jun 29, 2018

I'm still getting this crash as well - have tried the --max_old_space_size=4096 but still crashes after about 10 mins. The old vue-cli webpack worked great no problems at all. Any thoughts, solutions, anything???

@DRoet

This comment has been minimized.

Copy link
Contributor

@DRoet DRoet commented Jul 4, 2018

Might be related to: webpack/webpack#6929

octref added a commit to octref/vue-cli that referenced this issue Jul 5, 2018
@mercer08

This comment has been minimized.

Copy link

@mercer08 mercer08 commented Jul 8, 2018

i'm currently used version is 3.0.0-beta.15(often crashed because of memory leaks), what can I do to upgrade to the new version(v3.0.0-rc.3)?

@DRoet

This comment has been minimized.

Copy link
Contributor

@DRoet DRoet commented Jul 11, 2018

@vanderlin try updating your dev-dependencies:

rm -rf [package-lock.json] node_modules && npm cache clean -f && npm i

make sure u get webpack: ^4.15.1 and postcss-loader: ^2.1.6 since they fixed some memory leaks in these versions.

I have been seeing a lot less memory usage (no crashes yet) but I'll have to test it out more to tell if it is completely fixed.

@octref

This comment has been minimized.

Copy link
Member

@octref octref commented Jul 12, 2018

@DRoet I have been testing webpack 4.15.1 + and postcss-loader 2.1.6+ too, and it seems the memory leak issue is much better. Do you want to send a PR? I'm in China now and for some reason cannot download certain dependencies in vue-cli (chromedriver) and yarn would fail to update the lock file.

You can just update webpack / postcss-loader at /packges/@vue/cli-service.

@DRoet

This comment has been minimized.

Copy link
Contributor

@DRoet DRoet commented Jul 12, 2018

@octref I checked and the current dev branch already has webpack: ^4.15.1 and postcss-loader: ^2.1.6

@octref

This comment has been minimized.

Copy link
Member

@octref octref commented Jul 12, 2018

Oh...Seems @Akryum unintentionally fixed 3 memory leaks 93940c2#diff-90339eb44e295051bd16d3e099b219d8 😛

@Akryum

This comment has been minimized.

Copy link
Member

@Akryum Akryum commented Jul 12, 2018

That was intentional, I've been monitoring the issue on webpack repo 😸

@havefive

This comment has been minimized.

Copy link

@havefive havefive commented Oct 18, 2018

edit package.json

"serve":"node --max_old_space_size=4096 node_modules/@vue/cli-service/bin/vue-cli-service.js serve --open"

@ffxsam

This comment has been minimized.

Copy link
Contributor

@ffxsam ffxsam commented Oct 18, 2018

For what it's worth, I haven't had this issue in a very long time, and I didn't change anything about my setup.

@xrei

This comment has been minimized.

Copy link
Author

@xrei xrei commented Oct 18, 2018

That's right. in the latest vue-cli this isn't happening anymore.

moranje added a commit to moranje/meditor-one that referenced this issue Oct 31, 2018
as suggested here vuejs/vue-cli#1453
@NSKevin

This comment has been minimized.

Copy link

@NSKevin NSKevin commented Nov 1, 2018

  1. Try to increase the heap size
  2. If you don't ignore the /dist in .gitignore, then you need to add /dist into the .eslintignore
@soletan

This comment has been minimized.

Copy link

@soletan soletan commented Nov 27, 2018

I'm having similar issue with running some project using vue-cli-service as soon as I inject a local dev-version of a package to be used in that project using npm link. Raising heap size didn't work out. As soon as I publish the package on npm and replace the link with installation from npm repo everything is working as expected. Any clues on this?

UPDATE: Ok, kept searching just a bit. And it looks like quite several tools around and including webpack were/are having issues with handling symlinked deps (according to this and all the issues linked there). That's a bummer for trying to work with npm ecosystem e.g. by developing packages to be part of a VueJS project if there is no other way but fixing webpack configuration through vue CLI configuration the pretty hard way. What would be the best approach to control this from vue.config.js?

@avadhut123pisal

This comment has been minimized.

Copy link

@avadhut123pisal avadhut123pisal commented Jan 2, 2019

I was facing problems while importing .js files. I had a html formatter installed and that added spaces in js files and so was taking much space during compilation. I compressed js files again and it worked :)

@saraha33

This comment has been minimized.

Copy link

@saraha33 saraha33 commented Jan 6, 2019

for me this also happens in dev mode (vue-cli-service serve) when its building a local package i linked with npm link
increasing heap size did not work for me
(vue-cli 3.2.3 webpack 4.28.3)

ps: i am already using webpack 4 in another project but without vue-cli 3 and there i have never encountered this issue (i guess something could be different in the webpack config settings but checking around a bit i really could not figure out what it could be...)

@saraha33

This comment has been minimized.

Copy link

@saraha33 saraha33 commented Jan 9, 2019

ok, just in case this is helpful to anyone...i got rid of this issue with

  chainWebpack: (config) => {
    config.resolve.set('symlinks', process.env.NODE_ENV !== 'development');
  },

however turns out that vue-cli -service build command still has lots of problems with my npm linked package and unfortunately i cant figure out the correct webpack settings...but i guess thats a different story :/

@hz-ljq

This comment has been minimized.

Copy link

@hz-ljq hz-ljq commented Feb 17, 2019

change the scripts at package.json as below
node --max_old_space_size=4096 node_modules/@vue/cli-service/bin/vue-cli-service.js serve --open

@ryouaki
I encounter this problem always, even use your workaround. But this problem's gone when I disable eslint by [lintOnSave: false] in vue.config.js~~

ndelangen added a commit to storybookjs/storybook that referenced this issue Mar 11, 2019
ndelangen added a commit to storybookjs/storybook that referenced this issue Mar 11, 2019
ndelangen added a commit to storybookjs/storybook that referenced this issue Mar 11, 2019
@runxc1

This comment has been minimized.

Copy link

@runxc1 runxc1 commented Apr 9, 2019

So I see that this issue is closed but none of the fixes noted here in changing the package.json seem to fix the issue. I was using the Vue UI and am no longer able to and am starting to no longer be able to make more than 2 or 3 changes before it runs out of memory.

I am trying increase the memory for the following script

"build-watch": "vue-cli-service build --mode development --dest ../../wwwroot/vue --target app --watch"
and get an error when I attempt to change it to

"build-watch": "node --max_old_space_size=4096 node_modules/.bin/vue-cli-service build --mode development --dest ../../wwwroot/vue --target app --watch"

@leonardotessaroalves

This comment has been minimized.

Copy link

@leonardotessaroalves leonardotessaroalves commented Jun 13, 2019

solution:

npm i -D cross-env

"scripts"; { "dev": "cross-env NODE_OPTIONS=--max_old_space_size=8192 vue-cli-service serve --mode development" }

@jinzunyue

This comment has been minimized.

Copy link

@jinzunyue jinzunyue commented Jul 3, 2019

  1. Try to increase the heap size
  2. If you don't ignore the /dist in .gitignore, then you need to add /dist into the .eslintignore

thinks,it really helps me.

@anyderre

This comment has been minimized.

Copy link

@anyderre anyderre commented Jul 10, 2019

If your application crash while building scss files try installing those dependencies node-sass, style-loader and sass-loader, this worked for me!

@chelentos

This comment has been minimized.

Copy link

@chelentos chelentos commented Aug 3, 2019

solution:

npm i -D cross-env

"scripts"; { "dev": "cross-env NODE_OPTIONS=--max_old_space_size=8192 vue-cli-service serve --mode development" }

for me worked even just cross-env NODE_ENV=development vue-cli-service serve
thank you

@mirzaumersaleem

This comment has been minimized.

Copy link

@mirzaumersaleem mirzaumersaleem commented Sep 16, 2019

add .eslintignore to your project directory will do a workaround sample .eslintignore file

node_modules
.env
.env.development
.env.production
.vscode
coverage
upload
ngrok.exe
.babelrc
build
test
README.md

@billyc

This comment has been minimized.

Copy link

@billyc billyc commented Oct 1, 2019

There is definitely a (large) memory leak causing yarn serve / npm serve to crash consistently and reproducibly. Yes, I can delay that crash somewhat by increasing memory for Node until my dev machine has swapped everything else out of RAM. Not sure why this issue is closed?

This memory leak is most egregious when I use web workers, I think since they bundle several npm libraries, and apparently the entire thing is then leaked. I can crash the yarn serve process on my second build without fail, with my project's particular combination of web workers, typescript, and babel. But I can see that that combo is not required for the memory leak to occur.

Every time a file-save event triggers a build, the node process inches up in memory. (In my case it "inches up" by about 400k RAM on every save). I've bumped Node up to a max of 4GB of RAM which sometimes gets me through a couple hours of dev before it crashes.

I'd like to help find the actual cause. What can I do to help?

  • I've experienced this problem with every version of vue-cli since I started using the 3.x release
  • With every version of TypeScript from 2.6 up to current 3.6
  • Using each of Node 8.x, 10.x, and 12.x. No difference; the memory leak is always there.
  • Adding folders to .eslintignore does not solve the problem for me
  • This is on WSL Ubuntu 18.04 LTS if that makes any difference.
  • My project uses the following vue-cli plugins: babel, eslint, typescript, unit-jest.

Hoping these details help us find a real solution!

@verymuch

This comment has been minimized.

Copy link

@verymuch verymuch commented Nov 8, 2019

@xrei this is because of your code. for me , in my project , i have more than 500 pages and more 100 base components , and with a large assets and css with cssmodule.
some bad practice with cssmodule from junior developer , this will take large memory. ( ´▽`)

which kinds of bad practices will lead to this problem? Can you give me some suggestions?

@soletan

This comment has been minimized.

Copy link

@soletan soletan commented Nov 8, 2019

which kinds of bad practices will lead to this problem? Can you give me some suggestions?

It's kind of off-topic, but in server-side Javascript coding it's mostly unreleased event listeners that cause a running software to grow huge over time. When using streams quite frequently it's rather easy to have some recently used event listeners left unreleased. I'd suggest to use unit tests for they can help to find the culprits that prevent a test runner from exiting due to unreleased resources.

Apart from that there are other more obvious causes for wasting memory e.g. when caching stuff in memory without cleaning out from time to time or even the simplest algorithms implemented in a wrong way so e.g. arrays are growing huge...

In context of VueJS it might be related to accidentally saving references, like working with render() functions or using vm.$el to access some DOM element directly for attaching event listeners which has been bad without VueJS before and so it's still bad today. Caching issues are possible there as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.