Out of memory error

[ntrrgc]

Issue details

I use BrowserSync along with IntelliJ, which automatically saves files. After a few hours, BrowserSync crashes due to an out of memory error.

[BS] File changed: rjs-build.js
[BS] File changed: events.html
[BS] File changed: demo-templates.html

<--- Last few GCs --->

11097515 ms: Scavenge 1409.3 (1458.1) -> 1409.3 (1458.1) MB, 18.6 / 0 ms (+ 65.6 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
11098521 ms: Mark-sweep 1409.3 (1458.1) -> 1409.1 (1458.1) MB, 1006.1 / 14 ms (+ 141.5 ms in 11 steps since start of marking, biggest step 65.6 ms) [last resort gc].
11099530 ms: Mark-sweep 1409.1 (1458.1) -> 1409.1 (1458.1) MB, 1008.9 / 15 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x100a92ee3ac1 <JS Object>
    1: exec [native regexp.js:~88] [pc=0xb039a79cf12] (this=0x20afb6cf6361 <JS RegExp>,k=0x322526f2d809 <String[81]: /hdd/home/ntrrgc/Documentos/Dropbox/hepdata-explore/frontend/node_modules/type-is>)
    2: LOOP(aka LOOP) [fs.js:1614] [pc=0xb039a18ff5e] (this=0x100a92e04189 <undefined>)
    3: _tickCallback [internal/process/next_tick.js:~87] [pc=0xb039f097611] (this=0x100a92efc501 <a process ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

Steps to reproduce/test case

Serve a project. Make many changes for a few hours, triggering many reloads.

Please specify which version of Browsersync, node and npm you're running

• Browsersync 2.11.1
• Node 5.10.1
• Npm 3.8.7

Affected platforms

• linux
• windows
• OS X
• freebsd
• solaris
• other (please specify which)

Browsersync use-case

• API
• Gulp
• Grunt
• CLI

If CLI, please paste the entire command below

browser-sync start --server --files="**/*.js,**/*.html" --browser google-chrome --reload-delay 500

[lmannering]

Any update on this? I've started getting the same error :(

[patricknelson]

Originally posted about this at #1953 (comment) but CC'ing this issue since I'm seeing these "JavaScript heap out of memory" errors with BrowserSync via CLI as well (using npx, in my case on WSL2 with a VM that has 4GB of memory).

I can reproduce this error in less than a minute in my project, since editing one include may trigger recompiles in ~91 files. 😅

[shakyShane]

@patricknelson Browsersync doesn't process any files - so I'm wondering if your example is being caused by the gulp setup?

If you have the time, could you run browser-sync separately, whilst still compiling sass with gulp?

[patricknelson]

@shakyShane This is actually entirely CLI. See #1953 (comment) for the .browser-sync.cjs file, however: For me this can be easily reproduced in my setup with the following 2 separate commands running in separate terminals/tabs/panes. The only thing you need is a large number of source .scss files. 😅

First, start sass compiler in one tab/window (this is sass installed in the project, e.g. npm i sass)

npx sass -w -s compressed scss:css

Then kick off BrowserSync in another tab/window (also installed locally, e.g. npm i browser-sync)

npx browser-sync start --config .browser-sync.cjs

Then a couple of edits later, a JavaScript heap out of memory error.

[patricknelson]

There is definitely some kind of weird nesting or recursion going on. When launching in debug mode to enable the inspector of browser-sync itself (this time via globally installed version instead of the one instantiated by npx), i.e.

 NODE_OPTIONS='--inspect' browser-sync start --config .browser-sync.cjs

If you kick off a sass compile and the memory starts to build up, if you just go into the "Sources" tab and pause at any moment, it's almost always endlessly looping over this ReaddirpStream._read() method (in file:///home/patnelson/.nvm/versions/node/v20.10.0/lib/node_modules/browser-sync/node_modules/readdirp/index.js, in my case). Then, if you set a breakpoint at this.push(entry); you'll notice that it is constantly pushing the same files into the stream reader, over and over and over again (and memory just bloats).

No clue what's causing it though. Maybe after enough files + saves, it just gets so big that this starts happening? Maybe something isn't being cleaned up or freed? It's somewhere in the file watcher in browser-sync -> chokidar -> readdirp. Beyond that, I'm not sure. I had to put it down after playing with it for an hour or two. 😕

[patricknelson]

This may actually be a bug either in chokidar or readdirp. Someone else is experiencing very similar issues here: paulmillr/chokidar#1268.

Edit: Yep, after some tinkering with this setup purely with chokidar itself (eliminating browser-sync), it may be either in that package or in readdirp.

Edit: See also paulmillr/chokidar#1271 for fantastic detailed breakdown, which is what I think is also happening here.

[patricknelson]

Alright made some progress on this. I do believe this is a bug somewhere in chokidar or readdirp. I think the main trigger is:

1. Use of the **/ subdirectory glob.
2. A large number of files (maybe changing all at once or just over a long period of time, as long as they change)

So in my case, I was able to workaround the issue by changing my css/**/*.css in the files array to instead individually list each directory, e.g. css/sub/*.css. That allowed the memory usage to balance back out after a little while. Otherwise, it just bloats out of control until my VM runs out of RAM and browser-sync crashes.