Memory exhaustion after some time

[dsuarezv]

Issue details

Hi there,

I'm finding that browser-sync crashes because of memory exhaustion after a day or two running continuously. Here is the stack trace I get:

...
[Browsersync] Reloading Browsers...
[Browsersync] Reloading Browsers...

<--- Last few GCs --->

[2982:0x624d7c0] 66051559 ms: Mark-sweep (reduce) 2044.4 (2081.8) -> 2043.7 (2082.1) MB, 1055.1 / 10.8 ms  (average mu = 0.537, current mu = 0.018) allocation failure scavenge might not succeed
[2982:0x624d7c0] 66053046 ms: Mark-sweep (reduce) 2044.7 (2082.1) -> 2044.1 (2082.3) MB, 1478.2 / 8.3 ms  (average mu = 0.320, current mu = 0.006) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb09980 node::Abort() [gulp]
 2: 0xa1c235 node::FatalError(char const*, char const*) [gulp]
 3: 0xcf77be v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [gulp]
 4: 0xcf7b37 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [gulp]
 5: 0xeaf3d5  [gulp]
 6: 0xebf09d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [gulp]
 7: 0xec1d9e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [gulp]
 8: 0xe832da v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [gulp]
 9: 0x11fc026 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [gulp]
10: 0x15f0a99  [gulp]
Aborted (core dumped)

The machine is Linux 64bits running Ubuntu 21.

I don't know if it's related, but I'm including images and reloading if they change, maybe this is contributing to exhaust the memory more quickly. The images don't change very often, however.

I had core dumps disabled, I can enable them and send the dump if you need it.

Steps to reproduce/test case

See gulp script below. Just leave it running for a full day making changes here and there, it's crashed in the morning.

Please specify which version of Browsersync, node and npm you're running

• Browsersync: [2.27.9]
• Node: [16.14.0]
• Npm: [8.3.1]

Affected platforms

• linux
• windows
• OS X
• freebsd
• solaris
• other (please specify which)

Browsersync use-case

• API

• Gulp

• CLI

• Grunt

for all other use-cases, (gulp, grunt etc), please show us exactly how you're using Browsersync

var browserSync = require( 'browser-sync' ).create();
var gulp = require( 'gulp' );

var paths = {
    php: [ '*.php', '**/*.php' ],
    scripts: [ 'js/*.js', '*.js' ],
    styles: [ '*.css', 'css/*.css' ],
    images: [ '*.png', '*.jpg' ]
};

gulp.task( 'default', function() {
    browserSync.init({
        proxy: 'http://n1587.xxxxxxxxxx.com',
        host: 'n1587.xxxxxxxxxx.com',
        open: 'external',
        port: 8000
    });

    gulp.watch( paths.php ).on( 'change', browserSync.reload );
    gulp.watch( paths.scripts ).on( 'change', browserSync.reload );
    gulp.watch( paths.images ).on( 'change', browserSync.reload );
    gulp.watch( paths.styles ).on('change', function() {
        gulp.src( paths.styles ).pipe( browserSync.stream() );
    });
});

[patricknelson]

This is extremely easy for me to reproduce in my codebase, since I have many separate .scss files which compile to separate .css files. When I edit a globally included _shared.scss file (for example) it will cascade to recompiling about 91 files. Granted, that's a lot of files, but sass and postcss will crank through it pretty quick and painlessly.

However, running this as npx browser-sync start --config .browser-sync.cjs on a WSL2 VM with 4GB of RAM will crash after only 14 edits (simply commenting/uncommenting a shared .scss file). All 4GB get exhausted until it crashes. See the stack trace below.

Note: The Last few GCs will vary from crash to crash.

... very large number of these lines...
[Browsersync] File event [change] : css/themes.css
[Browsersync] File event [change] : css/main.css

<--- Last few GCs --->

[40874:0x74ddbf0]    52579 ms: Scavenge (reduce) 2045.5 (2083.1) -> 2045.3 (2083.6) MB, 9.15 / 0.00 ms  (average mu = 0.285, current mu = 0.303) allocation failure;
[40874:0x74ddbf0]    53803 ms: Mark-Compact (reduce) 2046.1 (2083.6) -> 2045.6 (2084.4) MB, 1218.87 / 0.00 ms  (average mu = 0.170, current mu = 0.025) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xc9e850 node::Abort() [node]
 2: 0xb720ff  [node]
 3: 0xec1a70 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 4: 0xec1d57 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 5: 0x10d3dc5  [node]
 6: 0x10ebc48 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 7: 0x10c1d61 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0x10c2ef5 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 9: 0x10a0466 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
10: 0x14fb386 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
11: 0x7ff26ee99ef6
Aborted

Here's the full contents of .browser-sync.cjs (it's pretty big, so put into a <details> field below).

Details

module.exports = {
	/**
	 * CUSTOMIZATIONS: Tweaks to the default config are below this line.
	 */
	"port": 3000,
	"listen": "0.0.0.0",
	"files": [
		'css/**/**.css', // CSS
		'templates/**/*.ss', // SilverStripe templates

		// JS (reloads page)
		'javascript/foot/**/*.js',
		'javascript/custom-elements/**/*.js',
		'javascript/head/**/*.js',
		'javascript/build/*.js',
	],
	"watchEvents": [
		"change"
	],
	"watch": true,
	"notify": false,


	/**
	 * DEFAULTS: Unchanged defaults are below this line.
	 */
	"ui": {
		"port": 3001
	},
	"ignore": [],
	"single": false,
	"watchOptions": {
		"ignoreInitial": true
	},
	"server": false,
	"proxy": false,
	"middleware": false,
	"serveStatic": [],
	"ghostMode": {
		"clicks": true,
		"scroll": true,
		"location": true,
		"forms": {
			"submit": true,
			"inputs": true,
			"toggles": true
		}
	},
	"logLevel": "info",
	"logPrefix": "Browsersync",
	"logConnections": false,
	"logFileChanges": true,
	"logSnippet": true,
	"rewriteRules": [],
	"open": "local",
	"browser": "default",
	"cors": false,
	"xip": false,
	"hostnameSuffix": false,
	"reloadOnRestart": false,
	"scrollProportionally": true,
	"scrollThrottle": 0,
	"scrollRestoreTechnique": "window.name",
	"scrollElements": [],
	"scrollElementMapping": [],
	"reloadDelay": 0,
	"reloadDebounce": 500,
	"reloadThrottle": 0,
	"plugins": [],
	"injectChanges": true,
	"startPath": null,
	"minify": true,
	"host": null,
	"localOnly": false,
	"codeSync": true,
	"timestamps": true,
	"clientEvents": [
		"scroll",
		"scroll:element",
		"input:text",
		"input:toggles",
		"form:submit",
		"form:reset",
		"click"
	],
	"socket": {
		"socketIoOptions": {
			"log": false
		},
		"socketIoClientConfig": {
			"reconnectionAttempts": 50
		},
		"path": "/browser-sync/socket.io",
		"clientPath": "/browser-sync",
		"namespace": "/browser-sync",
		"clients": {
			"heartbeatTimeout": 5000
		}
	},
	"tagNames": {
		"less": "link",
		"scss": "link",
		"css": "link",
		"jpg": "img",
		"jpeg": "img",
		"png": "img",
		"svg": "img",
		"gif": "img",
		"js": "script"
	},
	"injectNotification": false
};

[patricknelson]

p.s. Possibly related to #1083

[shakyShane]

Thanks @dsuarezv & @patricknelson - I'll look into this

[patricknelson]

Just circling back to this issue, too. See if it stops crashing after you remove **/. If that helps then I think it might be a deeper issue in chokidar or readdirp. See paulmillr/chokidar#1268 and for a nicely detailed breakdown, see paulmillr/chokidar#1271 (probably duplicates of each other).