High memory usage

[c0deslinger]

Hi, your code is very good and works well, but when I use it the memory usage goes up quite a lot and when the scraping is finished the memory doesn't come back to normal. Are you sure you closed the goroutines when they were finished?

[gosom]

thank you @c0deslinger .

The application starts on the background N browsers (chrome). This is memory intensive.

When scraping finishes the application does not exit.

Although indeed I can confirm that memory usage does not drop

smem |grep -w 'google-maps-scraper'

36621 giorgos  ./google-maps-scraper -dept        0   704424   704432   705612

It does not look that any goroutines are still running, but I need to check again the "parent" project scrapemate.

Although, this "problem" does not look a blocker. Even with a large number of queries the memory increases but at some point stops increasing.

you may try to tune the garbage collector frequency (use the GOGC environment variable). I noticed less memory usage when the GC triggers more often.

Also please read this: https://tip.golang.org/doc/gc-guide

The section 'A note about virtual memory', but I am not sure if that's the case here.

this requires deeper investigation since I didn't notice any obvious issue in code.

I will leave the issue open. Maybe someone assists here

[gosom]

closing issue because of inactivity

[arceushui]

Hi, I'm also facing the high memory issue and I noticed playwright took out a lot of space. I attached my local heap profile for reference and this is after finishing all the jobs and awaiting for jobs.

The high memory issue is also due to ms-playwright but this can't really be resolved