-
-
Notifications
You must be signed in to change notification settings - Fork 146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Big Bug - Memory leak when use aso lib #12
Labels
Comments
this is most likely a consequence of an unbounded memoization of the
scrapers. We should switch to the most recent version of the scrapers and
configure the memoization to limit the amount of cached results.
El ene 18, 2018 11:41 AM, "SpieGame" <notifications@github.com> escribió:
Hi facundoolano, thank you very much for very useful lib.
I use you lib to aso on google play without any problem. But when i try to
get score from over 14 keywords sequentially. It came out with error:
`<--- Last few GCs --->
692423 ms: Mark-sweep 1256.1 (1434.6) -> 1244.3 (1434.6) MB, 83.0 / 0.0 ms
[al
location failure] [GC in old space requested].
692501 ms: Mark-sweep 1244.3 (1434.6) -> 1244.2 (1434.6) MB, 77.8 / 0.0 ms
[al
location failure] [GC in old space requested].
692585 ms: Mark-sweep 1244.2 (1434.6) -> 1253.3 (1403.6) MB, 84.1 / 0.0 ms
[la
st resort gc].
692663 ms: Mark-sweep 1253.3 (1403.6) -> 1262.2 (1403.6) MB, 78.3 / 0.0 ms
[la
st resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 000000E62C0CFB49
1: slowToString [buffer.js:459] [pc=000002CB180F33F5] (this=0000010137FA80C1
<an Uint8Array with map 000001A00EA067D1>,encoding=000000E62C0DBC31
<String[4]:
utf8>,start=0,end=316600)
2: arguments adaptor frame: 1->3
3: toString [buffer.js:~487] [pc=000002CB17FD6C21] (this=0000010137FA80C1 <a
n Uint8Array with map 000001A00EA067D1>)
4: arguments adaptor frame: 1->0
5: /* ano...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of
memo
ry`
Then i try to debug this, and find out, when use aso lib, it will take very
large memory, and nodejs don't gc this. I try with simple code like this(
add option --expose-gc when execute this):
`var aso = require('aso')('gplay');
// Get memory usage
var heapUsed = process.memoryUsage().heapUsed;
console.log("Program is using " + heapUsed + " bytes of Heap.")
aso.scores("coin").then(function(data)
{
// Get memory usage
var heapUsed = process.memoryUsage().heapUsed;
console.log("Program is using " + heapUsed + " bytes of Heap.")
aso.scores("fun").then(function(a)
{
// Get memory usage
var heapUsed = process.memoryUsage().heapUsed;
console.log("Program is using " + heapUsed + " bytes of Heap.")
aso.scores("game").then(function(b)
{
// Get memory usage
var heapUsed = process.memoryUsage().heapUsed;
console.log("Program is using " + heapUsed + " bytes of Heap.")
aso.scores("coin").then(function(c)
{
// Get memory usage
var heapUsed = process.memoryUsage().heapUsed;
console.log("Program is using " + heapUsed + "
bytes of Heap.")
});
});
});
});
`
Result very bad:
Program is using 80766688 bytes of Heap. Program is using 213251752 bytes
of Heap. Program is using 390810216 bytes of Heap. Program is using
756238712 bytes of Heap. Program is using 796911968 bytes of Heap.
I think many object of aso lib can't destroy when gc working. Hope you will
find out what is the issue and fix it. Thank you very much!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#12>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA_iLSkaCRRaiFiZ4TAMea_9hNYeFjQpks5tL1g1gaJpZM4RjBj1>
.
|
So i must install your google-play-scraper with recent update or check your code of aso library to find then configure the memoization to limit the amount of cached results? Can you show me script name to edit/configure this? Thanks! |
Thank you very much! |
this should be fixed when #22 is done |
Awesome! Thank you! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi facundoolano, thank you very much for very useful lib.
I use you lib to aso on google play without any problem. But when i try to get score from over 14 keywords sequentially. It came out with error:
Then i try to debug this, and find out, when use aso lib, it will take very large memory, and nodejs don't gc this. I try with simple code like this:
Result very bad:
I think many object of aso lib can't destroy when gc working. Hope you will find out what is the issue and fix it. Thank you very much!
The text was updated successfully, but these errors were encountered: