mega memory use for JSON parsing in 0.10.x #5202

Closed
1o1brian opened this Issue Apr 2, 2013 · 4 comments

Projects

None yet

3 participants

@1o1brian
1o1brian commented Apr 2, 2013

When I load data from a json file (see link below) via require() or JSON.parse() in node 0.10.x I get:

FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory

The node process uses up about 1.25 GB of RAM before dying.

In node 0.8.x the node proces only uses about 100MB of memory. The JSON file uncompressed is about 2.8MB in size.

Here's the JSON file in question: https://dl.dropbox.com/u/10057662/data.json.bz2

@Mithgol
Mithgol commented Apr 3, 2013

Can confirm the issue on Windows XP 5.1.2600 with Node 0.10.2. (32bit system and Node.) The REPL dies after a mere require('./data.json') in a couple of minutes, and it takes several hundreds of megabytes of memory (both physical and virtual).

@indutny
Member
indutny commented Apr 3, 2013

For future reference, the most of this memory is allocated in v8::internal::HandleScope::Extend, basically Json parser creates a lot of handles without deallocating them.

@indutny
Member
indutny commented Apr 3, 2013

This seems to be fixing it, but I'm not sure at what price it comes:

diff --git a/deps/v8/src/json-parser.h b/deps/v8/src/json-parser.h
index 03ed22d..9844402 100644
--- a/deps/v8/src/json-parser.h
+++ b/deps/v8/src/json-parser.h
@@ -295,6 +295,7 @@ Handle<Object> JsonParser<seq_ascii>::ParseJsonObject() {
   AdvanceSkipWhitespace();
   if (c0_ != '}') {
     do {
+      HandleScope scope;
       if (c0_ != '"') return ReportUnexpectedCharacter();

       int start_position = position_;
@indutny
Member
indutny commented Apr 3, 2013

Fixed in 55d058e

@indutny indutny closed this Apr 3, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment