Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible memory leak in TLS(Wrap?) #1075

Closed
aredridel opened this Issue Mar 5, 2015 · 144 comments

Comments

Projects
None yet
@aredridel
Copy link
Contributor

aredridel commented Mar 5, 2015

I'm diagnosing a problem when I run paypal's npm proxy https://github.com/krakenjs/kappa when run on iojs. After 300 requests for a 50kb tarball, memory allocated grows from 250 mb base to 450 mb.

I've run it under valgrind, and there's some things that look a bit suspicious around tlswrap, smalloc and some other crypto related bits. The interesting bits of the valgrind report are:

==31542== 827,216 bytes in 656 blocks are still reachable in loss record 1,214 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C868B: asn1_enc_save (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C2866: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C14FF: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C1BAB: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C2F0F: ASN1_item_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67CC9E: ssl3_get_server_certificate (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x681323: ssl3_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67958B: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679E1A: ssl23_write (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDBCEF: _ZN4node7TLSWrap7ClearInEv.part.43 (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542== 
==31542== 1,343,488 bytes in 328 blocks are still reachable in loss record 1,215 of 1,223
==31542==    at 0x4C2AC27: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCD825D: node::NodeBIO::PeekWritable(unsigned long*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCD8EEC: node::TLSWrap::OnAllocImpl(unsigned long, uv_buf_t*, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xD031D0: uv__read (stream.c:1089)
==31542==    by 0xD03967: uv__stream_io (stream.c:1219)
==31542==    by 0xD08CEC: uv__io_poll (linux-core.c:319)
==31542==    by 0xCFA955: uv_run (core.c:324)
==31542==    by 0xC87650: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x5C8E76C: (below main) (libc-start.c:226)
==31542== 
==31542== 2,303,504 bytes in 131 blocks are still reachable in loss record 1,216 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BBAF: ssl3_setup_write_buffer (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BCB1: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679000: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679CBA: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCE78: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD2B4: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x801781: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x828E02: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A79060BA: ???
==31542==    by 0x1C66A7CDC7AD: ???
==31542== 
==31542== 2,323,416 bytes in 131 blocks are still reachable in loss record 1,217 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BD5F: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679000: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679CBA: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCE78: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD2B4: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x801781: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x828E02: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A79060BA: ???
==31542==    by 0x1C66A7CDC7AD: ???
==31542==    by 0x1C66A7CDC562: ???
==31542== 
==31542== 3,464,048 bytes in 197 blocks are still reachable in loss record 1,218 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BBAF: ssl3_setup_write_buffer (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BCB1: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679000: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679CBA: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCE78: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD2B4: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A7A43345: ???
==31542==    by 0x1C66A7EA169A: ???
==31542==    by 0x1C66A7CDC562: ???
==31542==    by 0x1C66A7924BC5: ???
==31542== 
==31542== 3,493,992 bytes in 197 blocks are still reachable in loss record 1,219 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BD5F: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679000: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679CBA: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCE78: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD2B4: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A7A43345: ???
==31542==    by 0x1C66A7EA169A: ???
==31542==    by 0x1C66A7CDC562: ???
==31542==    by 0x1C66A7924BC5: ???
==31542==    by 0x1C66A7C08177: ???
==31542== 
==31542== 5,373,952 bytes in 328 blocks are still reachable in loss record 1,220 of 1,223
==31542==    at 0x4C2AC27: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCD8325: node::NodeBIO::Commit(unsigned long) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD745: node::TLSWrap::OnReadImpl(long, uv_buf_t const*, uv_handle_type, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xD0335E: uv__read (stream.c:1156)
==31542==    by 0xD03967: uv__stream_io (stream.c:1219)
==31542==    by 0xD08CEC: uv__io_poll (linux-core.c:319)
==31542==    by 0xCFA955: uv_run (core.c:324)
==31542==    by 0xC87650: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x5C8E76C: (below main) (libc-start.c:226)
==31542== 
==31542== 7,422,722 bytes in 131 blocks are still reachable in loss record 1,221 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCAD6A2: node::smalloc::Alloc(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x801781: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x828E02: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A79060BA: ???
==31542==    by 0x1C66A79B20B8: ???
==31542==    by 0x1C66A791EA54: ???
==31542==    by 0x1C66A791EDA0: ???
==31542==    by 0x1C66A7D088BE: ???
==31542==    by 0x1C66A7D084D4: ???
==31542==    by 0x1C66A7D08272: ???
==31542==    by 0x1C66A7924BC5: ???
==31542== 
==31542== 11,162,414 bytes in 197 blocks are still reachable in loss record 1,222 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCAD6A2: node::smalloc::Alloc(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x801781: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x828E02: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A79060BA: ???
==31542==    by 0x1C66A79B20B8: ???
==31542==    by 0x1C66A791EA54: ???
==31542==    by 0x1C66A791EDA0: ???
==31542==    by 0x1C66A7EA5607: ???
==31542==    by 0x1C66A7D08272: ???
==31542==    by 0x1C66A7924BC5: ???
==31542==    by 0x1C66A7C08177: ???
==31542== 
==31542== 18,591,935 bytes in 20,677 blocks are still reachable in loss record 1,223 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCD8E91: node::TLSWrap::OnAllocSelf(unsigned long, uv_buf_t*, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCEA0: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD7B7: node::TLSWrap::OnReadImpl(long, uv_buf_t const*, uv_handle_type, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xD0335E: uv__read (stream.c:1156)
==31542==    by 0xD03967: uv__stream_io (stream.c:1219)
==31542==    by 0xD08CEC: uv__io_poll (linux-core.c:319)
==31542==    by 0xCFA955: uv_run (core.c:324)
==31542==    by 0xC87650: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x5C8E76C: (below main) (libc-start.c:226)
==31542== 
==31542== LEAK SUMMARY:
==31542==    definitely lost: 10,464 bytes in 327 blocks
==31542==    indirectly lost: 63,532 bytes in 3,258 blocks
==31542==      possibly lost: 4,568 bytes in 64 blocks
==31542==    still reachable: 69,011,541 bytes in 206,575 blocks
==31542==         suppressed: 0 bytes in 0 blocks
==31542== 
==31542== For counts of detected and suppressed errors, rerun with: -v
==31542== ERROR SUMMARY: 42 errors from 42 contexts (suppressed: 2 from 2)

Any help or suggestions for what to look at to track this down would be wonderful.

Node 0.12 and 0.10 don't show this behavior with the same code (we're stress-testing io.js in parallel to shake stuff like this loose)

@rvagg

This comment has been minimized.

Copy link
Member

rvagg commented Mar 5, 2015

@bnoordhuis

This comment has been minimized.

Copy link
Member

bnoordhuis commented Mar 5, 2015

I wonder if there is a handle leak. I noticed that TLSWrap::OnReadSelf() creates a JS buffer object without a HandleScope. Does this patch make a difference?

diff --git a/src/tls_wrap.cc b/src/tls_wrap.cc
index 1647ab6..2b00d9e 100644
--- a/src/tls_wrap.cc
+++ b/src/tls_wrap.cc
@@ -641,6 +641,7 @@ void TLSWrap::OnReadSelf(ssize_t nread,
                          uv_handle_type pending,
                          void* ctx) {
   TLSWrap* wrap = static_cast<TLSWrap*>(ctx);
+  HandleScope handle_scope(wrap->env()->isolate());
   Local<Object> buf_obj;
   if (buf != nullptr)
     buf_obj = Buffer::Use(wrap->env(), buf->base, buf->len);
@jasisk

This comment has been minimized.

Copy link

jasisk commented Mar 5, 2015

Providing a tiny bit of additional context: this is a graph of memory usage across 6 instances of our application from a few weeks back (4x node v0.10.3?, 1x node v0.12.0 in orange, 1x iojs v1.3.0 in red). Continuing to see the same behavior in 1.4.x.

rss

The application is a very simple proxy, seeing highs of ~350 reqs/sec though more consistently ~80-100. Each incoming request results in 1 to 4 outbound requests.

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 5, 2015

Looking.

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 5, 2015

It seems to be leaking TLSWrap instances, wtf.

@indutny indutny self-assigned this Mar 5, 2015

indutny added a commit to indutny/io.js that referenced this issue Mar 5, 2015

stream_wrap: add HandleScope's in uv callbacks
Ensure that no handles will leak into global HandleScope by adding
HandleScope's in all JS-calling libuv callbacks in `stream_wrap.cc`.

Fix: nodejs#1075
@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 5, 2015

Partial fix here: #1078

@GeoffreyPlitt

This comment has been minimized.

Copy link

GeoffreyPlitt commented Mar 5, 2015

I've got a test here that reproduces the problem, comparing HTTP and HTTPS on 0.10, 0.12, and iojs.

https://github.com/GeoffreyPlitt/iojs_tls_bug

I'm going to try the partial fix above and see if it fixes my tests.

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 6, 2015

It seems like these lines are partly responsible for this problem: https://github.com/iojs/io.js/blob/b27931b0fedc5fad638d637525aba84b2754ed5f/lib/_tls_wrap.js#L290-L305

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 6, 2015

Found a JSStream leak, but so far don't have much progress on the main issue. It seems that cross-reference between StreamWrap and TLSWrap creates some non-collectible cycle, which should not be really possible. Most likely some bug in my code, but I can't say what exactly it is yet. Will continue tomorrow.

@Qard

This comment has been minimized.

Copy link
Member

Qard commented Mar 6, 2015

Any particular reason that existence check/apply in the proxied methods runs on every call? Can it not be:

// Proxy HandleWrap, PipeWrap and TCPWrap methods
proxiedMethods.forEach(function(name) {
  res[name] = (handle[name] || function () {}).bind(handle);
});
@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 6, 2015

@Qard no reason, all of these methods are usually called once per handle lifetime.

@brendanashworth brendanashworth added the tls label Mar 6, 2015

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 6, 2015

Looks like my patch is fixing the issue.

indutny added a commit that referenced this issue Mar 6, 2015

stream_wrap: add HandleScope's in uv callbacks
Ensure that no handles will leak into global HandleScope by adding
HandleScope's in all JS-calling libuv callbacks in `stream_wrap.cc`.

Fix: #1075
PR-URL: #1078
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 6, 2015

Should be fixed now, please confirm! ;)

@gm112

This comment has been minimized.

Copy link

gm112 commented Mar 6, 2015

This bug has been causing me a lot of issues. I'll certainly build with this change and let you know how goes the memory usage.

@rvagg

This comment has been minimized.

Copy link
Member

rvagg commented Mar 6, 2015

https://iojs.org/download/nightly/v1.4.4-nightly20150306dee07e2983/

Nightly with this change is there, please test, would love to have some feedback by the time 1.5.0 goes out later today.

@jasisk

This comment has been minimized.

Copy link

jasisk commented Mar 6, 2015

Deployed the nightly. Will beat on it for a couple of hours and report back.

(fwiw, I know @aredridel reran her benchmark and saw the issue resolved as well.)

@gm112

This comment has been minimized.

Copy link

gm112 commented Mar 6, 2015

I can confirm that for me, my memory usage is certainly much more stable. Before I would climb to several hundred megs with no problems.

I look forward to there being a full blown release.

@jasisk

This comment has been minimized.

Copy link

jasisk commented Mar 6, 2015

Really early observations on the 1.4.4-nightly suggest there may be more. @aredridel is profiling it now.

blue: 0.10, purple: 0.12, green: 1.4.4-nightly

rss

@aredridel

This comment has been minimized.

Copy link
Contributor Author

aredridel commented Mar 6, 2015

Without valgrind, it seems to have leveled things off somewhat.

However, it seems to grow still once I take a deeper look. Here's the larger chunks of valgrind's output:

==27619== 327,760 bytes in 482 blocks are still reachable in loss record 1,244 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x6E8E9E: EVP_CipherInit_ex (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x6608F6: tls1_change_cipher_state (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x642A6A: ssl3_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63B52F: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63BD81: ssl23_write (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D1CF: node::TLSWrap::ClearIn() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D727: node::TLSWrap::DoRead(long, uv_buf_t const*, uv_handle_type) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBECBDA: node::StreamWrap::OnRead(uv_stream_s*, long, uv_buf_t const*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC3235D: uv__read (stream.c:1156)
==27619==    by 0xC32E8A: uv__stream_io (stream.c:1219)
==27619== 
==27619== 332,520 bytes in 489 blocks are still reachable in loss record 1,245 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x6E8E9E: EVP_CipherInit_ex (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x6608F6: tls1_change_cipher_state (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x648150: ssl3_do_change_cipher_spec (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x648F5E: ssl3_read_bytes (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63D657: ssl3_get_message (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63CFDC: ssl3_get_finished (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x642CAA: ssl3_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x64807F: ssl3_write_bytes (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D1CF: node::TLSWrap::ClearIn() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D727: node::TLSWrap::DoRead(long, uv_buf_t const*, uv_handle_type) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619== 
==27619== 356,000 bytes in 8,900 blocks are still reachable in loss record 1,246 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x66BAC8: c2i_ASN1_OBJECT (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67C7FA: asn1_ex_c2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67CBE4: asn1_d2i_ex_primitive (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DA2B: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DE2E: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E063: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D4E4: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DCED: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DFD5: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D4E4: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619== 
==27619== 366,592 bytes in 358 blocks are still reachable in loss record 1,247 of 1,265
==27619==    at 0x4C2AC27: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0xC0A7C3: node::NodeBIO::Write(char const*, unsigned long) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0A033: node::NodeBIO::Write(bio_st*, char const*, int) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x689237: BIO_write (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63BE9C: ssl23_write_bytes (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63AF21: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63BC21: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CAFD: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CA1E: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A4743605: ???
==27619==    by 0x3E39A4B880FE: ???
==27619==    by 0x3E39A4A3DE02: ???
==27619== 
==27619== 379,640 bytes in 2,264 blocks are still reachable in loss record 1,248 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x6A3501: BUF_MEM_grow (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x682B03: x509_name_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D1CB: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DE2E: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E063: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D4E4: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DE2E: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E063: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D4E4: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E193: ASN1_item_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619== 
==27619== 418,760 bytes in 10,469 blocks are still reachable in loss record 1,249 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x66BAC8: c2i_ASN1_OBJECT (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67C7FA: asn1_ex_c2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67CBE4: asn1_d2i_ex_primitive (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DA2B: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DE2E: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E063: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D4E4: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DCED: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E063: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D92E: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619== 
==27619== 419,840 bytes in 410 blocks are still reachable in loss record 1,250 of 1,265
==27619==    at 0x4C2AC27: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0xC0A7C3: node::NodeBIO::Write(char const*, unsigned long) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D652: node::TLSWrap::DoWrite(node::WriteWrap*, uv_buf_t*, unsigned long, uv_stream_s*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBEBADC: int node::StreamBase::WriteString<(node::encoding)4>(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0E9A4: void node::StreamBase::JSMethod<node::TLSWrap, &(int node::StreamBase::WriteString<(node::encoding)4>(v8::FunctionCallbackInfo<v8::Value> const&))>(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7A0145: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7C7F47: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A46060BA: ???
==27619==    by 0x3E39A4B343B0: ???
==27619==    by 0x3E39A490E90E: ???
==27619==    by 0x3E39A49DE737: ???
==27619==    by 0x3E39A4624BC5: ???
==27619== 
==27619== 471,600 bytes in 393 blocks are still reachable in loss record 1,251 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x64539F: ssl3_new (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x662200: tls1_new (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x65735A: SSL_new (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0B472: node::TLSWrap::TLSWrap(node::Environment*, node::crypto::SSLWrap<node::TLSWrap>::Kind, node::StreamBase*, node::crypto::SecureContext*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0C59F: node::TLSWrap::Wrap(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7A0145: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7C7F47: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A46060BA: ???
==27619==    by 0x3E39A49F0239: ???
==27619==    by 0x3E39A49EFD6A: ???
==27619== 
==27619== 524,280 bytes in 1 blocks are still reachable in loss record 1,252 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x77C42A: v8::internal::Malloced::New(unsigned long) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x8ED0A3: v8::internal::HeapProfiler::DefineWrapperClass(unsigned short, v8::RetainedObjectInfo* (*)(unsigned short, v8::Handle<v8::Value>)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBC7A13: node::Binding(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7A0145: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7C7F47: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A46060BA: ???
==27619==    by 0x3E39A4670CC4: ???
==27619==    by 0x3E39A461EA54: ???
==27619==    by 0x3E39A466B52D: ???
==27619==    by 0x3E39A466AD78: ???
==27619==    by 0x3E39A466E272: ???
==27619== 
==27619== 559,838 bytes in 8,900 blocks are still reachable in loss record 1,253 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x671A3C: ASN1_STRING_set (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67C991: asn1_ex_c2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67CBE4: asn1_d2i_ex_primitive (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DA2B: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DE2E: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E063: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D4E4: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DCED: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DFD5: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D4E4: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619== 
==27619== 739,392 bytes in 30,808 blocks are still reachable in loss record 1,254 of 1,265
==27619==    at 0x4C2B1C7: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0xBE4BB5: node::smalloc::Alloc(node::Environment*, v8::Handle<v8::Object>, char*, unsigned long, v8::ExternalArrayType) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBCC4EA: node::Buffer::Use(node::Environment*, char*, unsigned int) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0B8D8: node::TLSWrap::OnReadSelf(long, uv_buf_t const*, uv_handle_type, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CBAD: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D72F: node::TLSWrap::DoRead(long, uv_buf_t const*, uv_handle_type) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBECBDA: node::StreamWrap::OnRead(uv_stream_s*, long, uv_buf_t const*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC3235D: uv__read (stream.c:1156)
==27619==    by 0xC32E8A: uv__stream_io (stream.c:1219)
==27619==    by 0xC37F01: uv__io_poll (linux-core.c:319)
==27619==    by 0xC29D09: uv_run (core.c:324)
==27619==    by 0xBC9220: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619== 
==27619== 938,448 bytes in 114 blocks are still reachable in loss record 1,255 of 1,265
==27619==    at 0x4C2B1C7: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x8E9506: v8::internal::GlobalHandles::Create(v8::internal::Object*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBE4BCA: node::smalloc::Alloc(node::Environment*, v8::Handle<v8::Object>, char*, unsigned long, v8::ExternalArrayType) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBCC4EA: node::Buffer::Use(node::Environment*, char*, unsigned int) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0B8D8: node::TLSWrap::OnReadSelf(long, uv_buf_t const*, uv_handle_type, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CBAD: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D72F: node::TLSWrap::DoRead(long, uv_buf_t const*, uv_handle_type) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBECBDA: node::StreamWrap::OnRead(uv_stream_s*, long, uv_buf_t const*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC3235D: uv__read (stream.c:1156)
==27619==    by 0xC32E8A: uv__stream_io (stream.c:1219)
==27619==    by 0xC37F01: uv__io_poll (linux-core.c:319)
==27619==    by 0xC29D09: uv_run (core.c:324)
==27619== 
==27619== 1,215,604 bytes in 964 blocks are still reachable in loss record 1,256 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x680F2F: asn1_enc_save (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DAD1: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67DE2E: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E063: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67D4E4: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x67E193: ASN1_item_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63E927: ssl3_get_server_certificate (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x642EF1: ssl3_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63B52F: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63BD81: ssl23_write (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619== 
==27619== 2,002,944 bytes in 489 blocks are still reachable in loss record 1,257 of 1,265
==27619==    at 0x4C2AC27: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0xC0AD91: node::NodeBIO::PeekWritable(unsigned long*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0B829: node::TLSWrap::OnAllocImpl(unsigned long, uv_buf_t*, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBECB0C: node::StreamWrap::OnAlloc(uv_handle_s*, unsigned long, uv_buf_t*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC320CA: uv__read (stream.c:1089)
==27619==    by 0xC32E8A: uv__stream_io (stream.c:1219)
==27619==    by 0xC37F01: uv__io_poll (linux-core.c:319)
==27619==    by 0xC29D09: uv_run (core.c:324)
==27619==    by 0xBC9220: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x5C8E76C: (below main) (libc-start.c:226)
==27619== 
==27619== 2,303,504 bytes in 131 blocks are still reachable in loss record 1,258 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63DB07: ssl3_setup_write_buffer (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63DB6E: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63B02B: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63BC21: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CAFD: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CA1E: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7A0145: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7C7F47: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A46060BA: ???
==27619==    by 0x3E39A4A3E0A7: ???
==27619== 
==27619== 2,323,416 bytes in 131 blocks are still reachable in loss record 1,259 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63D9DD: ssl3_setup_read_buffer (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63DB60: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63B02B: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63BC21: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CAFD: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CA1E: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7A0145: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7C7F47: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A46060BA: ???
==27619==    by 0x3E39A4A3E0A7: ???
==27619== 
==27619== 6,295,072 bytes in 358 blocks are still reachable in loss record 1,260 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63DB07: ssl3_setup_write_buffer (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63DB6E: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63B02B: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63BC21: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CAFD: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CA1E: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A4743605: ???
==27619==    by 0x3E39A4B880FE: ???
==27619==    by 0x3E39A4A3DE02: ???
==27619==    by 0x3E39A4624BC5: ???
==27619== 
==27619== 6,349,488 bytes in 358 blocks are still reachable in loss record 1,261 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0x6F6004: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63D9DD: ssl3_setup_read_buffer (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63DB60: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63B02B: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x63BC21: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CAFD: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CA1E: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A4743605: ???
==27619==    by 0x3E39A4B880FE: ???
==27619==    by 0x3E39A4A3DE02: ???
==27619==    by 0x3E39A4624BC5: ???
==27619== 
==27619== 7,422,722 bytes in 131 blocks are still reachable in loss record 1,262 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0xBE4AA8: node::smalloc::Alloc(node::Environment*, v8::Handle<v8::Object>, unsigned long, v8::ExternalArrayType) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBE49D3: node::smalloc::Alloc(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7A0145: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7C7F47: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A46060BA: ???
==27619==    by 0x3E39A46B2198: ???
==27619==    by 0x3E39A461EA54: ???
==27619==    by 0x3E39A461EDA0: ???
==27619==    by 0x3E39A4A54D9E: ???
==27619==    by 0x3E39A4A549B4: ???
==27619==    by 0x3E39A4A54752: ???
==27619== 
==27619== 8,011,776 bytes in 489 blocks are still reachable in loss record 1,263 of 1,265
==27619==    at 0x4C2AC27: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0xC0AE7A: node::NodeBIO::Commit(unsigned long) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D6FE: node::TLSWrap::DoRead(long, uv_buf_t const*, uv_handle_type) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBECBDA: node::StreamWrap::OnRead(uv_stream_s*, long, uv_buf_t const*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC3235D: uv__read (stream.c:1156)
==27619==    by 0xC32E8A: uv__stream_io (stream.c:1219)
==27619==    by 0xC37F01: uv__io_poll (linux-core.c:319)
==27619==    by 0xC29D09: uv_run (core.c:324)
==27619==    by 0xBC9220: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x5C8E76C: (below main) (libc-start.c:226)
==27619== 
==27619== 20,284,996 bytes in 358 blocks are still reachable in loss record 1,264 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0xBE4AA8: node::smalloc::Alloc(node::Environment*, v8::Handle<v8::Object>, unsigned long, v8::ExternalArrayType) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBE49D3: node::smalloc::Alloc(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7A0145: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x7C7F47: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x3E39A46060BA: ???
==27619==    by 0x3E39A46B2198: ???
==27619==    by 0x3E39A461EA54: ???
==27619==    by 0x3E39A461EDA0: ???
==27619==    by 0x3E39A4B8B0C7: ???
==27619==    by 0x3E39A4A54752: ???
==27619==    by 0x3E39A4624BC5: ???
==27619== 
==27619== 27,708,244 bytes in 30,808 blocks are still reachable in loss record 1,265 of 1,265
==27619==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27619==    by 0xC0B881: node::TLSWrap::OnAllocSelf(unsigned long, uv_buf_t*, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0CB69: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC0D72F: node::TLSWrap::DoRead(long, uv_buf_t const*, uv_handle_type) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xBECBDA: node::StreamWrap::OnRead(uv_stream_s*, long, uv_buf_t const*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0xC3235D: uv__read (stream.c:1156)
==27619==    by 0xC32E8A: uv__stream_io (stream.c:1219)
==27619==    by 0xC37F01: uv__io_poll (linux-core.c:319)
==27619==    by 0xC29D09: uv_run (core.c:324)
==27619==    by 0xBC9220: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==27619==    by 0x5C8E76C: (below main) (libc-start.c:226)
==27619== 
==27619== LEAK SUMMARY:
==27619==    definitely lost: 15,616 bytes in 488 blocks
==27619==    indirectly lost: 94,945 bytes in 4,868 blocks
==27619==      possibly lost: 4,369 bytes in 54 blocks
==27619==    still reachable: 101,753,796 bytes in 297,899 blocks
==27619==         suppressed: 0 bytes in 0 blocks
==27619== 
==27619== For counts of detected and suppressed errors, rerun with: -v
==27619== ERROR SUMMARY: 42 errors from 42 contexts (suppressed: 2 from 2)
@aredridel

This comment has been minimized.

Copy link
Contributor Author

aredridel commented Mar 6, 2015

(ignore the v1.4.2 in the paths -- the actual binary being run is compiled from the pr/1078 branch)

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Mar 6, 2015

@aredridel this is bad news, as it could be related to https://code.google.com/p/v8/issues/detail?id=3949&thanks=3949&ts=1425654431 which I was experiencing with JSStream during debugging.

Being not able to reproduce this issue myself, I've pushed suggested patch to the: https://github.com/indutny/io.js/tree/test/paypal-leak . May I ask you to give it a try?

Thanks a lot!

@bnoordhuis

This comment has been minimized.

Copy link
Member

bnoordhuis commented Mar 6, 2015

@aredridel It's a crude hack but can you try this patch? If it keeps running, great. If it aborts, there's still a handle leak somewhere.

@aredridel

This comment has been minimized.

Copy link
Contributor Author

aredridel commented Mar 6, 2015

Building now!

The actual instrumentation I'm using is the kappa package, configured to talk to a private (empty) npm registry over http, then to https://registry.npmjs.org/, then logging via good. The bulk of the work is that proxy, try each, moving on if there's a 404. I just tested with ab -c 10 -n 1000 on it. Pretty simple overall, doing https as a client from that hapi proxy.

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Apr 3, 2015

@jasisk could you please give a try to #1330 ? I hope it'll help

@jasisk

This comment has been minimized.

Copy link

jasisk commented Apr 3, 2015

👍 deploying now.

@Fishrock123

This comment has been minimized.

Copy link
Member

Fishrock123 commented Apr 6, 2015

@jasisk How's it looking? :D

Fishrock123 added a commit that referenced this issue Apr 6, 2015

2015-04-06 io.js v1.6.4 Release
Notable changes:

* npm: upgrade npm to 2.7.5. See the npm CHANGELOG.md for details.
Includes two important security fixes.
https://github.com/npm/npm/blob/master/CHANGELOG.md#v275-2015-03-26
* openssl: preliminary work has been done for an upcoming upgrade to
OpenSSL 1.0.2a #1325 (Shigeki Ohtsu). See #589 for additional details.
* timers: a minor memory leak when timers are unreferenced was fixed,
alongside some related timers issues #1330 (Fedor Indutny). This
appears to have fixed the remaining leak reported in #1075.
* android: it is now possible to compile io.js for Android and related
devices #1307 (Giovanny Andres Gongora Granada).
@jasisk

This comment has been minimized.

Copy link

jasisk commented Apr 6, 2015

Hasn't calmed down yet but it's looking really good! I've learned to hold off for a few days—for some reason we've historically seen explosive memory growth on the order of 120mb / day, starting a couple of days in—but can't ask for a better graph so far.

purple: 0.12, blue: 1.x
RSS

@Fishrock123

This comment has been minimized.

Copy link
Member

Fishrock123 commented Apr 10, 2015

@jasisk Mind giving an update? :D

@jasisk

This comment has been minimized.

Copy link

jasisk commented Apr 10, 2015

Was subscribing to, "no news is good news." 😀

These two machines are still instrumented to take heap snapshots periodically which is why it's not super clean, however they're acting similarly which makes me think you guys nailed it!

blue: iojs, purple: 0.12
RSS

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Apr 10, 2015

Hooray! I suggest closing the issue then, and reopening it in case of any trouble ;)

@jasisk

This comment has been minimized.

Copy link

jasisk commented Apr 10, 2015

👍

@Fishrock123

This comment has been minimized.

Copy link
Member

Fishrock123 commented Apr 10, 2015

Indeed. I'd gone ahead and said this was fixed as we knew in the 1.6.4 release. Closing, hopefully for good! :)

@JCMais

This comment has been minimized.

Copy link
Contributor

JCMais commented Apr 10, 2015

Congratulations to everyone envolved. I was accompanying this issue, and was one of the best efforts to exterminate memory leaks that I've seen in my short history with open source projects. 👏

rvagg added a commit that referenced this issue Apr 14, 2015

2015-04-14 io.js v1.7.0 Release
Notable changes:

* C++ API: Fedor Indutny contributed a feature to V8 which has been
  backported to the V8 bundled in io.js. SealHandleScope allows a C++
  add-on author to seal a HandleScope to prevent further, unintended
  allocations within it. Currently only enabled for debug builds of
  io.js. This feature helped detect the leak in #1075 and is now
  activated on the root HandleScope in io.js. (Fedor Indutny) #1395.
* ARM: This release includes significant work to improve the state of
  ARM support for builds and tests. The io.js CI cluster's ARMv6,
  ARMv7 and ARMv8 build servers are now all (mostly) reporting passing
  builds and tests.
  - ARMv8 64-bit (AARCH64) is now properly supported, including a
    backported fix in libuv that was mistakenly detecting the
    existence of `epoll_wait()`. (Ben Noordhuis) #1365.
  - ARMv6: #1376 reported a problem with Math.exp() on ARMv6 (incl
    Raspberry Pi). The culprit is erroneous codegen for ARMv6 when
    using the "fast math" feature of V8. --nofast_math has been turned
    on for all ARMv6 variants by default to avoid this, fast math can
    be turned back on with --fast_math. (Ben Noordhuis) #1398.
  - Tests: timeouts have been tuned specifically for slower platforms,
    detected as ARMv6 and ARMv7. (Roman Reiss) #1366.
* npm: Upgrade npm to 2.7.6. See the release notes
  (https://github.com/npm/npm/releases/tag/v2.7.6) for details.
@capaj

This comment has been minimized.

Copy link

capaj commented Apr 14, 2015

wow they must love @indutny at google V8 team. Pushing features just so iojs can fix memory leak. Great stuff 👌

@mscreenie

This comment has been minimized.

Copy link

mscreenie commented Apr 14, 2015

Congrats, your commitment to this problem and enthusiasm is outstanding and infectious. 👍

@indutny

This comment has been minimized.

Copy link
Member

indutny commented Apr 14, 2015

Thanks guys!

@interstellerS

This comment has been minimized.

Copy link

interstellerS commented Apr 16, 2015

This thread was exciting to watch as a node/io newbie , inspired by @indutny work . 👍

@devinus

This comment has been minimized.

@gm112

This comment has been minimized.

Copy link

gm112 commented Apr 16, 2015

@devinus probably node-inspector or the blink developer tools

@devinus

This comment has been minimized.

Copy link

devinus commented Apr 16, 2015

@gm112 My first guess was node-inspector, but it doesn't support io.js. I'm not sure how it could be Blink developer tools either.

@gm112

This comment has been minimized.

Copy link

gm112 commented Apr 16, 2015

@devinus You can tell Blink Developer tools to connect to a remote debugging session, though I'm not 100% sure if this is just on v8 or a remote chrome instance. However Node-Inspector has limited support on iojs, some things work and some things don't. But I could be wrong :)

@Olegas

This comment has been minimized.

Copy link
Contributor

Olegas commented Apr 16, 2015

@devinus I think this was a heap snapshot file generated by https://github.com/bnoordhuis/node-heapdump and opened in Chrome Developer Tools.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.