Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crawler example crashes in libcurl 7.64-DEV --with-nghttp2 #3463

Closed
jeroen opened this issue Jan 11, 2019 · 3 comments

Comments

Projects
None yet
3 participants
@jeroen
Copy link
Contributor

commented Jan 11, 2019

I tried running the crawler.c example from the docs on the master branch. However when building curl --with-nghttp2 the example occasionally crashes.

This is curl@ba243235ec04af62aee2cfc31bf0f05488e59fe7 with nghttp2 1.35.1. Below a few backtraces of where it may crash (the common denominator seems to be http2_connisdead())

Process 18044 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x228)
    frame #0: 0x00000001001551bd libcurl.4.dylib`h2_session_send(data=0x0000000100837808, h2=0x000000010150b000) at http2.c:1512
   1509
   1510	    H2BUGF(infof(data, "Queuing PRIORITY on stream %u (easy %p)\n",
   1511	                 stream->stream_id, data));
-> 1512	    rv = nghttp2_submit_priority(h2, NGHTTP2_FLAG_NONE, stream->stream_id,
   1513	                                 &pri_spec);
   1514	    if(rv)
   1515	      return rv;
Target 0: (crawler.exe) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x228)
  * frame #0: 0x00000001001551bd libcurl.4.dylib`h2_session_send(data=0x0000000100837808, h2=0x000000010150b000) at http2.c:1512
    frame #1: 0x0000000100153474 libcurl.4.dylib`h2_process_pending_input(conn=0x0000000103809408, httpc=0x0000000103809a70, err=0x00007ffeefbff384) at http2.c:1348
    frame #2: 0x0000000100156147 libcurl.4.dylib`http2_connisdead(conn=0x0000000103809408) at http2.c:219
    frame #3: 0x0000000100155ec1 libcurl.4.dylib`http2_conncheck(check=0x0000000103809408, checks_to_perform=1) at http2.c:239
    frame #4: 0x000000010010e9a5 libcurl.4.dylib`extract_if_dead(conn=0x0000000103809408, data=0x0000000100a47a08) at url.c:969
    frame #5: 0x000000010010e8e8 libcurl.4.dylib`call_extract_if_dead(conn=0x0000000103809408, param=0x00007ffeefbff510) at url.c:998
    frame #6: 0x0000000100150070 libcurl.4.dylib`Curl_conncache_foreach(data=0x0000000100a47a08, connc=0x00000001005226a0, param=0x00007ffeefbff510, func=(libcurl.4.dylib`call_extract_if_dead at url.c:996)) at conncache.c:379
    frame #7: 0x000000010010b43a libcurl.4.dylib`prune_dead_connections(data=0x0000000100a47a08) at url.c:1020
    frame #8: 0x00000001001088cf libcurl.4.dylib`create_conn(data=0x0000000100a47a08, in_connect=0x00007ffeefbff640, async=0x00007ffeefbff7ee) at url.c:3867
    frame #9: 0x0000000100107e78 libcurl.4.dylib`Curl_connect(data=0x0000000100a47a08, asyncp=0x00007ffeefbff7ee, protocol_done=0x00007ffeefbff7ed) at url.c:4152
    frame #10: 0x000000010012b3a0 libcurl.4.dylib`multi_runsingle(multi=0x00000001005225a8, now=(tv_sec = 25332, tv_usec = 529218), data=0x0000000100a47a08) at multi.c:1457
    frame #11: 0x000000010012ae00 libcurl.4.dylib`curl_multi_perform(multi=0x00000001005225a8, running_handles=0x00007ffeefbff940) at multi.c:2211
    frame #12: 0x0000000100001ac1 crawler.exe`main at crawler.c:172

Another crash:

Process 18052 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x68)
    frame #0: 0x00007fff79428e5f libsystem_c.dylib`flockfile + 18
libsystem_c.dylib`flockfile:
->  0x7fff79428e5f <+18>: movq   0x68(%rbx), %rdi
    0x7fff79428e63 <+22>: addq   $0x8, %rdi
    0x7fff79428e67 <+26>: callq  0x7fff7946f6f2            ; symbol stub for: pthread_mutex_lock
    0x7fff79428e6c <+31>: callq  0x7fff7946f218            ; symbol stub for: __error
Target 0: (crawler.exe) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x68)
  * frame #0: 0x00007fff79428e5f libsystem_c.dylib`flockfile + 18
    frame #1: 0x00007fff7942bea7 libsystem_c.dylib`fwrite + 66
    frame #2: 0x00000001000faaa3 libcurl.4.dylib`Curl_debug(data=0x0000000102162208, type=CURLINFO_TEXT, ptr="144 bytes stray data read before trying h2 connection\n", size=54) at sendf.c:830
    frame #3: 0x00000001000fa994 libcurl.4.dylib`Curl_infof(data=0x0000000102162208, fmt="%d bytes stray data read before trying h2 connection\n") at sendf.c:254
    frame #4: 0x000000010015611b libcurl.4.dylib`http2_connisdead(conn=0x0000000102193c08) at http2.c:214
    frame #5: 0x0000000100155ec1 libcurl.4.dylib`http2_conncheck(check=0x0000000102193c08, checks_to_perform=1) at http2.c:239
    frame #6: 0x000000010010e9a5 libcurl.4.dylib`extract_if_dead(conn=0x0000000102193c08, data=0x000000010097f608) at url.c:969
    frame #7: 0x000000010010b861 libcurl.4.dylib`ConnectionExists(data=0x000000010097f608, needle=0x00000001038b8808, usethis=0x00007ffeefbff5f0, force_reuse=0x00007ffeefbff5ed, waitpipe=0x00007ffeefbff5ec) at url.c:1135
    frame #8: 0x00000001001089a3 libcurl.4.dylib`create_conn(data=0x000000010097f608, in_connect=0x00007ffeefbff640, async=0x00007ffeefbff7ee) at url.c:3885
    frame #9: 0x0000000100107e78 libcurl.4.dylib`Curl_connect(data=0x000000010097f608, asyncp=0x00007ffeefbff7ee, protocol_done=0x00007ffeefbff7ed) at url.c:4152
    frame #10: 0x000000010012b3a0 libcurl.4.dylib`multi_runsingle(multi=0x000000010101f828, now=(tv_sec = 25378, tv_usec = 612133), data=0x000000010097f608) at multi.c:1457
    frame #11: 0x000000010012ae00 libcurl.4.dylib`curl_multi_perform(multi=0x000000010101f828, running_handles=0x00007ffeefbff940) at multi.c:2211
    frame #12: 0x0000000100001ac1 crawler.exe`main at crawler.c:172
    frame #13: 0x00007fff793a1ed9 libdyld.dylib`start + 1

Or here:

Process 18059 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
    frame #0: 0x000000010012e69d libcurl.4.dylib`Curl_set_in_callback(data=0x0000000101065408, value=true) at multi.c:3202
   3199	  /* might get called when there is no data pointer! */
   3200	  if(data) {
   3201	    if(data->multi_easy)
-> 3202	      data->multi_easy->in_callback = value;
   3203	    else if(data->multi)
   3204	      data->multi->in_callback = value;
   3205	  }
Target 0: (crawler.exe) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x000000010012e69d libcurl.4.dylib`Curl_set_in_callback(data=0x0000000101065408, value=true) at multi.c:3202
    frame #1: 0x00000001000faa0e libcurl.4.dylib`Curl_debug(data=0x0000000101065408, type=CURLINFO_TEXT, ptr="9 bytes stray data read before trying h2 connection\n", size=52) at sendf.c:821
    frame #2: 0x00000001000fa994 libcurl.4.dylib`Curl_infof(data=0x0000000101065408, fmt="%d bytes stray data read before trying h2 connection\n") at sendf.c:254
    frame #3: 0x000000010015611b libcurl.4.dylib`http2_connisdead(conn=0x000000010101ba08) at http2.c:214
    frame #4: 0x0000000100155ec1 libcurl.4.dylib`http2_conncheck(check=0x000000010101ba08, checks_to_perform=1) at http2.c:239
    frame #5: 0x000000010010e9a5 libcurl.4.dylib`extract_if_dead(conn=0x000000010101ba08, data=0x000000010191f808) at url.c:969
    frame #6: 0x000000010010e8e8 libcurl.4.dylib`call_extract_if_dead(conn=0x000000010101ba08, param=0x00007ffeefbff510) at url.c:998
    frame #7: 0x0000000100150070 libcurl.4.dylib`Curl_conncache_foreach(data=0x000000010191f808, connc=0x000000010061f920, param=0x00007ffeefbff510, func=(libcurl.4.dylib`call_extract_if_dead at url.c:996)) at conncache.c:379
    frame #8: 0x000000010010b43a libcurl.4.dylib`prune_dead_connections(data=0x000000010191f808) at url.c:1020
    frame #9: 0x00000001001088cf libcurl.4.dylib`create_conn(data=0x000000010191f808, in_connect=0x00007ffeefbff640, async=0x00007ffeefbff7ee) at url.c:3867
    frame #10: 0x0000000100107e78 libcurl.4.dylib`Curl_connect(data=0x000000010191f808, asyncp=0x00007ffeefbff7ee, protocol_done=0x00007ffeefbff7ed) at url.c:4152
    frame #11: 0x000000010012b3a0 libcurl.4.dylib`multi_runsingle(multi=0x000000010061f828, now=(tv_sec = 25411, tv_usec = 736041), data=0x000000010191f808) at multi.c:1457
    frame #12: 0x000000010012ae00 libcurl.4.dylib`curl_multi_perform(multi=0x000000010061f828, running_handles=0x00007ffeefbff940) at multi.c:2211
    frame #13: 0x0000000100001ac1 crawler.exe`main at crawler.c:172
    frame #14: 0x00007fff793a1ed9 libdyld.dylib`start + 1

Or this:

Process 18063 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
    frame #0: 0x00000001000faa3e libcurl.4.dylib`Curl_debug(data=0x0000000101025608, type=CURLINFO_TEXT, ptr="17 bytes stray data read before trying h2 connection\n", size=53) at sendf.c:822
   819
   820 	  if(data->set.fdebug) {
   821 	    Curl_set_in_callback(data, true);
-> 822 	    rc = (*data->set.fdebug)(data, type, ptr, size, data->set.debugdata);
   823 	    Curl_set_in_callback(data, false);
   824 	  }
   825 	  else {
Target 0: (crawler.exe) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x00000001000faa3e libcurl.4.dylib`Curl_debug(data=0x0000000101025608, type=CURLINFO_TEXT, ptr="17 bytes stray data read before trying h2 connection\n", size=53) at sendf.c:822
    frame #1: 0x00000001000fa994 libcurl.4.dylib`Curl_infof(data=0x0000000101025608, fmt="%d bytes stray data read before trying h2 connection\n") at sendf.c:254
    frame #2: 0x000000010015611b libcurl.4.dylib`http2_connisdead(conn=0x000000010101b208) at http2.c:214
    frame #3: 0x0000000100155ec1 libcurl.4.dylib`http2_conncheck(check=0x000000010101b208, checks_to_perform=1) at http2.c:239
    frame #4: 0x000000010010e9a5 libcurl.4.dylib`extract_if_dead(conn=0x000000010101b208, data=0x0000000101052408) at url.c:969
    frame #5: 0x000000010010b861 libcurl.4.dylib`ConnectionExists(data=0x0000000101052408, needle=0x00000001038c8c08, usethis=0x00007ffeefbff5f0, force_reuse=0x00007ffeefbff5ed, waitpipe=0x00007ffeefbff5ec) at url.c:1135
    frame #6: 0x00000001001089a3 libcurl.4.dylib`create_conn(data=0x0000000101052408, in_connect=0x00007ffeefbff640, async=0x00007ffeefbff7ee) at url.c:3885
    frame #7: 0x0000000100107e78 libcurl.4.dylib`Curl_connect(data=0x0000000101052408, asyncp=0x00007ffeefbff7ee, protocol_done=0x00007ffeefbff7ed) at url.c:4152
    frame #8: 0x000000010012b3a0 libcurl.4.dylib`multi_runsingle(multi=0x000000010071f828, now=(tv_sec = 25442, tv_usec = 602049), data=0x0000000101052408) at multi.c:1457
    frame #9: 0x000000010012ae00 libcurl.4.dylib`curl_multi_perform(multi=0x000000010071f828, running_handles=0x00007ffeefbff940) at multi.c:2211
    frame #10: 0x0000000100001ac1 crawler.exe`main at crawler.c:172
    frame #11: 0x00007fff793a1ed9 libdyld.dylib`start + 1

@bagder bagder added the crash label Jan 11, 2019

@bagder

This comment has been minimized.

Copy link
Member

commented Jan 11, 2019

Another case of 'conn->data' being wrong. I'm trying out a fix that might work, stand by.

bagder added a commit that referenced this issue Jan 11, 2019

extract_if_dead: use a known working transfer when checking connections
Make sure that this function sets a proper "live" transfer for the
connection before calling the protocol-specific connection check
function, and then clear it again afterward as a non-used connection has
no current transfer.

Fixes #3463

@bagder bagder changed the title crawler example crashes in libcurl 7.64 --with-nghttp2 crawler example crashes in libcurl 7.64-DEV --with-nghttp2 Jan 11, 2019

@bagder bagder closed this in 54b201b Jan 13, 2019

@donny-dont

This comment has been minimized.

Copy link
Contributor

commented Feb 8, 2019

@bagder were hitting a similar callstack in the WinCairo port of WebKit with 7.64.0 and nghttp2 1.36.0

https://bugs.webkit.org/show_bug.cgi?id=194429

> [Inline Frame] libcurl.dll!h2_pri_spec(Curl_easy *) Line 1482	C
> libcurl.dll!h2_session_send(Curl_easy * data, nghttp2_session * h2) Line 1508	C
> libcurl.dll!h2_process_pending_input(connectdata * conn, http_conn * httpc, CURLcode * err) Line 1349	C
> [Inline Frame] libcurl.dll!http2_connisdead(connectdata *) Line 219	C
> libcurl.dll!http2_conncheck(connectdata * check, unsigned int checks_to_perform) Line 239	C
> libcurl.dll!extract_if_dead(connectdata * conn, Curl_easy * data) Line 968	C
> libcurl.dll!ConnectionExists(Curl_easy * data, connectdata * needle, connectdata * * usethis, bool * force_reuse, bool * waitpipe) Line 1135	C
> libcurl.dll!create_conn(Curl_easy * data, connectdata * * in_connect, bool * async) Line 3888	C
> libcurl.dll!Curl_connect(Curl_easy * data, bool * asyncp, bool * protocol_done) Line 4153	C
> libcurl.dll!multi_runsingle(Curl_multi * multi, curltime now, Curl_easy * data) Line 1463	C
> libcurl.dll!curl_multi_perform(Curl_multi * multi, int * running_handles) Line 2223	C
> WebKit.dll!WebCore::CurlMultiHandle::perform(int & runningHandles) Line 259	C++
> WebKit.dll!WebCore::CurlRequestScheduler::workerThread() Line 169	C++
> WebKit.dll!WebCore::CurlRequestScheduler::startThreadIfNeeded::<unnamed-tag>::operator()() Line 89	C++
> WebKit.dll!WTF::Function<void ()>::CallableWrapper<`lambda at ..\..\Source\WebCore\platform\network\curl\CurlRequestScheduler.cpp:87:49'>::call() Line 101	C++
> WTF.dll!WTF::Function<void ()>::operator()() Line 56	C++
> WTF.dll!WTF::Thread::entryPoint(WTF::Thread::NewThreadContext * newThreadContext) Line 137	C++
> WTF.dll!WTF::wtfThreadEntryPoint(void * data) Line 152	C++
> [External Code]
@bagder

This comment has been minimized.

Copy link
Member

commented Feb 8, 2019

Then please submit a new bug. Maybe similar to #3541 ?

@lock lock bot locked as resolved and limited conversation to collaborators May 9, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.