Socket use after close when connecting to remote HTTPS via HTTPS proxy

[intxgo]

I did this

1. Connect to non-existent HTTPS server via HTTPS proxy tunnel.
2. Observe the process crash

We use ProcessStrictHandleCheckPolicy which reveals socket use after close.

I expected the following

Curl returns "Received HTTP code 502 from proxy after CONNECT" and gracefully close the HTTPS tunnel.

curl/libcurl version

7.79.1

[curl -V output]
curl 7.81.0-DEV (x86_64-pc-win32) libcurl/7.81.0-DEV OpenSSL/1.1.1i zlib/1.2.11 WinIDN
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS HSTS HTTPS-proxy IDN IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI UnixSockets alt-svc libz

operating system

Windows 10 20H2

[bagder]

You didn't show the curl -V output though, so we're clueless about your TLS config.

[intxgo]

Thanks, I see, but I'm only linking with curl lib in my project.

Running curl.exe built from this repo, top of master branch, the problem is still there. I added curl -V output to the description above.

Patch which enables strict handle check policy on Windows:
https://github.com/intxgo/curl/blob/8c04acbaecd4071b86c962998545c77fcf49b066/src/tool_main.c#L262-L275

With the above policy, unpatched code crash when executed like this (with no server on port 9999):

.\curl --proxy-cacert ca.pem https://localhost:9999 -x https://localhost:8899

with my PR all is fine:

.\curl --proxy-cacert ca.pem https://localhost:9999 -x https://localhost:8899
curl: (56) Received HTTP code 502 from proxy after CONNECT

[bagder]

I want to A) reproduce the problem and then B) verify that the fix is fine.

[bagder]

It does not reproduce for me on Linux, with or without valgrind:

./src/curl -x https://localhost:8843 https://localhost:7999/ --proxy-insecure

Where is the socket use after close happening? Can you get a stack trace from your crash?

[intxgo]

The stack trace:

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 00007ffbc7670d2a (ntdll!KiRaiseUserExceptionDispatcher+0x000000000000003a)
   ExceptionCode: c0000008 (Invalid handle)
  ExceptionFlags: 00000000
NumberParameters: 0
Thread tried to close a handle that was invalid or illegal to close

FAULTING_THREAD:  00002f00

PROCESS_NAME:  curl.exe

ERROR_CODE: (NTSTATUS) 0xc0000008 - An invalid HANDLE was specified.

EXCEPTION_CODE_STR:  c0000008

STACK_TEXT:  
0000008d`8b8fdbc0 00007ffb`c4f048b0     : 00000000`67f2bc60 00000247`9566f880 00000247`95677d90 00000000`00000003 : ntdll!KiRaiseUserExceptionDispatcher+0x3a
0000008d`8b8fdc90 00007ffb`c5854b31     : 00000247`9566f720 00000247`956776a0 0000008d`8b8fde48 00000000`00000000 : KERNELBASE!GetHandleInformation+0x40
0000008d`8b8fdcd0 00007ffb`c584627b     : 00000247`95677fbb 00000000`00000000 00000000`67e0db00 00000247`95665fb0 : WS2_32!DSOCKET::FindIFSSocket+0x21
0000008d`8b8fdd00 00000000`67e4070d     : 00000000`00000018 0000008d`8b8fde58 00000247`95677fb3 0000008d`8b8fde53 : WS2_32!send+0x13f5b
0000008d`8b8fdda0 00000000`67e3b12f     : 0000008d`8b8fde70 00000000`6d48e55a cccccccc`cccccccc cccccccc`cccccccc : libcrypto_1_1_x64!BIO_s_null+0x1dd
0000008d`8b8fdde0 00000000`67e39585     : 0000008d`00000003 00000247`00000001 0000008d`8b8fdf68 00000000`00000015 : libcrypto_1_1_x64!BIO_number_written+0x8f
0000008d`8b8fde10 00000000`6d487aed     : 0000008d`00000000 0000008d`8b8fe580 cccccccc`cccccccc 0000008d`8b8fdfb0 : libcrypto_1_1_x64!BIO_write+0xb5
0000008d`8b8fde90 00000000`6d488c05     : cccccccc`cccccccc cccccccc`cccccccc cccccccc`cccccccc cccccccc`cccccccc : libssl_1_1_x64!SSL_rstate_string+0x4ad
0000008d`8b8fdef0 00000000`6d4941c6     : cccccccc`cccccccc cccccccc`00000015 00000247`95663e10 0000008d`8b8fef10 : libssl_1_1_x64!SSL_rstate_string+0x15c5
0000008d`8b8feed0 00000000`6d493116     : cccccccc`cccccccc cccccccc`cccccccc cccccccc`cccccccc 00000000`0000000b : libssl_1_1_x64!SSL_rstate_string+0xcb86
0000008d`8b8fef40 00007ff6`a528f4d7     : cccccccc`cccccccc cccccccc`cccccccc 00000001`00000001 cccccccc`cccccccc : libssl_1_1_x64!SSL_rstate_string+0xbad6
0000008d`8b8fefb0 00007ff6`a528f59c     : 00000247`9547b560 00000247`954792a0 00000247`95479510 cccccccc`cccccccc : curl!ossl_closeone+0xa7
0000008d`8b8ff040 00007ff6`a525d476     : 00000247`9547b560 00000247`954792a0 cccccccc`00000000 0000edfe`e00c382a : curl!ossl_close+0x3c
0000008d`8b8ff070 00007ff6`a5261a6c     : 00000247`9547b560 00000247`954792a0 00000000`00000000 00000247`9547b598 : curl!Curl_ssl_close+0x36
0000008d`8b8ff0a0 00007ff6`a5260846     : 00000247`9547b560 00000247`954792a0 00000247`954975c0 00000247`954769e8 : curl!conn_shutdown+0xbc
0000008d`8b8ff0d0 00007ff6`a52513c7     : 00000247`9547b560 00000247`954792a0 0000008d`8b8ff200 0000008d`8b8ff1e4 : curl!Curl_disconnect+0xf6
0000008d`8b8ff100 00007ff6`a524b13d     : 00000247`9547a170 0000008d`8b8ff478 00000247`9547b560 ffffffff`ffffffff : curl!multi_runsingle+0x1c47
0000008d`8b8ff420 00007ff6`a523a389     : 00000247`9547a170 0000008d`8b8ff544 cccccccc`00000000 cccccccc`000003e8 : curl!curl_multi_perform+0xdd
0000008d`8b8ff500 00007ff6`a523a5a4     : 00000247`9547a170 00000247`9547b560 00000000`00000005 0000008d`8b8ff4f9 : curl!easy_transfer+0x99
0000008d`8b8ff5a0 00007ff6`a5239906     : 00000247`9547b560 cccccccc`cccccc00 cccccccc`cccccccc 0000edfe`e00c255a : curl!easy_perform+0x174
0000008d`8b8ff5f0 00007ff6`a522db29     : 00000247`9547b560 00000247`95479c50 0000008d`8b8ff664 00007ffb`8f79e1c7 : curl!curl_easy_perform+0x16
0000008d`8b8ff620 00007ff6`a522dfed     : 0000008d`8b8ff810 00000247`954769c0 00000000`00000005 00007ff6`a522f6e3 : curl!serial_transfers+0x109
0000008d`8b8ff6d0 00007ff6`a5223df6     : 0000008d`8b8ff810 00000247`954769c0 00000000`00000000 00007ff6`a5272b60 : curl!run_all_transfers+0x9d
0000008d`8b8ff760 00007ff6`a52236d1     : 0000008d`8b8ff810 00000000`00000005 00000247`9545b3a0 00000003`00000003 : curl!operate+0x2d6
0000008d`8b8ff7d0 00007ff6`a52d2a29     : 0000edfe`00000005 00000247`9545b3a0 00000000`00000000 00007ff6`a52d25bd : curl!main+0x1f1
0000008d`8b8ff8d0 00007ff6`a52d297e     : 00007ff6`a52d8e00 00007ff6`a52d8e30 00000000`00000000 00000000`00000000 : curl!invoke_main+0x39
0000008d`8b8ff920 00007ff6`a52d283e     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : curl!__scrt_common_main_seh+0x12e
0000008d`8b8ff990 00007ff6`a52d2a9e     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : curl!__scrt_common_main+0xe
0000008d`8b8ff9c0 00007ffb`c6e97034     : 0000008d`8b750000 00000000`00000000 00000000`00000000 00000000`00000000 : curl!mainCRTStartup+0xe
0000008d`8b8ff9f0 00007ffb`c7622651     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!BaseThreadInitThunk+0x14
0000008d`8b8ffa20 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21


STACK_COMMAND:  ~0s ; .cxr ; kb

SYMBOL_NAME:  kernelbase!GetHandleInformation+40

MODULE_NAME: KERNELBASE

IMAGE_NAME:  KERNELBASE.dll

FAILURE_BUCKET_ID:  INVALID_HANDLE_c0000008_KERNELBASE.dll!GetHandleInformation

OS_VERSION:  10.0.19041.1

BUILDLAB_STR:  vb_release

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

IMAGE_VERSION:  10.0.19041.1151

FAILURE_ID_HASH:  {9770812b-4494-dd8c-5dcb-c58d3d8140ae}

Followup:     MachineOwner
---------

0:000> k
 # Child-SP          RetAddr               Call Site
00 0000008d`8b8fdbc0 00007ffb`c4f048b0     ntdll!KiRaiseUserExceptionDispatcher+0x3a
01 0000008d`8b8fdc90 00007ffb`c5854b31     KERNELBASE!GetHandleInformation+0x40
02 0000008d`8b8fdcd0 00007ffb`c584627b     WS2_32!DSOCKET::FindIFSSocket+0x21
03 0000008d`8b8fdd00 00000000`67e4070d     WS2_32!send+0x13f5b
04 0000008d`8b8fdda0 00000000`67e3b12f     libcrypto_1_1_x64!BIO_s_null+0x1dd
05 0000008d`8b8fdde0 00000000`67e39585     libcrypto_1_1_x64!BIO_number_written+0x8f
06 0000008d`8b8fde10 00000000`6d487aed     libcrypto_1_1_x64!BIO_write+0xb5
07 0000008d`8b8fde90 00000000`6d488c05     libssl_1_1_x64!SSL_rstate_string+0x4ad
08 0000008d`8b8fdef0 00000000`6d4941c6     libssl_1_1_x64!SSL_rstate_string+0x15c5
09 0000008d`8b8feed0 00000000`6d493116     libssl_1_1_x64!SSL_rstate_string+0xcb86
0a 0000008d`8b8fef40 00007ff6`a528f4d7     libssl_1_1_x64!SSL_rstate_string+0xbad6
0b 0000008d`8b8fefb0 00007ff6`a528f59c     curl!ossl_closeone+0xa7 [C:\Users\lesio\code\curl\lib\vtls\openssl.c @ 1444] 
0c 0000008d`8b8ff040 00007ff6`a525d476     curl!ossl_close+0x3c [C:\Users\lesio\code\curl\lib\vtls\openssl.c @ 1463] 
0d 0000008d`8b8ff070 00007ff6`a5261a6c     curl!Curl_ssl_close+0x36 [C:\Users\lesio\code\curl\lib\vtls\vtls.c @ 685] 
0e 0000008d`8b8ff0a0 00007ff6`a5260846     curl!conn_shutdown+0xbc [C:\Users\lesio\code\curl\lib\url.c @ 754] 
0f 0000008d`8b8ff0d0 00007ff6`a52513c7     curl!Curl_disconnect+0xf6 [C:\Users\lesio\code\curl\lib\url.c @ 874] 
10 0000008d`8b8ff100 00007ff6`a524b13d     curl!multi_runsingle+0x1c47 [C:\Users\lesio\code\curl\lib\multi.c @ 2571] 
11 0000008d`8b8ff420 00007ff6`a523a389     curl!curl_multi_perform+0xdd [C:\Users\lesio\code\curl\lib\multi.c @ 2638] 
12 0000008d`8b8ff500 00007ff6`a523a5a4     curl!easy_transfer+0x99 [C:\Users\lesio\code\curl\lib\easy.c @ 606] 
13 0000008d`8b8ff5a0 00007ff6`a5239906     curl!easy_perform+0x174 [C:\Users\lesio\code\curl\lib\easy.c @ 696] 
14 0000008d`8b8ff5f0 00007ff6`a522db29     curl!curl_easy_perform+0x16 [C:\Users\lesio\code\curl\lib\easy.c @ 716] 
15 0000008d`8b8ff620 00007ff6`a522dfed     curl!serial_transfers+0x109 [C:\Users\lesio\code\curl\src\tool_operate.c @ 2380] 
16 0000008d`8b8ff6d0 00007ff6`a5223df6     curl!run_all_transfers+0x9d [C:\Users\lesio\code\curl\src\tool_operate.c @ 2551] 
17 0000008d`8b8ff760 00007ff6`a52236d1     curl!operate+0x2d6 [C:\Users\lesio\code\curl\src\tool_operate.c @ 2665] 
18 0000008d`8b8ff7d0 00007ff6`a52d2a29     curl!main+0x1f1 [C:\Users\lesio\code\curl\src\tool_main.c @ 291] 
19 0000008d`8b8ff8d0 00007ff6`a52d297e     curl!invoke_main+0x39 [d:\a01\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 79] 
1a 0000008d`8b8ff920 00007ff6`a52d283e     curl!__scrt_common_main_seh+0x12e [d:\a01\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 
1b 0000008d`8b8ff990 00007ff6`a52d2a9e     curl!__scrt_common_main+0xe [d:\a01\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 331] 
1c 0000008d`8b8ff9c0 00007ffb`c6e97034     curl!mainCRTStartup+0xe [d:\a01\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp @ 17] 
1d 0000008d`8b8ff9f0 00007ffb`c7622651     KERNEL32!BaseThreadInitThunk+0x14
1e 0000008d`8b8ffa20 00000000`00000000     ntdll!RtlUserThreadStart+0x21

First I set a debug log on CloseHandle and I found out that we are closing the same handle in the place which I patched. When the code unwinds from there it triggers the stream_error on

curl/lib/multi.c

Line 2006 in e7f9c90

stream_error = TRUE;

then it falls into

curl/lib/multi.c

Line 2569 in e7f9c90

Curl_disconnect(data, conn, dead_connection);

then here

curl/lib/url.c

Line 753 in e7f9c90

Curl_ssl_close(data, conn, FIRSTSOCKET);

which is an interesting code. Curl_ssl_close will dereference SSL context at curl_easy_t->conn->ssl[0]->backend which has the same socket handle in it's read/write structures as curl_easy_t->conn->sock[0] which was just closed.

A blind try to fix it with

if(CURL_SOCKET_BAD != conn->sock[FIRSTSOCKET])
    Curl_ssl_close(data, conn, FIRSTSOCKET);

also worked but walking the code, reading comments, I made an educated guess to fix it as submitted in my PR

[bagder]

Awesome, that was just the info I needed to get a full picture.