Memory leak using libcurl with sftp

[And-yW]

I did this

There seems to be a fairly substantial memory leak in libcurl when using sftp:

==2071464== LEAK SUMMARY:
==2071464==    definitely lost: 7,589,424 bytes in 99 blocks
==2071464==    indirectly lost: 842,010 bytes in 4,950 blocks
==2071464==      possibly lost: 85,967 bytes in 63 blocks
==2071464==    still reachable: 16,768 bytes in 7 blocks
==2071464==         suppressed: 0 bytes in 0 blocks
==2071464== Reachable blocks (those to which a pointer was found) are not shown.
==2071464== To see them, rerun with: --leak-check=full --show-leak-kinds=all

Bisect showed this commit:

commit 09fed294601cdc8c48f268ee980c7f0df0ff3adc
Author: Stefan Eissing <stefan@eissing.org>
Date:   Wed May 7 13:45:41 2025 +0200

    ssh: move easy handle/connection protocol structs to meta

     
    Closes #17273

I've attached test code that reproduces the issue, when I run this through valgrind against the host version of curl (ubuntu 8.5.0) I get no leaks but when running against curl master it leaks as above but the problem doesn't seem to be present in version 8.13.0 but is in 8.14.0

gcc curl_test.c -o -lcurl  -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -lglib-2.0

valgrind --leak-check=full --error-exitcode=30 ./curl_test

This test app uploads 100 test files using gstreamer via sftp, the destination URL is hard-coded in the main function and username/password is also hard-coded so change to appropriate values.

I expected the following

No response

curl/libcurl version

WARNING: this libcurl is Debug-enabled, do not use in production

curl 8.17.0-DEV (x86_64-pc-linux-gnu) libcurl/8.17.0-DEV GnuTLS/3.8.3 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 libssh2/1.11.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli Debug HSTS HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP TrackMemory UnixSockets zstd

Ubuntu:
curl 8.5.0 (x86_64-pc-linux-gnu) libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
Release-Date: 2023-12-06, security patched: 8.5.0-2ubuntu10.6
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

operating system

curl_test.c

[bagder]

Here's a version of the test code that builds without... is it gstreamer headers?

curl_test.c

[bagder]

The loop needs at least 10 laps for the problem to trigger for me.

[bagder]

==2222060== 
==2222060== 78,080 (73,624 direct, 4,456 indirect) bytes in 1 blocks are definitely lost in loss record 62 of 62
==2222060==    at 0x4A4B818: malloc (vg_replace_malloc.c:446)
==2222060==    by 0x406C507: my_libssh2_malloc (libssh2.c:272)
==2222060==    by 0x4136BF2: libssh2_session_init_ex (session.c:461)
==2222060==    by 0x40746E7: ssh_connect (libssh2.c:3342)
==2222060==    by 0x402DF14: protocol_connect (multi.c:1860)
==2222060==    by 0x402F443: multi_runsingle (multi.c:2493)
==2222060==    by 0x402FF97: curl_multi_perform (multi.c:2795)
==2222060==    by 0x4016FC0: handle_transfer (debugit.c:102)
==2222060==    by 0x4017401: main (debugit.c:194)

[And-yW]

Yes it's using glib.

I'm seeing the same with 10 loops but I get it even with 1 but I'm using a remote host to connect to, don't know if that makes any difference.

[icing]

Testing debug build curl via:

> CURL_DEBUG=multi ./src/curl -vv sftp://myserver/tmp/ --user myuser: -T test.txt
...
* [MULTI] [SHARE] [CPOOL] destroy, 1 connections
* connection #0, shutdown protocol handler (aborted=0)
* Failed to stop libssh2 sftp subsystem
* Failed to free libssh2 session: -37 would block
* [MULTI] shutdown start on connection
* [SETUP] shut down successfully
* [MULTI] [SHUTDOWN] shutdown, done=1
* [MULTI] [SHUTDOWN] closing connection #0
* [SETUP] close
* [SETUP] destroy

So, we try to close the SFTP subsystem, but it says it refuses. Which leaves us no choice really. The question is what is the difference to curl 8.13 here that seems to manage to avoid that.

[icing]

Proposed fix in #19295. @And-yW if you could verify from your side, that'd be great.

[And-yW]

@icing Thanks, I'm no longer seeing the memory leak.

[And-yW]

@icing I'll see if I can get a more long term stability test running on it as well, hopefully over the weekend but otherwise next week.

[icing]

Thanks. The behaviour should effectively been back to curl 8.13.0 regarding SCP/SFTP.