FTPS data connection resumption with FileZilla

[lsabouri74]

I did this

FTPS fIle upload to FileZilla with libcurl fails since updating to 8.21.0. It was working with 8.20.0. I am using SChannel and I had issues before where I had to cap TLS version to 1.2, since SChannel doesn't seem to handle session resumption well in TLS 1.3.

Here is my code snipet:

m_CurlHandle = curl_easy_init(), &curl_easy_cleanup )
...
CURL_SETOPT(CURLOPT_READDATA, src_file );
CURL_SETOPT(CURLOPT_UPLOAD, 1L );
CURL_SETOPT(CURLOPT_INFILESIZE_LARGE, (curl_off_t)file_size );
const std::string url("ftp://ftp_server/~/321AD0C9-EDE2-40E1-8DF7-7778C7495D90_0-IF_1_test.pdf" );
CURL_SETOPT(CURLOPT_URL, url.c_str() );
CURL_SETOPT(CURLOPT_SSL_VERIFYPEER, 0L);
CURL_SETOPT(CURLOPT_SSL_VERIFYHOST, 0L);
CURL_SETOPT(CURLOPT_USE_SSL, CURLUSESSL_ALL );
CURL_SETOPT(CURLOPT_FTPSSLAUTH, CURLFTPAUTH_DEFAULT );
CURL_SETOPT(CURLOPT_USERNAME, UserName.c_str());
CURL_SETOPT(CURLOPT_PASSWORD, UserPassword.c_str());
CURL_SETOPT(CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_2 | CURL_SSLVERSION_MAX_TLSv1_2);
CURL_SETOPT(CURLOPT_FTP_CREATE_MISSING_DIRS, 1L );
CURL_SETOPT(CURLOPT_LOW_SPEED_TIME, low_speed_timeout);
CURL_SETOPT(CURLOPT_LOW_SPEED_LIMIT, low_speed_limit);
CURL_SETOPT( CURLOPT_CONNECTTIMEOUT, config.Get(L"FileTransmissionConnectionTimeOut", 10));   // 10 seconds to connect to remote server
CURLcode curl_res = curl_easy_perform( m_CurlHandle.get() );

here is the curl debug output:

handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Host ftp_server:21 was resolved.
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->IPv6: (none)
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->IPv4: 10.9.79.97
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->  Trying 10.9.79.97:21...
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Established connection to ftp_server (10.9.79.97 port 21) from 10.9.79.28 port 53119 
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->220-FileZilla Server 1.12.6
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->220 Please visit https://filezilla-project.org/
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->AUTH SSL
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->234 Using authentication type TLS.
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->schannel: disabled automatic use of client certificate
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Established connection to ftp_server (10.9.79.97 port 21) from 10.9.79.28 port 53119 
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->USER *****
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->331 Please, specify the password.
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->PASS *****
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->230 Login successful.
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->PBSZ 0
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->200 PBSZ=0
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->PROT P
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->200 Protection level set to P
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->PWD
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->257 "/" is current directory.
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Entry path is '/'
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->CWD ~
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->250 CWD command successful
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->EPSV
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Connect data stream passively
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->229 Entering Extended Passive Mode (|||49274|)
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Connecting to 10.9.79.97 port 49274
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->  Trying 10.9.79.97:49274...
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->schannel: disabled automatic use of client certificate
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->TYPE I
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->200 Type set to I
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_OUT->STOR EB0C5352-5ACF-4614-A692-74E217559676_0-IF_1_test.pdf
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->150 Starting data transfer.
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Established 2nd connection to ftp_server (10.9.79.97 port 49274) from 10.9.79.28 port 53120 
handle(0x00000198B6CF7FB0) CURLINFO_DATA_OUT->
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->schannel: shutting down SSL/TLS connection with ftp_server port 21
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Shutdown send direction error: 81. Broken server? Proceeding as if everything is ok.
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->upload completely sent off: 102 bytes
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Remembering we are in directory "~/"
handle(0x00000198B6CF7FB0) CURLINFO_HEADER_IN->425 Unable to build data connection: TLS session of data connection not resumed.
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->server did not report OK, got 425
handle(0x00000198B6CF7FB0) CURLINFO_TEXT->Connection #0 to host ftp_server:21 left intact

Please let me know if it is a bug, or if there is something I am doing wrong.

I expected the following

I would expect the upload to work as it was working with 8.20.0.

curl/libcurl version

curl-8.21.0 with SChannel

operating system

Windows 11

[lsabouri74]

I did the following change in lib/cf-setup.c:284:

        struct Curl_peer *peer = (cf->sockindex == SECONDARYSOCKET) ?
          origin : Curl_conn_get_destination(cf->conn, cf->sockindex);

With that change, it is now working. Not sure if that modification is ok, since I have not a great understanding of the whole curl code. It feels more like a bandaid to me.

[Zenju]

Same issue here: https://freefilesync.org/forum/viewtopic.php?t=13126

CURLE_PARTIAL_FILE: server did not report OK, got 425
425 Cannot secure data connection - TLS session resumption required. [curl_easy_perform]

Reproducible with FTP test server: test.rebex.net

username: demo
password: password

I did the following change in lib/cf-setup.c:284:

        struct Curl_peer *peer = (cf->sockindex == SECONDARYSOCKET) ?
          origin : Curl_conn_get_destination(cf->conn, cf->sockindex);

With that change, it is now working. Not sure if that modification is ok, since I have not a great understanding of the whole curl code.

Can confirm! This "fixes" the issue, apparently.

[icing]

Thanks for the report. @Zenju I try to reproduce the problem with test.rebex.net, but a top dir listing just works fine for me. What platform, curl -V and command line are you running?

[Zenju]

curl 8.20.0:

[zenju@centos-vm Source]$ '/home/zenju/Desktop/Projects/libcurl-old/Build/CentOS_gcc_16.1.0_25_x86_64_release/bin/curl' -k --ssl-reqd -u demo:password ftp://test.rebex.net/
drwx------ 2 demo users 0 Mar 31 2023 pub
-rw------- 1 demo users 379 Sep 19 2023 readme.txt

curl 8.21.0:

[zenju@centos-vm Source]$ '/home/zenju/Desktop/Projects/libcurl/Build/CentOS_gcc_16.1.0_25_x86_64_release/bin/curl' -k --ssl-reqd -u demo:password ftp://test.rebex.net/
curl: (18) server did not report OK, got 425

[Zenju]

And using latest OpenSSL 4.0.1, perhaps this matters.

[icing]

@Zenju I can see the problem when adding -k. Without it, it works here. The insure TLS setting causes the lookups of SSL sessions to be more picky. This then misses the CONTROL TLS connection for the DATA TLS one.

Could you verify that without -k it works for you as well? Just to see if we are on the same page.

Now thinking on how to best fix this. The provided workaround is not wrong, maybe there is a better one...

[Zenju]

I can confirm, without "-k" the rebex example works with latest libcurl 8.21!

[icing]

Ok, I have a fix in #22246 that somewhat includes the proposed patch. Adding test configuration with a vsftpd server revealed however that parallel transfers required more effort.

@Zenju would be glad to hear if this works for you.
@lsabouri74 this should also make serial transfers (one after another) work with schannel.

[lsabouri74]

Hello @icing, I am confirming that your changes fix the issue for us! Thanks! What is the ETA to have it in a official release? I am asking because we need to decide if we run with a patched version of curl-8.21.0 or wait for the next release.