425 Unable to build data connection: TLS session of data connection not resumed #7234
Labels
Milestone
Comments
ncw
added a commit
that referenced
this issue
Aug 17, 2023
As an extra security feature some FTP servers (eg FileZilla) require that the data connection re-use the same TLS connection as the control connection. This is a good thing for security. The message "TLS session of data connection not resumed" means that it was not done. The problem turned out to be that rclone was re-using the TLS session cache between concurrent connections so the resumed TLS data connection could from any of the control connections. This patch makes each TLS connection have its own session cache which should fix the problem. Fixes #7234
|
Relevant issue upstream jlaffaye/ftp#342 I added a comment there with the info above (thank you very helpful). I think what the error As an extra security feature some FTP servers require that the data connection re-use the same TLS connection as the control connection. This is a good thing for security. However that message means that rclone failed to do that. Given that this works when concurrency == 1 then what must be happening is that rclone or jlaffaye/ftp is mixing up the streams so we are attempting to do file transfers on the wrong TLS connection. I think the problem is that we are re-using the session cache between connections so the TLS library has a chance of picking the wrong connection. Try this which uses a different session cache for each connection and should hopefully fix the problem. v1.64.0-beta.7241.6efb5de13.fix-7234-ftp-tls on branch fix-7234-ftp-tls (uploaded in 15-30 mins) |
|
the new beta doesn't work even when concurrency=1 |
|
OK. That is obvious when looking at the code since rclone implements a dialler and we are using a different tls.Conf for each connection. Try this as a test - this uses a new TLS cache for every connection but gets the library to make the connections. v1.64.0-beta.7242.fe23696ce.fix-7234-ftp-tls on branch fix-7234-ftp-tls (uploaded in 15-30 mins) If that works, then the upstream library is going to need a patch to make it work with external dial functions as the above test wipes out a huge chunk of rclone's features like rate limiting and proxying. |
|
Yes this latest beta works fine, the default concurrency picks 8 connections. |
ncw
added a commit
that referenced
this issue
Aug 17, 2023
As an extra security feature some FTP servers (eg FileZilla) require that the data connection re-use the same TLS connection as the control connection. This is a good thing for security. The message "TLS session of data connection not resumed" means that it was not done. The problem turned out to be that rclone was re-using the TLS session cache between concurrent connections so the resumed TLS data connection could from any of the control connections. This patch makes each TLS connection have its own session cache which should fix the problem. This needed to cache the TLS connection in the context which needed a patch to the upstream library. Fixes #7234
|
OK give this a go - this is a proper fix. This has a patch to the upstream library in it too which if it works for you I'll submit. v1.64.0-beta.7241.6200e0197.fix-7234-ftp-tls on branch fix-7234-ftp-tls (uploaded in 15-30 mins) |
|
Sorry, does not work for me :( |
ncw
added a commit
that referenced
this issue
Aug 18, 2023
As an extra security feature some FTP servers (eg FileZilla) require that the data connection re-use the same TLS connection as the control connection. This is a good thing for security. The message "TLS session of data connection not resumed" means that it was not done. The problem turned out to be that rclone was re-using the TLS session cache between concurrent connections so the resumed TLS data connection could from any of the control connections. This patch makes each TLS connection have its own session cache which should fix the problem. This also reverts the ftp library to the upstream version which now contains all of our patches. Fixes #7234
Yes there is a logic error in there. I had a re-think and came up with this. This doesn't require an upstream fix and is very close to the version which did work, so hopefully this will do the trick! If it doesn't I'll have to work out how to replicate this myself! v1.64.0-beta.7241.2f9db83ec.fix-7234-ftp-tls on branch fix-7234-ftp-tls (uploaded in 15-30 mins) |
|
Good news, it works now! Thanks! |
|
Thanks for your patience testing this @PhilMakower ! I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.64 |
leekiernan
pushed a commit
to leekiernan/rclone
that referenced
this issue
Aug 24, 2023
As an extra security feature some FTP servers (eg FileZilla) require that the data connection re-use the same TLS connection as the control connection. This is a good thing for security. The message "TLS session of data connection not resumed" means that it was not done. The problem turned out to be that rclone was re-using the TLS session cache between concurrent connections so the resumed TLS data connection could from any of the control connections. This patch makes each TLS connection have its own session cache which should fix the problem. This also reverts the ftp library to the upstream version which now contains all of our patches. Fixes rclone#7234
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The associated forum post URL from
https://forum.rclone.orghttps://forum.rclone.org/t/rclone-mount-425-unable-to-build-data-connection-tls-session-of-data-connection-not-resumed/40038
What is the problem you are having with rclone?
doing sync to FileZilla server remote using explicit TLS and concurrency > 1 results in some 425 errors.
Doing sync to PureFTP servers with explicit TLS and default concurrency OK
Doing sync to FileZilla server without explicit TLS and default concrrency OK
What is your rclone version (output from
rclone version)Which OS you are using and how many bits (e.g. Windows 7, 64 bit)
client Ubuntu 18 64 bit
Server Window 2019 64 bit
Which cloud storage system are you using? (e.g. Google Drive)
ftp
The command you were trying to run (e.g.
rclone copy /tmp remote:tmp)rclone sync :ftp: . --ftp-host=io19.blackpig.net --ftp-user="acuitus.onproof.net" --ftp-pass=*** --ftp-disable-epsv=true --ftp-explicit-tls --filter-from ~/ftpsync.blackpig.net/filter-rclone-sites.txt --skip-links -vvA log from the command with the
-vvflag (e.g. output fromrclone -vv copy /tmp remote:tmp)(first timestamp is UTC from log of script that calls rclone)
How to use GitHub
The text was updated successfully, but these errors were encountered: