Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple file uploads keep re-opening connection #40

Closed
jc1001 opened this issue Jan 30, 2017 · 7 comments

Comments

Projects
None yet
2 participants
@jc1001
Copy link

commented Jan 30, 2017

Hi there,

I am trying to upload multiple files synchronously using the same FTP connection (using FluentFTP 16.1.0). I would expect to see only one connection being used on the FTP server for the whole operation, but I can see that the connection is closed and reopened after every file upload. I've replicated the problem with the following test program (using FileZilla as my FTP server):

class Program
{
	static void Main(string[] args)
	{
		FtpTrace.AddListener(new ConsoleTraceListener());

		using (FtpClient client = new FtpClient())
		{
			client.Host = "127.0.0.1";
			client.Port = 21;
			client.Credentials = new NetworkCredential("anonymous", string.Empty);
			client.EnableThreadSafeDataConnections = false;
			client.Connect();

			foreach (string file in Directory.GetFiles(@"C:\Temp"))
			{
				client.UploadFile(file, "./" + Path.GetFileName(file), true);
			}

			client.Disconnect();
		}
	}
}

In the console, I can see the following trace output (for every file):

...
STOR ./File1.txt
150 Opening data channel for file upload to server of "/File1.txt"
TYPE I
200 Type set to I
There is stale data on the socket, maybe our connection timed out. Re-connecting.
The data was: 226 Successfully transferred "/File1.txt"
...

I can see that it is closing the connection because of the StaleDataCheck failing. I can set this flag to false, but I'm not sure what other consequences this will have on our code (we use a generic client for upload, download, etc.). If you need any other information from my side, please let me know.

@robinrodricks

This comment has been minimized.

Copy link
Owner

commented Jan 30, 2017

This is because in FTP a new connection is created for transferring data (the data channel). This behavior matches SmartFTP, for example, whose logs for uploading a single file I have pasted here:

...
[16:49:21] Operation Begin
[16:49:21] PASV
[16:49:21] 227 Entering Passive Mode
[16:49:21] Opening data connection to X Port: Y
[16:49:21] STOR big2.txt
[16:49:22] 150 Accepted data connection
[16:49:29] 3932160 bytes transferred. (493 KB/s) (00:00:07)
[16:49:30] 226-File successfully transferred
[16:49:30] 226 8.517 seconds (measured here), 450.84 Kbytes per second
[16:49:30] Setting properties (Date modified) of "/public_html/temp/big2.txt"
[16:49:30] MFMT 20170129094800 big2.txt
[16:49:30] 213 UTIME OK
[16:49:30] MLST big2.txt
[16:49:31] 250-Begin
[16:49:31]  type=file;size=3932160;modify=20170129094800;UNIX.mode=0644;UNIX.uid=13616;UNIX.gid=13615;unique=fd07g4ca28fd; big2.txt
[16:49:31] 250 End.
[16:49:31] Source File Size=3932160, Destination File Size=3932160
[16:49:31] Operation End
...
@robinrodricks

This comment has been minimized.

Copy link
Owner

commented Jan 30, 2017

Thanks for reproducing the bug with a test example. I appreciate your efforts, however this issue seems to be by design and not a bug. If you think there is a bug, let me know and I'll re-open this issue.

@jc1001

This comment has been minimized.

Copy link
Author

commented Jan 30, 2017

Thank you for the quick response. Unfortunately I'm not that familiar with the FTP specification, so it is hard for me to say whether my scenario is even allowed. You can leave the issue closed.

@robinrodricks robinrodricks reopened this Jan 30, 2017

@robinrodricks

This comment has been minimized.

Copy link
Owner

commented Jan 30, 2017

Your slow performance seems to be a legacy feature "EnableThreadSafeDataConnections". I'm considering removing this feature to speedup uploads/downloads and to reduce the FTP threads created. I'm reopening this issue as I will now work on it.

@robinrodricks

This comment has been minimized.

Copy link
Owner

commented Jan 30, 2017

I've fixed the library so UploadFile() and Download() commands work perfectly on the same FTP thread.

Can you download the latest version from nuget? Don't need to change EnableThreadSafeDataConnections since it is now OFF by default, improving performance. My tests succeeded without any "stale data" errors that you were getting before. I was able to transfer small & large files with greatly improved performance.

@jc1001

This comment has been minimized.

Copy link
Author

commented Jan 31, 2017

I've downloaded the latest library (from NuGet) and I can see that my upload test code from above now only creates and use one FTP connection to the server. Thank you very much for the quick response and fix.

@robinrodricks

This comment has been minimized.

Copy link
Owner

commented Jan 31, 2017

Thank you for the great test case. Its what motivated me into looking deeper thru the library and solving the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.