New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
libssh: SFTP transfer: slow progress meter update #11020
Comments
It was done that way by the team that wrote the support. I don't know why but I also know it causes problems (see #8632) as everything internally in libcurl is meant to always be non-blocking. |
@nmav do you recall maybe? |
Thanks for the quick reply. The weird thing for me is that the sftp code is clearly prepared to deal with the non-blocking mode.
|
What I remember is that the libssh2 backend was supporting async but libssh not at the time. As I didn't know how the libssh async/non-blocking mode will work (when developed) the curl code was modeled similar to libssh2. I do not know if libssh today supports non-blocking mode for sftp transfers. |
I found the post on the libssh mailing list, that is refering SCP, which I have not looked at yet.
But maybe someone else should test it as well. :) |
The function is documented in a single brief line with no explanation whatsoever. It is also SFTP-specific so while it addresses this exact issue the same problem remains for SCP. Sure I'm biased, but man @libssh this is lame. |
Reported-by: Andreas Huebner Fixes #11020
I've built curl 7.87.0 with libssh 0.10.4 for sftp support and I'm trying to download a 30 MiB file with a relatively slow bandwith of ~ 10 Mbit/s.
The update rate of the progress meter is then something like once every 4 seconds.
(To someone who doesn't know what's going on, the download appears to be stalling at first. No progress at all until ~ 6.5 MiB are transfered.)
It works fine when curl is built with libssh2. (Progress meter updates at least once every second.)
When looking into this I came to the conclusion, that the sftp transfer with libssh is done in blocking mode.
That means the do-while loop in readwrite_data() (lib/transfer.c) runs until maxloops are exceeded. So depending on buffer size, there would be no updating on the progress meter for a long time.
Is there any specific reason why libssh is used in blocking mode for this?
Curl seems to prefer nonblocking mode in general.
I tried to fix this by adding
sftp_file_set_nonblocking(sshc->sftp_file)
aftersftp_open()
and as far as I can tell this works without a problem. (No thorough testing done yet, though.)The text was updated successfully, but these errors were encountered: