[Question] Concurrent get from sftp connection

[hn-sl]

import asyncssh
import asyncio

conn = await asyncssh.connect(host=host, username=username, password=password)
sftp = await conn.start_sftp_client() # single sftp channel 
task1 = sftp.get(file1, localpath1)
task1 = sftp.get(file2, localpath2)
await asyncio.gather(task1, task2) # concurrent download through single sftp channel? 

I confirmed that file1 and file2 are downloaded in concurrency. If there are multiple sftp channels this makes sense, but If there is only one sftp channel as in this code, how is this possible ?

[ronf]

Unlike SCP, an SFTP file transfer is not a single request on the channel which needs to complete before any other requests can be sent. Instead, SFTP breaks the transfer into a collection of requests for different ranges of file data. To speed things up, AsyncSSH will issue multiple requests for file data in parallel even when you are transferring only a single file.

If you issue multiple SFTP file transfer requests on a channel without waiting for each one to finish, requests for data from all of these files will proceed in parallel on that channel. You'll need to be careful that you don't exceed the limit on the number of SFTP file handles that the SFTP server will let you open at once on a channel, but otherwise it should be fine to take advantage of this parallelism.

You can control the number of outstanding requests at once for a given transfer with the "max_requests" argument and the block size of the data in each request with the "block_size" argument. By default, it will issue up to 128 simultaneous requests for 16 KB each (2 MB total outstanding at once per transfer).

[hn-sl]

I see, thanks for the kind explanation.

[Fisch37]

This sounds closed

[ronf]

Yes - thanks. Feel free to open a new issue if you have other questions...

[natangirma]

Does this mean sftp can be faster for uploads/downloads than scp using asyncssh?

[ronf]

Yes, absolutely.