SFTP download hangs after a certain number of files. #560

Closed
Dreamsorcerer opened this Issue Jul 20, 2015 · 4 comments

Projects

None yet

2 participants

@Dreamsorcerer

When recursively downloading a directory from an SFTP connection, the download will always hang after a certain number of files (always somewhere around 60).

The number of files that are downloaded before hanging is always the same for the same directory. I have checked this by having it skip some files, to test if it was the specific file or the number of files.

With a few different parent directories I tested with, it would hang on file 59, 60, 61 and 66, depending on the directory. It is always the same count for the same directory.

When interrupting the program in this hung state, this is the stack trace:

Traceback (most recent call last):
  File "test_ftp.py", line 41, in <module>
    para()
  File "test_ftp.py", line 33, in para
    recursive_download("plugins/jetpack", "/home/s/Desktop/wordpress-seo")
  File "test_ftp.py", line 31, in recursive_download
    ftp.get(new_ftp_path, new_path)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 720, in get
    size = self.getfo(remotepath, fl, callback)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 688, in getfo
    with self.open(remotepath, 'rb') as fr:
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 327, in open
    t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 728, in _request
    num = self._async_request(type(None), t, *arg)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 750, in _async_request
    self._send_packet(t, msg)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp.py", line 170, in _send_packet
    self._write_all(out)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp.py", line 133, in _write_all
    n = self.sock.send(out)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/channel.py", line 698, in send
    return self._send(s, m)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/channel.py", line 1059, in _send
    size = self._wait_for_send_window(size)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/channel.py", line 1174, in _wait_for_send_window
    self.out_buffer_cv.wait(timeout)
  File "/usr/lib/python3.4/threading.py", line 290, in wait
    waiter.acquire()
KeyboardInterrupt
@bitprophet
Member

Assuming this is based on the comments you left in #60, noting that here for issue link. thanks!

@Dreamsorcerer

Yep. If you want a working example of the bug, I can send you a complete example (privately, due to containing login details).

@Dreamsorcerer

I'm trying to use paramiko for business purposes, and this is stopping us from being able to rely on it. Is there anything I can do to help this along?

@bitprophet
Member

Hoping this falls under #577 which is seeing a fix merged, please reopen if not so.

@bitprophet bitprophet closed this Apr 25, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment