New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SFTP download hangs after a certain number of files. #560

Closed
Dreamsorcerer opened this Issue Jul 20, 2015 · 4 comments

Comments

Projects
None yet
2 participants
@Dreamsorcerer

Dreamsorcerer commented Jul 20, 2015

When recursively downloading a directory from an SFTP connection, the download will always hang after a certain number of files (always somewhere around 60).

The number of files that are downloaded before hanging is always the same for the same directory. I have checked this by having it skip some files, to test if it was the specific file or the number of files.

With a few different parent directories I tested with, it would hang on file 59, 60, 61 and 66, depending on the directory. It is always the same count for the same directory.

When interrupting the program in this hung state, this is the stack trace:

Traceback (most recent call last):
  File "test_ftp.py", line 41, in <module>
    para()
  File "test_ftp.py", line 33, in para
    recursive_download("plugins/jetpack", "/home/s/Desktop/wordpress-seo")
  File "test_ftp.py", line 31, in recursive_download
    ftp.get(new_ftp_path, new_path)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 720, in get
    size = self.getfo(remotepath, fl, callback)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 688, in getfo
    with self.open(remotepath, 'rb') as fr:
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 327, in open
    t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 728, in _request
    num = self._async_request(type(None), t, *arg)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp_client.py", line 750, in _async_request
    self._send_packet(t, msg)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp.py", line 170, in _send_packet
    self._write_all(out)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/sftp.py", line 133, in _write_all
    n = self.sock.send(out)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/channel.py", line 698, in send
    return self._send(s, m)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/channel.py", line 1059, in _send
    size = self._wait_for_send_window(size)
  File "/usr/local/lib/python3.4/dist-packages/paramiko/channel.py", line 1174, in _wait_for_send_window
    self.out_buffer_cv.wait(timeout)
  File "/usr/lib/python3.4/threading.py", line 290, in wait
    waiter.acquire()
KeyboardInterrupt
@bitprophet

This comment has been minimized.

Member

bitprophet commented Jul 20, 2015

Assuming this is based on the comments you left in #60, noting that here for issue link. thanks!

@Dreamsorcerer

This comment has been minimized.

Dreamsorcerer commented Jul 21, 2015

Yep. If you want a working example of the bug, I can send you a complete example (privately, due to containing login details).

@Dreamsorcerer

This comment has been minimized.

Dreamsorcerer commented Jul 24, 2015

I'm trying to use paramiko for business purposes, and this is stopping us from being able to rely on it. Is there anything I can do to help this along?

@bitprophet

This comment has been minimized.

Member

bitprophet commented Apr 25, 2016

Hoping this falls under #577 which is seeing a fix merged, please reopen if not so.

@bitprophet bitprophet closed this Apr 25, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment