Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High memory usage when downloading #11049

Closed
cyberduck opened this issue May 8, 2020 · 12 comments
Closed

High memory usage when downloading #11049

cyberduck opened this issue May 8, 2020 · 12 comments
Assignees
Labels
bug duplicate high priority sftp SFTP Protocol Implementation
Milestone

Comments

@cyberduck
Copy link
Collaborator

cyberduck commented May 8, 2020

exmate created the issue

Cyberduck has been using an insane amount of memory since 7.3.1 on MacOS when downloading files using the SFTP protocol. I have tried FTP and the memory usage is fine. The bug still exists in the latest snapshot build (7.3.2 (32852)).


Attachments

@cyberduck
Copy link
Collaborator Author

cyberduck commented May 11, 2020

ce6d08c commented

I've just reported something similar with #11051 (https://trac.cyberduck.io/ticket/11051).

Must be a new issue as I've never encountered this before.

@cyberduck
Copy link
Collaborator Author

cyberduck commented May 26, 2020

@ylangisc commented

I have tried to replicate the issue with different file sizes and protocols but was not successful unfortunately. As long as I'm not able to replicate it the only way to find the cause would be to create a heap dump of the Cyberduck process. Not sure how technical you are but the steps would as follows:

jmap -dump:file=cyberduck-dump.hprof [PID of Cyberduck process]
  • provide a link to the dump file

The dump file could be quite large. Let me know if you want me to share a link where you can drop the file.

@cyberduck
Copy link
Collaborator Author

cyberduck commented May 26, 2020

ce6d08c commented

I'm quite technical so I'm sure we can find that issue together. Before I install this on my productive system I'd like to share a video I've made showing the memory leak. I'll try to exactly explain again what my setup is and what I've done to reproduce it.

  • Updated to the newest Cyberduck version 7.4.0
  • Logged onto OneDrive and selected Sicherungen\Eigene Dateien\Musik - this is an online only vault and NOT synched to my hard drive
  • Inside the "Musik" folder there are 3 sub folders and within those sub folders there are tons of other folders with singles and albums. Basically this is my (legal!) backup of all my music sorted in sub directories
  • I drag and dropped the "Musik" main folder onto a finder window. I did NOT select single files - I want to have the whole "Musik" folder downloaded
  • The preparation of the files and directories starts which took about 30 minutes. During that time no excessive memory usage was there
  • After the preparation the download has started and as soon as this happens the memory usage goes up

I monitored it with the terminal and the "top" command. Maybe you did not try with a directory or drag/drop and that's why you weren't able to reproduce it.

I can offer you a TeamViewer session together with a telephone call. This might ease up things significantly if you're still not able to reproduce this.

Best regards
rootie

@cyberduck
Copy link
Collaborator Author

cyberduck commented May 26, 2020

ce6d08c commented

One addition: As soon as i click on "Stop" on the transfer window to interrupt the download, the memory usage is back to normal. I think the garbage collector is then executed.

@cyberduck
Copy link
Collaborator Author

cyberduck commented May 31, 2020

ce6d08c commented

Any news on this? Please give an update on whether or not you were able to reproduce that issue.

@cyberduck
Copy link
Collaborator Author

cyberduck commented Jun 5, 2020

@dkocher commented

#11051 closed as duplicate.

@cyberduck
Copy link
Collaborator Author

cyberduck commented Nov 2, 2020

@dkocher commented

Milestone renamed

@cyberduck
Copy link
Collaborator Author

cyberduck commented Nov 2, 2020

@dkocher commented

Ticket retargeted after milestone closed

@cyberduck
Copy link
Collaborator Author

cyberduck commented Nov 2, 2020

@dkocher commented

Milestone renamed

@cyberduck
Copy link
Collaborator Author

cyberduck commented Jan 13, 2021

9e9f682 commented

Cyberduck 7.8 is still using excessive RAM for SFTP transfers. RAM usage grows the longer Cyberduck is open and transferring files. One of my organization's users has seen up to 65 GB RAM in use by Cyberduck after transferring a large amount of files over sftp.

@cyberduck
Copy link
Collaborator Author

cyberduck commented Mar 24, 2021

@dkocher commented

Following up in #11151.

@cyberduck
Copy link
Collaborator Author

cyberduck commented Apr 1, 2021

@dkocher commented

Milestone renamed

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug duplicate high priority sftp SFTP Protocol Implementation
Projects
None yet
Development

No branches or pull requests

2 participants