-
-
Notifications
You must be signed in to change notification settings - Fork 298
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backblaze B2 bucket file listing speed regression #14527
Comments
I'm experiencing this in 8.5.9 as well after updating from 8.5.7. Some of my b2 buckets with large folder hierarchies containing about 15k files effectively just loads forever. |
Another +1, though I wouldn't describe it as just a "speed regression", as it has taken Cyberduck 8.5.9 10 minutes to list the root directory of a B2 bucket - and the root directory only has 9 folders and zero files. 10 minutes for every Backblaze B2 folder list command makes Cyberduck completely unusable. 8.5.7 works fine, but the new behavior occurred in 8.5.8 and continues in 8.5.9. I'll need to downgrade back to Cyberduck 8.5.7 until resolved. |
This is resolved in the current snapshot and beta builds. |
This still fails on 8.6.0 for me. It just times out after waiting 1 minute. I have about 600.000 files in the bucket. But the folder I am listing only has 10 folders (keys) Last part of debug log:
|
@ylangisc Would it be helpful to have access to a B2 bucket with a large number of files to test this? I could spin one up and give you a read-only app key. |
Reopened in #14775. |
Thanks for the help. We should be able to reproduce easily. |
Describe the bug
Since version 8.5.8, listing files in large Backblaze B2 buckets is SIGNIFICANTLY slower. A bucket that would previous take a few seconds to load, now takes over 20x that. It appears to be caused by the changes in the following PR, listing files in every folder. During my testing, some of these folders have thousands of files in them. #14431
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Logs
I enabled debug logging to try and get an idea about how many requests were made. I then simply connected to B2, and expanded the root of the bucket.
8.5.9
Time Taken: ~120s
Number of times
B2FileInfoResponse
is in log file: >20,0000Total debug log size: 81,508 KB
8.5.7
Time Taken: ~3s
Number of times
B2FileInfoResponse
is in log file: 131Total debug log size: 70 KB
If there is any further debug I can do, please let me know.
Versions
The text was updated successfully, but these errors were encountered: