Does not show 'folders' that do not contain a .bzEmpty object #14768
Comments
dkocher
changed the title
|
@metadaddy Thanks for your thorough analysis of the issue. Previously discussed in #14759. |
|
@metadaddy To give you some context on the changeset in #14431 and what it is trying to solve. We are using this client implementation in Mountain Duck working with B2 as a filesystem. Thus it must be possible to delete folders aka directory placeholder files and these should no longer appear in directory listings. Otherwise it is impossible for users to delete folders when making us of b2_hide_file which was introduced with #13265. |
|
@dkocher If I "create a folder" in an empty bucket using the web UI or by uploading the zero length placeholder, then {
"files": [
{
"accountId": "...",
"action": "upload",
"bucketId": "...",
"contentLength": 0,
"fileId": "4_z0145cfc9e3f5ec0f74ed0c1b_f404ce2fe0a91f670_d20230613_m015603_c004_v0402018_t0038_u01686621363510",
"fileName": "testfolder/.bzEmpty",
...
},
],
"nextFileName": null
}If I now "upload a file into the folder", {
"files": [
{
"accountId": "...",
"action": "upload",
"bucketId": "...",
"contentLength": 0,
"fileId": "4_z0145cfc9e3f5ec0f74ed0c1b_f404ce2fe0a91f670_d20230613_m015603_c004_v0402018_t0038_u01686621363510",
"fileName": "testfolder/.bzEmpty",
...
},
{
"accountId": "...",
"action": "upload",
"bucketId": "...",
"fileId": "4_z0145cfc9e3f5ec0f74ed0c1b_f110de288a449bf16_d20230613_m020029_c004_v0402017_t0058_u01686621629033",
"fileName": "testfolder/data.csv",
...
},
],
"nextFileName": null
}If I now hide the files, {
"files": [],
"nextFileName": null
}So, you can get exactly the file tree that you want from the output of You could also use {
"files": [
{
"accountId": "...",
"action": "hide",
"bucketId": "...",
"contentLength": 0,
"fileId": "4_z0145cfc9e3f5ec0f74ed0c1b_f4104e52bfe48aa99_d20230613_m020319_c004_v0402006_t0025_u01686621799026",
"fileName": "testfolder/.bzEmpty",
...
},
{
"accountId": "...",
"action": "upload",
"bucketId": "...",
"contentLength": 0,
"fileId": "4_z0145cfc9e3f5ec0f74ed0c1b_f404ce2fe0a91f670_d20230613_m015603_c004_v0402018_t0038_u01686621363510",
"fileName": "testfolder/.bzEmpty",
...
},
{
"accountId": "...",
"action": "hide",
"bucketId": "...",
"contentLength": 0,
"fileId": "4_z0145cfc9e3f5ec0f74ed0c1b_f41848b64ec7f090e_d20230613_m020311_c004_v0402017_t0000_u01686621791337",
"fileName": "testfolder/data.csv",
...
},
{
"accountId": "...",
"action": "upload",
"bucketId": "...",
"contentLength": 766,
"fileId": "4_z0145cfc9e3f5ec0f74ed0c1b_f110de288a449bf16_d20230613_m020029_c004_v0402017_t0058_u01686621629033",
"fileName": "testfolder/data.csv",
...
},
],
"nextFileId": null,
"nextFileName": null
}Note that the results are ordered "by reverse of date/time uploaded for versions of files with the same name", so it's easy to build the desired tree, with similar logic. Bottom line - the B2 APIs give you everything you need to implement your desired functionality, without hiding "folders" that don't contain a |
|
@metadaddy We make use of the b2_list_file_versions API where prefixes are always returned regardless if all containing files are hidden. When working on a single level in the hierarchy I don't see a way to achieve the described result (hiding prefixes only containing hidden files) with a single API call. |
|
@dkocher I don't think there is a way to consider a level at a time (in a bounded amount of time). For each "folder", you have to examine all of its "descendants" to determine whether that folder should be hidden, so, to list the entries in a bucket's "root folder", you have to examine all objects in the bucket.
Buckets with 100k+ files are quite common - the bucket with my Synology NAS backup contains just 4 TB of data, but nearly 200k files. |
|
I don't consider working with the API with no |
Revert looking up directory placeholder file
Describe the bug
The
.bzEmptyobject is optional in Backblaze B2. Many applications create objects with path-like keys without ever creating a.bzEmptyobject.Cyberduck version 8.6.0 fails to display a 'folder' if it does not contain a
.bzEmptyobject.To Reproduce
Steps to reproduce the behavior:
Expected behavior
The 'folder' should be displayed.
Desktop:
Log Files
Additional context
This bug seems to have been introduced with 8287d23 or c8bf429. Tracing through the code, the issue is in this section of code in
B2ObjectListService.parse:As evidenced by the comment, this code assumes that
.bzEmptywill be present. Theattr.find()call ends up callingB2VersionIdProvider.getVersionId(), which, correctly, throwsNotfoundException, since/my-bucket/folderdoes not have afileId. The code interprets this as meaning that it has found.bzEmpty, and sets the duplicate flag on/my-bucket/folder.Note - I'm not sure that it's possible to successfully implement #14431 . Consider a bucket with millions of objects with keys of the form
folder00000001/foo,folder00000002/foo,folder00000003/foo, etc. This is not a particularly contrived example - analytical data can have this kind of structure. Every 'folder name' will need to be inspected, via a B2 API call, to determine whether it should be shown.The text was updated successfully, but these errors were encountered: