Skip to content

Commit

Permalink
apacheGH-38772: [C++] Implement directory semantics even when the sto…
Browse files Browse the repository at this point in the history
…rage account doesn't support HNS (apache#39361)

### Rationale for this change

The `FileSystem` implementation based on Azure Blob Storage should implement directory operations according to filesystem semantics. When Hierarchical Namespace (HNS) is enabled, we can rely on Azure Data Lake Storage Gen 2 APIs implementing the filesystem semantics for us, but when all we have is the Blobs API, we should emulate it.

### What changes are included in this PR?

 - Skip fewer tests
 - Re-implement `GetFileInfo` using `ListBlobsByHierarchy` instead of `ListBlobs`
 - Re-implement `CreateDir` with an upfront HNS support check instead of falling back to Blobs API after an error
 - Add comprehensive tests to `CreateDir`
 - Add `HasSubmitBatchBug` to check if a test inside any scenario is affected by a certain Azurite issue
 - Implement `DeleteDir` to work properly on flat namespace storage accounts (non-HNS accounts)
 - 

### Are these changes tested?

Yes. By existing and new tests added by this PR itself.
* Closes: apache#38772

Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
  • Loading branch information
felipecrv authored and dgreiss committed Feb 17, 2024
1 parent 0966e61 commit 94b6455
Show file tree
Hide file tree
Showing 3 changed files with 731 additions and 424 deletions.

0 comments on commit 94b6455

Please sign in to comment.