-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[C++] GCS: report common prefixes as directories #32403
Comments
Will Jones / @wjones127: |
Carlos O'Ryan / @coryan:
|
I wanted to leave this breadcrumb somewhere, but not sure where. I noticed a discrepancy between "directories" created via Arrow vs directories created via the GCS cloud console. One uses a traliing slash while the other does not. In my C++ code, I have to defensively call |
I've encountered this again, and I think the tradeoff of making |
I got confused at the behavior differences between S3 and GCS, only to realize GCS only reports special directory markers as "directories" and not the common prefixes. This can have the effect of making a directory look empty in GCS, when it in fact has many folders (see example below).
We currently use the ListObjects method, but perhaps it would be more appropriate to use the ListObjectsWithPrefix. Since they are returned in the same API call, it shouldn't add much overhead.
Reporter: Will Jones / @wjones127
Note: This issue was originally created as ARROW-17097. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: