-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[C++][Dataset] arrow::dataset::Partitioning::Default() can't be used for writing dataset #15256
Comments
https://issues.apache.org/jira/browse/ARROW-15409 is probably related. |
Thanks! I didn't notice it. It seems that we want to use |
I agree. Either |
…ing::Default() It writes all data into one directory.
…ing::Default() It writes all data into one directory.
…ing::Default() It writes all data into one directory.
…ing::Default() (apache#33674) ### What changes are included in this PR? It writes all data into one directory. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * Closes: apache#15256 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Weston Pace <weston.pace@gmail.com>
…ing::Default() (apache#33674) ### What changes are included in this PR? It writes all data into one directory. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * Closes: apache#15256 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Weston Pace <weston.pace@gmail.com>
Describe the bug, including details regarding any error messages, version, and platform.
Because
arrow::dataset::DefaultPartitioning::Format()
isn't implemented:arrow/cpp/src/arrow/dataset/partition.cc
Lines 92 to 95 in 2acc51a
It's required in
WriteBatch()
:arrow/cpp/src/arrow/dataset/file_base.cc
Lines 380 to 381 in 2acc51a
Is it expected that we can't use
arrow::dataset::Partitioning::Default()
for writing dataset?If it's expected, how about removing
arrow::dataset::Partitioning::Default()
because it's useless?If it's not expected, how about implementing
arrow::dataset::DefaultPartitioning::Format()
like the following?Component(s)
C++
The text was updated successfully, but these errors were encountered: