[Python] Extend RecordBatch.filter to take an Expression in addition to a boolean mask Array
#39220
Comments
|
Looks like this is supported in Rust as well: https://arrow.apache.org/rust/arrow_select/filter/fn.filter_record_batch.html |
jorisvandenbossche
added a commit
to jorisvandenbossche/arrow
that referenced
this issue
Jun 25, 2024
…ession in addition to mask array
wjones127
pushed a commit
that referenced
this issue
Jun 26, 2024
… in addition to mask array (#43043) ### Rationale for this change `Table.filter()` already accepted either a boolean mask array or a boolean expression. But the equivalent method on `RecordBatch` only accepted the array. This makes both methods consistent in accepting both types of mask. ### What changes are included in this PR? Consolidate the `Table.filter` and `RecordBatch.fitler` methods into a single shared method on the base class, and expanded the `_filter_table` Acero helper to also work with RecordBatch in addition to Table (and ensure to return a batch if the input was a batch) ### Are these changes tested? Yes * GitHub Issue: #39220 Authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
|
Issue resolved by pull request 43043 |
zanmato1984
pushed a commit
to zanmato1984/arrow
that referenced
this issue
Jul 9, 2024
…ession in addition to mask array (apache#43043) ### Rationale for this change `Table.filter()` already accepted either a boolean mask array or a boolean expression. But the equivalent method on `RecordBatch` only accepted the array. This makes both methods consistent in accepting both types of mask. ### What changes are included in this PR? Consolidate the `Table.filter` and `RecordBatch.fitler` methods into a single shared method on the base class, and expanded the `_filter_table` Acero helper to also work with RecordBatch in addition to Table (and ensure to return a batch if the input was a batch) ### Are these changes tested? Yes * GitHub Issue: apache#39220 Authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the enhancement requested
Currently RecordBatches can only be filtered using a boolean mask
Array, unlike Tables which can be filtered using either a mask or anExpression. It would be useful to allowRecordBatch.filterto also accept anExpressionto make it consistent withTable.filter.See also discussion here
Component(s)
Python
The text was updated successfully, but these errors were encountered: