ARROW-6996: [Python] Expose boolean filter kernel on ChunkedArray/RecordBatch/Table #6021
Conversation
python/pyarrow/table.pxi
Outdated
| # FilterKernel(_context(), CDatum(self.sp_chunked_array), | ||
| # CDatum(mask.sp_array), &out)) | ||
|
|
||
| # return wrap_datum(out) |
There was a problem hiding this comment.
Currently, the C++ Filter version that accepts Datums only accepts Array datums. If that would accept Datums of all kinds (and do the dispatch to the specialized Filter version there), the code here could be simpler (it could be something as the above).
Would that be a desired feature of the C++ Filter kernel function?
There was a problem hiding this comment.
I brought this up when the ChunkedArray Filter functions were first implemented, the tracking JIRA is https://issues.apache.org/jira/browse/ARROW-7009
There was a problem hiding this comment.
So it's ok to leave that for a separate PR and go ahead here with the different overloads?
jorisvandenbossche
force-pushed
the
ARROW-6996
branch
2 times, most recently
from
9ee58d3
to
224cf5c
Compare
|
Could this be rehabilitated in time for 0.17.0? Seems useful |
jorisvandenbossche
force-pushed
the
ARROW-6996
branch
from
35bb296
to
424cf20
Compare
|
I rebased this, and cleaned up the code (in the meantime, the Filter kernel added Datum capabilities for all types, so that made the implementation here much cleaner) |
|
Cool, I'm having another look |
There was a problem hiding this comment.
+1. There might be a way to improve code reuse (or docstrings) across the different filter functions but that can be addressed later. Thanks @jorisvandenbossche!
No description provided.