-
Notifications
You must be signed in to change notification settings - Fork 590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: filter for PySpark backend #1943
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1943 +/- ##
==========================================
- Coverage 87.77% 86.13% -1.64%
==========================================
Files 93 93
Lines 17068 17088 +20
Branches 2157 2159 +2
==========================================
- Hits 14981 14719 -262
- Misses 1679 1964 +285
+ Partials 408 405 -3
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a request to file a TODO issue
|
Most recent push is a rebase on top of master. |
|
This LGTM, @hjoo please just rebase on top of master, and i'll merge on green |
|
Rebase complete |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @hjoo!
Introduces predicate handling in `ops.Selection` for `filter()` functionality. Also implemented a few missing comparison operations: - `And` - `Or` - `NotEquals` - `Less` - `LessEquals` Author: Hyonjee <hyonjee.joo@twosigma.com> Closes ibis-project#1943 from hjoo/pyspark-filter and squashes the following commits: 42c2a3c [Hyonjee] parametrize pyspark test_filter 0480921 [Hyonjee] added ibis issue for missing sort_keys handling for pyspark selection op 28e6ad6 [Hyonjee] pyspark backend filter
Introduces predicate handling in
ops.Selectionforfilter()functionality.Also implemented a few missing comparison operations:
AndOrNotEqualsLessLessEquals