-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARROW-16113: [Python] Partitioning.dictionaries in case of a subset of fields are dictionary encoded #12791
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, this is a better way to handle this, thanks for this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Just a few tiny code comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Benchmark runs are scheduled for baseline = 7616fba and contender = 85809a9. 85809a9 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
…f fields are dictionary encoded This PR modifies the `dictionaries` method to have entries for all the fields. With this change, the method will return a list with the values if present, otherwise it shall contain `None`, thus returning a list of same length. Ref: apache#12530 (comment) Closes apache#12791 from sanjibansg/partitioning_dictionaries Authored-by: Sanjiban Sengupta <sanjiban.sg@gmail.com> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
This PR modifies the
dictionaries
method to have entries for all the fields. With this change, the method will return a list with the values if present, otherwise it shall containNone
, thus returning a list of same length.Ref: #12530 (comment)