New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create projection plugin #2000
Create projection plugin #2000
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have you also looked at the Arrow-native functions for projection?
I would be surprised if projection is not easily doable out of the box. There is a lot there: https://github.com/apache/arrow/search?l=C%2B%2B&q=project
9551897
to
242125e
Compare
242125e
to
ac4acfd
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some initial thoughts on the code. I've verified that the functionality works as expected for both Arrow- and MsgPack-encoded table slices.
91a970e
to
4229c9d
Compare
519c720
to
440b53b
Compare
440b53b
to
292a17c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both delete and project work nicely for me locally. I just have some minor notes.
292a17c
to
a3b4862
Compare
It calls arrow transform also when there is no generic transform, and the slice is not arrow encoded. In this case, the slice is converted to arrow first.
a3b4862
to
a0b5812
Compare
📔 Description
Projection: removes one or more columns
📝 Checklist
🎯 Review Instructions
Review this pull request commit-by-commit.