-
-
Notifications
You must be signed in to change notification settings - Fork 98
Upgrade partition transformer to new pipelines #3064
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the naive implementation here is fine. I left a few comments, mostly looking for clarification. if the compaction unit tests pass on this one I think it should be good to go.
Dismissing as there are further really involved changes that need to be done; discussed earlier in a call.
c5fcda6
to
bf0de05
Compare
bf0de05
to
32cfb2f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The tests all pass now, and I tested rebuilding and temporal compaction locally in a minimal setup. Just two places to go now! 🚀
This PR changes the partition transformer to make use of the new pipelines. Furthermore, there are a few necessary transitive changes, as well as some small additions to the operator API.