Table union top level construction #589
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dcoutts
force-pushed
the
dcoutts/table-union
branch
from
1aa5566 to
698a6ef
Compare
Collaborator
Author
dcoutts
force-pushed
the
dcoutts/table-union
branch
from
698a6ef to
3eddb7a
Compare
dcoutts
requested review from
jorisdral,
mheinzel,
recursion-ninja and
wenkokke
as code owners
Collaborator
Author
dcoutts
force-pushed
the
dcoutts/table-union
branch
from
1ee31bb to
fa13687
Compare
Collaborator
Author
|
Rebased on main, reordered and squashed some patches. Ready for review. |
dcoutts
force-pushed
the
dcoutts/table-union
branch
from
fa13687 to
bbef5d0
Compare
mheinzel
reviewed
Mar 4, 2025
Collaborator
mheinzel
left a comment
mheinzel
left a comment
There was a problem hiding this comment.
Just two things to look at, but looks good otherwise.
jorisdral
reviewed
Mar 4, 2025
We have to update the ParentTable when we have union/unions operations. These are sub-optimal since we only track a single parent, and union(s) create new tables with multiple parents.
The config of the resulting table is actually independent from the config of the input tables, so we do not need them to match. So we take the config from the first input table.
We were getting some thunks in the lists which was a bit awkward to sort out. Easier to make them vectors. Using lists vs vectors in the first place didn't have any strong argument either way.
We'll reuse it in the next patch.
dcoutts
force-pushed
the
dcoutts/table-union
branch
from
bbef5d0 to
ff67e82
Compare
Collaborator
Author
|
@jorisdral @mheinzel thanks for the PR review. I've pushed a patch with all the changes. If you're happy I'll squash it in. |
jorisdral
approved these changes
Mar 5, 2025
mheinzel
approved these changes
Mar 5, 2025
The API modules all implement union in terms of unions themselves.
dcoutts
force-pushed
the
dcoutts/table-union
branch
from
ff67e82 to
c98fea8
Compare
We rely on doing interruptable operations that could receive async exceptions _before_ doing operations that allocate fresh resources.
dcoutts
force-pushed
the
dcoutts/table-union
branch
from
c98fea8 to
19bf835
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement the top level part of the
unionandunionsAPI.Construct the new result table, as an empty table except for a union level merging tree, which is the pending union of the pending merges of all existing runs and merges in each input table.
We don't enable the state machine tests yet because lookups are not done yet.