New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(WIP-BEAM-7049)Changes made to make a simple case of threeway union work #9358
Conversation
"CreateRightIndex", | ||
MapElements.via(new BeamSetOperatorsTransforms.BeamSqlRow2KvFn()))) | ||
.apply(CoGroupByKey.create()); | ||
.and( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note that to make it general, it would be a for loop to create one tag per input.
@@ -141,6 +142,7 @@ | |||
|
|||
private static final List<RelOptRule> BEAM_CONVERTERS = | |||
ImmutableList.of( | |||
UnionMergeRule.INSTANCE, //Added for three way union |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah. The only difference of us might just be the place to add this rule:
I added to LOGICAL_OPTIMIZATIONS above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it still does not work, you could
- re-clone a new repo of Beam
- add UnionMergeRule
- change
BeamCostModel.FACTORY
tonull
at https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/CalciteQueryPlanner.java#L116
Then quickly test
SELECT 1
UNION ALL
SELECT 2
UNION ALL
SELECT 3
UNION ALL
SELECT 4
UNION ALL
SELECT 5
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions. |
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
Code changes to implement 3 way union work. This is still in very early stages and may break existing 2 way union. More changes to come.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username
).[BEAM-XXX] Fixes bug in ApproximateQuantiles
, where you replaceBEAM-XXX
with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.Post-Commit Tests Status (on master branch)
Pre-Commit Tests Status (on master branch)
See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.