Skip to content

[INLONG-7589][Sort] Support multi node relation with same output but different input nodes#7590

Merged
dockerzhang merged 4 commits intoapache:masterfrom
EMsnap:7589
Mar 15, 2023
Merged

[INLONG-7589][Sort] Support multi node relation with same output but different input nodes#7590
dockerzhang merged 4 commits intoapache:masterfrom
EMsnap:7589

Conversation

@EMsnap
Copy link
Member

@EMsnap EMsnap commented Mar 14, 2023

Prepare a Pull Request

Motivation

Now it's not capable for sort to generate multi insert sqls for one output with multiple inputs which is not acceptable for user cases.

For instance, user may want to get data from multiple data sources, they can only use union operation for now. But union operation in flink is too heavy.

We need to support generate sqls from one stream such that:

Insert into sink select * from sourceA;
Insert into sink select * from sourceB;

Modifications

Change the parseNode to ParseSingleNode because the recuisive method will be multi branches which is not suitable for reading. And move the input nodes parsing outside of parseNode method to make it easy for reading

Verifying this change

Run AllmigrateTestMultiRelations

@healchow healchow changed the title [INLONG-7589][Sort][Feature] Sort support multi node relation with same output but different input nodes [INLONG-7589][Sort] Support multi node relation with same output but different input nodes Mar 14, 2023
@dockerzhang dockerzhang merged commit d8f1354 into apache:master Mar 15, 2023
@EMsnap EMsnap deleted the 7589 branch May 30, 2023 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][Sort] Support multi node relation with same output but different input nodes

5 participants