Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(meta): generate delta join plan #1871

Merged
merged 5 commits into from
Apr 15, 2022
Merged

feat(meta): generate delta join plan #1871

merged 5 commits into from
Apr 15, 2022

Conversation

skyzh
Copy link
Contributor

@skyzh skyzh commented Apr 15, 2022

Signed-off-by: Alex Chi iskyzh@gmail.com

What's changed and what's your intention?

  • Add a new executor union. Will implement later.
  • Generate delta-join plan.
  • If an exchange node has been processed, its inputs will be removed from fragment.

Not yet tested, will have follow-up PRs.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

close #1807
ref #719

Signed-off-by: Alex Chi <iskyzh@gmail.com>
Signed-off-by: Alex Chi <iskyzh@gmail.com>
@codecov
Copy link

codecov bot commented Apr 15, 2022

Codecov Report

Merging #1871 (1b0b6bc) into main (0f6f2ac) will decrease coverage by 0.18%.
The diff coverage is 15.62%.

@@            Coverage Diff             @@
##             main    #1871      +/-   ##
==========================================
- Coverage   70.91%   70.72%   -0.19%     
==========================================
  Files         611      611              
  Lines       79668    79898     +230     
==========================================
+ Hits        56494    56506      +12     
- Misses      23174    23392     +218     
Flag Coverage Δ
rust 70.72% <15.62%> (-0.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/frontend/src/catalog/schema_catalog.rs 81.94% <ø> (ø)
src/meta/src/stream/graph/stream_graph.rs 61.94% <0.00%> (-0.42%) ⬇️
src/meta/src/stream/rewrite/delta_join.rs 0.00% <0.00%> (ø)
src/meta/src/stream/fragmenter.rs 85.90% <72.00%> (+2.78%) ⬆️
src/meta/src/stream/graph/fragment_graph.rs 97.10% <100.00%> (+0.15%) ⬆️
src/connector/src/filesystem/file_common.rs 80.44% <0.00%> (-0.45%) ⬇️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@skyzh skyzh requested a review from BugenZhao April 15, 2022 08:12
Copy link
Contributor

@yezizp2012 yezizp2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

@skyzh skyzh enabled auto-merge (squash) April 15, 2022 08:50
@skyzh skyzh merged commit dcd17ec into main Apr 15, 2022
@skyzh skyzh deleted the skyzh/delta-join-plan branch April 15, 2022 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

meta: rewrite HashJoin into delta join fragments
2 participants