Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(meta): actor placement rule and dispatcher refactor #1843

Merged
merged 1 commit into from
Apr 14, 2022

Conversation

skyzh
Copy link
Contributor

@skyzh skyzh commented Apr 14, 2022

Signed-off-by: Alex Chi iskyzh@gmail.com

What's changed and what's your intention?

  • Remove ExchangeNode before fragment -> actor stage. As now fragments might have multiple dispatchers, information of exchange node is not useful anymore. Dispatch strategy is now recorded in FragmentEdge.
  • Prepare for actor placement rule implementation. We need to ensure some actors stay on the same worker node. Now we add this field, set this field according to FragmentEdge. Will later add logic to enforce such rule.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

ref #719

Signed-off-by: Alex Chi <iskyzh@gmail.com>
Copy link
Contributor

@yezizp2012 yezizp2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@skyzh skyzh enabled auto-merge (squash) April 14, 2022 10:04
@codecov
Copy link

codecov bot commented Apr 14, 2022

Codecov Report

Merging #1843 (4ed695e) into main (c1c7440) will decrease coverage by 0.02%.
The diff coverage is 72.02%.

@@            Coverage Diff             @@
##             main    #1843      +/-   ##
==========================================
- Coverage   70.86%   70.84%   -0.03%     
==========================================
  Files         609      610       +1     
  Lines       79632    79689      +57     
==========================================
+ Hits        56431    56452      +21     
- Misses      23201    23237      +36     
Flag Coverage Δ
rust 70.84% <72.02%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/meta/src/model/stream.rs 53.57% <ø> (ø)
src/meta/src/stream/graph/fragment_graph.rs 96.94% <ø> (ø)
src/meta/src/stream/rewrite/delta_join.rs 0.00% <0.00%> (ø)
src/meta/src/stream/stream_manager.rs 72.50% <ø> (ø)
src/meta/src/stream/graph/stream_graph.rs 66.66% <72.91%> (-0.52%) ⬇️
src/meta/src/stream/fragmenter.rs 82.95% <73.33%> (-6.10%) ⬇️
src/meta/src/stream/scheduler.rs 97.20% <100.00%> (+0.03%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@skyzh skyzh merged commit 76c22f6 into main Apr 14, 2022
@skyzh skyzh deleted the skyzh/placement-rule-prepare branch April 14, 2022 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants