Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(fragmenter): make graph its submodule & some refactors to improve style & readability #2066

Merged
merged 16 commits into from
Apr 24, 2022

Conversation

xxchan
Copy link
Member

@xxchan xxchan commented Apr 23, 2022

What's changed and what's your intention?

  • make graph submodule of fragmenter, because graph is only used there
  • and also some code style refactors
  • and also some refactors to make it more readable and understandable, including
    • remove some unnecessary pub, to make the dependency more clear
    • group some fields & codes into sections
    • remove fragment_graph & stream_graph_builder from fields (make them as variables/parameters), so that the relationship is more clear, and mutable variables are more distinguishable. (Next PR)

Checklist

  • [ ] I have written necessary docs and comments
  • [ ] I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

@xxchan xxchan enabled auto-merge (squash) April 23, 2022 08:05
@codecov
Copy link

codecov bot commented Apr 23, 2022

Codecov Report

Merging #2066 (a5e3af3) into main (df1fd30) will decrease coverage by 0.00%.
The diff coverage is 88.27%.

@@            Coverage Diff             @@
##             main    #2066      +/-   ##
==========================================
- Coverage   70.83%   70.82%   -0.01%     
==========================================
  Files         633      633              
  Lines       81240    81365     +125     
==========================================
+ Hits        57544    57628      +84     
- Misses      23696    23737      +41     
Flag Coverage Δ
rust 70.82% <88.27%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/meta/src/rpc/service/ddl_service.rs 0.00% <0.00%> (ø)
src/meta/src/rpc/service/stream_service.rs 0.00% <0.00%> (ø)
...c/meta/src/stream/fragmenter/rewrite/delta_join.rs 0.00% <ø> (ø)
src/meta/src/stream/test_fragmenter.rs 99.26% <83.33%> (-0.37%) ⬇️
src/meta/src/stream/fragmenter/mod.rs 83.17% <93.97%> (+<0.01%) ⬆️
...meta/src/stream/fragmenter/graph/fragment_graph.rs 97.27% <100.00%> (ø)
...c/meta/src/stream/fragmenter/graph/stream_graph.rs 64.04% <100.00%> (ø)
src/batch/src/executor/executor2_wrapper.rs 41.02% <0.00%> (-35.90%) ⬇️
src/meta/src/model/hash_mapping.rs 75.00% <0.00%> (-8.73%) ⬇️
src/batch/src/executor/test_utils.rs 91.66% <0.00%> (-0.18%) ⬇️
... and 16 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@xxchan xxchan changed the title refactor: make graph submodule of fragmenter refactor(meta): make graph submodule of fragmenter Apr 23, 2022
@xxchan xxchan changed the title refactor(meta): make graph submodule of fragmenter refactor(meta): make graph submodule of fragmenter & some style refactors Apr 23, 2022
@xxchan xxchan requested a review from skyzh April 23, 2022 11:35
@xxchan xxchan changed the title refactor(meta): make graph submodule of fragmenter & some style refactors refactor(fragmenter): make graph its submodule & some refactors to improve style & readability Apr 23, 2022
@xxchan xxchan disabled auto-merge April 23, 2022 21:01
@xxchan xxchan requested review from yezizp2012 and zbzbw April 23, 2022 21:01
Copy link
Contributor

@skyzh skyzh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work!

/// dependent table ids
dependent_table_ids: HashSet<TableId>,
/// upstream dispatch keys
distribution_keys: Vec<i32>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be multiple sets of distribution keys, one set per upstream?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @zbzbw why there's only one set of distribution keys? 🤣

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @zbzbw why there's only one set of distribution keys? 🤣

My fault😢. The comment was misleading, this field contains current fragment's distribution key instead of upstream.

This comment was marked as outdated.

Copy link
Contributor

@zbzbw zbzbw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM!
Please fix the comment I marked, thanks!

src/meta/src/stream/fragmenter/mod.rs Outdated Show resolved Hide resolved
Co-authored-by: Zhou, Bowen <44491322+zbzbw@users.noreply.github.com>
@xxchan xxchan enabled auto-merge (squash) April 24, 2022 04:04
@xxchan xxchan merged commit 2226755 into main Apr 24, 2022
@xxchan xxchan deleted the xxchan/fragmenter branch April 24, 2022 04:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants