Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(streaming): spin loop when too many row-id requests in single ms #2213

Merged
merged 9 commits into from
Apr 29, 2022

Conversation

yezizp2012
Copy link
Contributor

@yezizp2012 yezizp2012 commented Apr 28, 2022

What's changed and what's your intention?

As title, after #2205.

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • Summarize your change (mandatory)
  • How does this PR work? Need a brief introduction for the changed logic (optional)
  • Describe clearly one logical change and avoid lazy messages (optional)
  • Describe any limitations of the current code (optional)

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

@yezizp2012 yezizp2012 enabled auto-merge (squash) April 28, 2022 12:06
src/source/src/row_id.rs Outdated Show resolved Hide resolved
src/source/src/row_id.rs Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Apr 28, 2022

Codecov Report

Merging #2213 (d52b92d) into main (55cfc66) will increase coverage by 0.00%.
The diff coverage is 94.73%.

@@           Coverage Diff           @@
##             main    #2213   +/-   ##
=======================================
  Coverage   70.94%   70.94%           
=======================================
  Files         657      657           
  Lines       83065    83072    +7     
=======================================
+ Hits        58928    58938   +10     
+ Misses      24137    24134    -3     
Flag Coverage Δ
rust 70.94% <94.73%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/source/src/row_id.rs 89.65% <94.73%> (+1.41%) ⬆️
src/common/src/types/ordered_float.rs 24.30% <0.00%> (+0.19%) ⬆️
.../src/executor/managed_state/aggregation/extreme.rs 90.29% <0.00%> (+0.26%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@yezizp2012 yezizp2012 enabled auto-merge (squash) April 28, 2022 12:47
@yezizp2012 yezizp2012 changed the title feat(streaming): avoid assert, spin loop when too many row-id requests in single ms feat(streaming): spin loop when too many row-id requests in single ms Apr 28, 2022
@yezizp2012 yezizp2012 enabled auto-merge (squash) April 28, 2022 13:58
Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@yezizp2012 yezizp2012 merged commit 7bfe95e into main Apr 29, 2022
@yezizp2012 yezizp2012 deleted the feat/row-id-spin-loop branch April 29, 2022 07:19
@skyzh
Copy link
Contributor

skyzh commented Apr 29, 2022

In fact, I didn't think this really solves the problem. The clock can go forward or backward after the spin loop. So the next time we acquire an epoch, it might still be less than last generated one.

@skyzh
Copy link
Contributor

skyzh commented Apr 29, 2022

Okay, maybe this PR doesn't take clock-go-backward into account. Then let's fix this altogether after monolithic clock has been implemented.

@yezizp2012
Copy link
Contributor Author

Okay, maybe this PR doesn't take clock-go-backward into account. Then let's fix this altogether after monolithic clock has been implemented.

This is just for row-id generate. Yes, clock-go-backward also need to be considered when compute node reboot. This will be fixed together with epoch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants