Skip to content

STORM-2984: Add jitter in executor heartbeat#2582

Closed
zodvik wants to merge 2 commits into
apache:masterfrom
zodvik:master
Closed

STORM-2984: Add jitter in executor heartbeat#2582
zodvik wants to merge 2 commits into
apache:masterfrom
zodvik:master

Conversation

@zodvik
Copy link
Copy Markdown

@zodvik zodvik commented Mar 1, 2018

The executorHeartbeatTimer is scheduled without jitter. In cases where state storage is Zookeeper and not pacemaker, introducing jitter will help spread out the I/O over time on Zookeeper. This will be helpful for large clusters.

The jitter time is worker.heartbeat.frequency.secs

@HeartSaVioR
Copy link
Copy Markdown
Contributor

@zodvik
Thanks for the contribution! Btw, we're planning to change the heartbeat mechanism to RPC which should reduce overall ZK load, so I'd rather rely on new mechanism. Please take a look at #2433.

I think the patch is still effective on 1.x version line. Could you raise a pull request on 1.x-branch?

@zodvik
Copy link
Copy Markdown
Author

zodvik commented Mar 2, 2018

@HeartSaVioR
Makes sense. I'll close this PR.

@zodvik zodvik closed this Mar 2, 2018
@HeartSaVioR
Copy link
Copy Markdown
Contributor

@zodvik We may still want to apply this to 1.x version line. So if you struggled and the patch helped your case, please submit the patch against 1.x-branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants