Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix stuck leader when cosigner goes down #232

Merged
merged 4 commits into from
Dec 11, 2023
Merged

Conversation

agouin
Copy link
Member

@agouin agouin commented Dec 9, 2023

If a cosigner goes down, all nonces may need to be cleared. In the case that horcrux is serving multiple chains, this means that GetNonces may be called repetitively when the nonce cache is empty. When this happens, we do not want it to block, it should return immediately.

Additionally, this scenario was queuing up many items in the empty channel which would cause rapid back to back reconciles to occur, which could lead to massive numbers of nonces being requested due to the incrementing lastReconcileNonces and such short duration since lastReconcileTime. We only need to enqueue at most 1 item in the empty channel. Due to race conditions on the channel len check, The buffer is oversized.

@agouin agouin marked this pull request as draft December 9, 2023 01:19
@agouin agouin marked this pull request as ready for review December 9, 2023 01:50
@agouin agouin requested a review from misko9 December 9, 2023 01:51
@agouin agouin merged commit 26c340c into main Dec 11, 2023
13 checks passed
@agouin agouin deleted the andrew/fix_stuck_leader branch December 11, 2023 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants