fix: address review findings for leader election recovery#94
Draft
ssteele110 wants to merge 1 commit intossteele/leader-election-recoveryfrom
Draft
fix: address review findings for leader election recovery#94ssteele110 wants to merge 1 commit intossteele/leader-election-recoveryfrom
ssteele110 wants to merge 1 commit intossteele/leader-election-recoveryfrom
Conversation
- Make current_generation public so BuildRecord can access it - Remove duplicate generation_key/learn_generation from Worker (inherit from Base) - Base generation_key now raises instead of silent fallback - Fence push() with Lua script to prevent stale master overwrite - Add lock refresh during population to prevent TTL expiry - Supervisor rescues MasterDied alongside LostMaster - wait_for_master detects immediate nil status (no prior 'setup' needed) - Move generation_stale? check to idle path to reduce Redis GETs - Increase default master_lock_ttl from 30s to 120s - Add 8 tests covering election retry, fenced push, generation staleness, and error handling paths
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses findings from multi-perspective code review of #92.
Critical fixes:
current_generationmade public — fixes BuildRecordrespond_to?always returning false (requeue data was silently lost)push()via Lua script — prevents stale master from overwriting active generation after lock expirywait_for_masterdetects immediate nil status (workers no longer spin 120s when master died before they started)Code quality:
generation_key/learn_generation/generation_stale?from Worker (inherits from Base)generation_keyraises instead of silent fallback to non-generation keysMasterDiedalongsideLostMastergeneration_stale?check moved to idle path (reduces Redis GETs per poll iteration)master_lock_ttlincreased from 30s to 120sTests (8 new):
learn_generationraises when key missingmax_election_attemptsexhausted raisesLostMasterMasterDiedwait_for_masterdetects immediate nil statusTest plan