Lint fixes for EC2 fix#1
Closed
rallytime wants to merge 1 commit intodwoz:awscloudfrom
Closed
Conversation
dwoz
pushed a commit
that referenced
this pull request
Sep 29, 2019
dwoz
pushed a commit
that referenced
this pull request
Jan 3, 2020
dwoz
pushed a commit
that referenced
this pull request
Mar 25, 2026
Prevent exceptions with "fileserver.update" when called via state
dwoz
added a commit
that referenced
this pull request
May 9, 2026
…L bug Adds the structural plumbing for ring-based cluster sharding without changing today's broadcast semantics, and fixes a latent ``MmapCache.get`` corruption bug along the way. Ring rollout — stage 0 (wire-in, no behaviour change) * salt/cluster/ring_membership.py: per-process HashRing singleton with owns/rebuild/reset/get_ring. EventMonitor's ring stays empty in its subprocess so owns() always returns True and the existing broadcast behaviour is preserved. * salt/cluster/consensus/service.py: RaftService._on_membership_change now goes through _apply_ring_policy, which honours the ring SM's current members policy. Default "self" rebuilds with the local node id only. * salt/master.py: the three EventMonitor peer-receive sites (job /new, job /ret, key state) gate via ring_membership.owns(). With self-only ring the gate is a tautology; future flip to voters policy activates real sharding without code change. Ring rollout — stage 1 (RingConfigStateMachine + runner skeleton) * salt/cluster/consensus/raft/log.py: RingConfigStateMachine, new LogEntryType.RING_CONFIG = 3. Ring policy commits through Raft and survives compaction via the multi-SM snapshot envelope. * salt/cluster/consensus/raft/node.py: apply_entries dispatches RING_CONFIG entries to the SM registered under "ring_sm". Adds Node.reconcile_membership() for post-snapshot peer-table reconciliation (Gap #1 from GAPS.md). * salt/cluster/consensus/service.py: propose_ring_config(members, replicas) leader-side API; _on_ring_config_change rebuilds the ring with the new policy. cluster_max_log_size opt plumbed through to Node. * salt/runners/cluster.py: ring_info read-only runner; ring_set raises NotImplementedError pending runner-to-RaftService IPC. MmapCache NUL-byte corruption fix * salt/utils/mmap_cache.py: drop the unconditional rstrip(b"\x00") in MmapCache.get and the rstrip-for-CRC inside _overwrite_in_heap. The slot's LENGTH field is the single source of truth for the value byte boundary; trailing bytes from a previously-larger value remain in the heap as unreferenced garbage and are reclaimed by the next atomic_rebuild. Pre-fix, any binary value (msgpack, protobuf, BSON) whose serialised form ended in NUL would either come back one byte short or fail CRC on read. mmap heap segment cap is operator-tunable * salt/cache/mmap_cache.py / salt/cache/mmap_key.py: read mmap_cache_max_segment_bytes / mmap_key_max_segment_bytes from __opts__ and forward to MmapCache(max_segment_bytes=…). Defaults to 1 GiB. mmap_key falls back to the generic opt so an operator setting the cap once covers both backends. Tests added * TestNodeReconcileMembership, TestSnapshotEnvelopeMultiSM, TestRingConfigStateMachine, TestNodeAppliesRingConfigEntry in tests/pytests/unit/cluster/consensus/test_raft_log.py. * tests/pytests/unit/cluster/test_ring_membership.py — owns/rebuild semantics with empty and populated rings. * tests/pytests/unit/test_event_monitor_ring_gating.py — gate sites drop writes when key not owned; preserve writes under self-only ring (pytest-asyncio is not installed in salt; tests drive EventMonitor.handle_event via asyncio.run rather than the silent no-op @pytest.mark.asyncio decorator). * tests/pytests/unit/cache/test_mmap_cache.py — NUL-tail regression parametrised over representative trailing bytes; in-place overwrite with shorter NUL-tailed values; mmap_cache_max_segment _bytes opt plumbing. * tests/pytests/unit/cache/test_mmap_key.py — opt plumbing + precedence rules. * tests/pytests/unit/utils/test_mmap_cache_segments.py — multi-segment atomic_rebuild round-trip + packed-offset arithmetic. * tests/pytests/unit/runners/test_cluster_runner.py — ring_info read-only contract; ring_set raises until IPC lands. * tests/pytests/functional/cluster/consensus/test_raft_compaction.py — membership + ring policy survive log compaction; install_snapshot reconciles peer table. * tests/pytests/functional/cluster/consensus/test_raft_service.py — TestRingRebuildHook + TestRingPolicyPlumbing covering the on_membership_change → _apply_ring_policy path under both members=self and members=voters. Smoke list extended; full smoke run is 239 passed, 1 skipped (the documented placeholder for the disruptive-candidate test).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.