-
Notifications
You must be signed in to change notification settings - Fork 2
5.2 Checkpoint Resume Property Suite
Relevant source files
The following files were used as context for generating this wiki page:
The Checkpoint Resume Property Suite (tests/checkpoint_resume/) verifies state persistence and restoration invariants for SynapticMesh tests/checkpoint_resume/main.rs:3-9. It asserts that a SynapticMesh restored from a serialized checkpoint continues to execute tick-for-tick identically to an uninterrupted live mesh tests/checkpoint_resume/main.rs:3-4. The suite tests both JSON and postcard (binary) serialization formats across bounded CI runs and extensive nightly property-testing profiles tests/checkpoint_resume/compare.rs:10-36, tests/checkpoint_resume/main.rs:21-38.
Sources:
-
tests/checkpoint_resume/main.rs:1-41tests/checkpoint_resume/main.rs:1-41 -
tests/checkpoint_resume/compare.rs:10-36tests/checkpoint_resume/compare.rs:10-36
The test harness is structured across multiple submodules in tests/checkpoint_resume/:
-
main.rs: Test suite entry points, regression seeds, and frozen JSON boundary fixtures tests/checkpoint_resume/main.rs:42-140. -
harness.rs: Core types (Scenario,Recipe,TickEvent), RNG (SplitMix64), and global configuration constants tests/checkpoint_resume/harness.rs:1-166. -
generate.rs: Deterministic scenario generation from seed values tests/checkpoint_resume/generate.rs:1-29. -
compare.rs: Checkpoint round-trip execution, live-vs-restored equivalence validation, and counterexample shrinking tests/checkpoint_resume/compare.rs:1-180. -
recipes.rs: Hand-built and randomized scenario constructors tests/checkpoint_resume/recipes.rs:1-187.
The suite defines execution caps and environment overrides in tests/checkpoint_resume/harness.rs:
-
CI_CASES(512): Number of deterministic cases executed during standardcargo testtests/checkpoint_resume/harness.rs:9-10. -
NIGHTLY_CASES_DEFAULT(10_000): Default seed count for ignored nightly profiles tests/checkpoint_resume/harness.rs:12-13. -
NIGHTLY_CASES_ENV("CHECKPOINT_RESUME_CASES"): Environment variable override for nightly scale tests/checkpoint_resume/harness.rs:15-16. -
REGRESSION_SEEDS: Static slice storing once-failing seeds discovered outside the CI range tests/checkpoint_resume/harness.rs:18-20.
graph TD
A["resume_equivalence_seeded_ci"] --> B["Scenario::from_seed"]
B --> C["Scenario::mesh"]
C --> D["apply_event Prefix"]
D --> E["SerdeFormat::restore"]
E --> F["meshes_equivalent"]
F --> G["apply_event Suffix Tick"]
G --> H["meshes_equivalent Final"]
subGraph "Harness Modules"
M1["tests/checkpoint_resume/main.rs"]
M2["tests/checkpoint_resume/harness.rs"]
M3["tests/checkpoint_resume/generate.rs"]
M4["tests/checkpoint_resume/compare.rs"]
M5["tests/checkpoint_resume/recipes.rs"]
end
Figure 1: Checkpoint-resume execution flow from seed generation to live-vs-restored equivalence verification.
Sources:
-
tests/checkpoint_resume/harness.rs:9-20tests/checkpoint_resume/harness.rs:9-20 -
tests/checkpoint_resume/main.rs:60-74tests/checkpoint_resume/main.rs:60-74
Scenarios are built deterministically using a custom SplitMix64 pseudo-random number generator seeded by an integer index tests/checkpoint_resume/harness.rs:129-166, tests/checkpoint_resume/generate.rs:12-29. Each seed maps to one of ten distinct architectural Recipe variants defined in tests/checkpoint_resume/harness.rs tests/checkpoint_resume/harness.rs:25-56:
-
EmptyGraph: Zero-neuron topology with idle ticks tests/checkpoint_resume/generate.rs:43-53 -
SingleNeuron: Single neuron with optional self-loop and variable axonal delay tests/checkpoint_resume/recipes.rs:10-31 -
DelayZero: Multi-neuron graphs with zero-delay synaptic connections tests/checkpoint_resume/recipes.rs:33-56 -
DelayCapacity: Ring-buffer saturation and capacity boundary testing tests/checkpoint_resume/recipes.rs:58-81 -
MultiSpikeSameSlot: Multiple spikes arriving at identical ring-buffer delay slots tests/checkpoint_resume/recipes.rs:83-106 -
CheckpointBeforeDelivery: Checkpointing precisely one tick before in-flight spike delivery tests/checkpoint_resume/recipes.rs:108-128 -
SignedWeights: Interleaved excitatory and inhibitory polarities with graded and binary activations tests/checkpoint_resume/recipes.rs:130-151 -
EmptyTicks: Extended sequences of empty propagation ticks tests/checkpoint_resume/recipes.rs:153-168 -
GeneratedRandom: Erdos-Renyi random topology generation viagenerate_randomtests/checkpoint_resume/recipes.rs:170-177 -
GeneratedSmallWorld: Watts-Strogatz small-world topology generation viagenerate_small_worldtests/checkpoint_resume/recipes.rs:179-187
graph TD
Seed["u64 seed"] --> RecipeSel["Recipe::from_seed"]
RecipeSel --> SplitGen["SplitMix64::new"]
SplitGen --> ScenGen["Scenario::from_seed"]
subGraph "Recipes [tests/checkpoint_resume/recipes.rs]"
ScenGen --> R1["EmptyGraph"]
ScenGen --> R2["SingleNeuron"]
ScenGen --> R3["DelayZero"]
ScenGen --> R4["DelayCapacity"]
ScenGen --> R5["MultiSpikeSameSlot"]
ScenGen --> R6["CheckpointBeforeDelivery"]
ScenGen --> R7["SignedWeights"]
ScenGen --> R8["EmptyTicks"]
ScenGen --> R9["GeneratedRandom"]
ScenGen --> R10["GeneratedSmallWorld"]
end
Figure 2: Mapping from seed space to structural recipes within tests/checkpoint_resume/recipes.rs.
Sources:
-
tests/checkpoint_resume/harness.rs:25-56tests/checkpoint_resume/harness.rs:25-56 -
tests/checkpoint_resume/generate.rs:12-29tests/checkpoint_resume/generate.rs:12-29 -
tests/checkpoint_resume/recipes.rs:10-187tests/checkpoint_resume/recipes.rs:10-187
The comparison module (tests/checkpoint_resume/compare.rs) defines the SerdeFormat enum supporting Json and Postcard tests/checkpoint_resume/compare.rs:9-13.
The restoration method serializes a live SynapticMesh into a byte vector or string, then deserializes it into a distinct restored instance tests/checkpoint_resume/compare.rs:23-35.
The meshes_equivalent function serializes both live and restored meshes into serde_json::Value snapshots to compare graph layout, internal ring buffer slots, and current ticks tests/checkpoint_resume/compare.rs:42-79. If discrepancies occur, it generates a comprehensive diff summary reporting differing fields, ticks, and queued deliveries tests/checkpoint_resume/compare.rs:48-77.
graph TD
LiveMesh["SynapticMesh live"] --> SerdeAction["SerdeFormat::restore"]
SerdeAction -->|Json / Postcard| RestoredMesh["SynapticMesh restored"]
LiveMesh --> SnapLive["checkpoint_snapshot"]
RestoredMesh --> SnapRest["checkpoint_snapshot"]
SnapLive --> Comp["meshes_equivalent"]
SnapRest --> Comp
Comp -->|Match| Success["Ok(())"]
Comp -->|Mismatch| Err["Err(String diff)"]
Figure 3: Live-vs-restored equivalence check pipeline in tests/checkpoint_resume/compare.rs.
Sources:
-
tests/checkpoint_resume/compare.rs:9-79tests/checkpoint_resume/compare.rs:9-79
When property tests encounter a failure, the harness invokes a greedy minimization shrinker (shrink) to reduce counterexamples before panicking tests/checkpoint_resume/compare.rs:126-134.
The shrinking loop (shrink_step) attempts three reduction strategies sequentially tests/checkpoint_resume/compare.rs:136-140:
-
shrink_remove_one_event: Iteratively removes individual tick events from theprefixsequence tests/checkpoint_resume/compare.rs:142-159. -
shrink_remove_one_event: Iteratively removes individual tick events from thesuffixsequence tests/checkpoint_resume/compare.rs:142-159. -
shrink_remove_one_descriptor: Iteratively removes individualSynapseDescriptorentries from the graph topology while updatingbuffer_max_delaytests/checkpoint_resume/compare.rs:161-180.
Sources:
-
tests/checkpoint_resume/compare.rs:126-180tests/checkpoint_resume/compare.rs:126-180
The test suite includes explicit regression tests and frozen snapshots in tests/checkpoint_resume/main.rs:
-
resume_equivalence_seeded_ci: ExecutesCI_CASES(512) generated scenarios tests/checkpoint_resume/main.rs:60-65. -
resume_equivalence_regression_seeds: Replays persisted seeds fromREGRESSION_SEEDStests/checkpoint_resume/main.rs:67-74. -
regression_in_flight_delay2_json_and_postcard: Tests specific in-flight spike states with a delay of 2 ticks tests/checkpoint_resume/main.rs:76-98. -
regression_frozen_json_in_flight_before_delivery: Hardcodes a JSON checkpoint snapshot immediately prior to delivery to guard against accidental serde contract or ring layout breakages tests/checkpoint_resume/main.rs:100-140.
Sources:
-
tests/checkpoint_resume/main.rs:60-140tests/checkpoint_resume/main.rs:60-140
- 1. Overview
- 1.1. Getting Started & Public API
- 1.2. Release History and Versioning
- 2. Core Runtime: SynapticMesh
- 2.1. Propagation APIs and Tick Semantics
- 2.2. Spike Delay Buffer (Ring Buffer)
- 2.3. Core Types and Error Model
- 2.4. Checkpointing and Serde State Restoration
- 3. Topology Subsystem
- 3.1. SynapticGraph and CSR Representation
- 3.2. Topology Generators
- 3.3. Wiring Rules, Dale's Law and Delay Assignment
- 3.4. Topology Digest
- 4. Sparse Maps and Channel Routing
- 4.1. ChannelRouter and RouterConfig
- 4.2. Neuromodulation and Plasticity
- 4.3. SparseSynapticMap (CSR)
- 5. Testing, Benchmarks and Quality Gates
- 5.1. Propagation Contract Tests
- 5.2. Checkpoint Resume Property Suite
- 5.3. Benchmarks and Unit Test Module
- 6. Build, CI and Project Tooling
- 6.1. Cargo Manifest, Profiles and Dependencies
- 6.2. CI Workflows and Packaging Validation
- 6.3. Code Quality, Licensing and Review Gates
- 7. Glossary