Skip to content

feat(governor): Lane H PR-2 — TOML policy file loader + validator#1349

Closed
joelteply wants to merge 1 commit into
feat/substrate-governor-pr1-typesfrom
feat/substrate-governor-pr2-toml-loader
Closed

feat(governor): Lane H PR-2 — TOML policy file loader + validator#1349
joelteply wants to merge 1 commit into
feat/substrate-governor-pr1-typesfrom
feat/substrate-governor-pr2-toml-loader

Conversation

@joelteply
Copy link
Copy Markdown
Contributor

Summary

Lane H PR-2 per GENOME-FOUNDRY-SENTINEL #1327 Part 11 'Policy File Format'. Stacks on #1345 (PR-1 governor-types).

PR-1 defined the published GovernorPolicy shape. This PR-2 reads a TOML file matching the spec's schema and converts it to a GovernorPolicy. PR-3 wires the file watcher + cascade state machine.

What ships

src/workers/continuum-core/src/governor/policy_file.rs:

  • PolicyFile + file-format sibling structs (TierSizesFile, CadenceMultipliersFile, etc.) — snake_case for TOML idiom, separate from camelCase wire-format types in types.rs. The file-format → wire-format hop happens in into_governor_policy.
  • parse_policy_text(text) — pure parser, testable with embedded TOML strings.
  • load_policy_file(path) — thin file-opener wrapping the parser.
  • validate() — enforces semantic invariants:
    • recall_weights sum to 1.0 within RECALL_WEIGHTS_TOLERANCE = 0.01
    • tier_sizes all > 0 (zero would disable a tier)
    • cadence_multipliers >= 1.0 (< 1.0 would speed up cadence; typo)
  • into_governor_policy(file, hw_class, ts) — composes file + caller-supplied HardwareClass + timestamp into the published GovernorPolicy.
  • PolicyFileError typed enum with Display + Error + From<io::Error> + From<toml::de::Error>.

Failure-mode discipline

  • Imbalanced recall_weights returns RecallWeightsImbalanced { sum, tolerance } — not silently rescaled. Operator sees what they typed.
  • Zero tier_size returns InvalidTierSize { field, value } per-field.
  • cadence_multiplier < 1.0 returns InvalidCadenceMultiplier { field, value }.
  • TOML syntax errors propagate as PolicyFileError::Toml.
  • Missing file returns PolicyFileError::Io with the path named.

Test plan

17 passing on cargo test --lib --features metal,accelerate governor::policy_file::

  • Canonical M-Air policy parses + validates (from spec)
  • Canonical Blackwell 5090 policy parses + validates (same schema, larger numbers — pins scaling across hardware range)
  • Imbalanced recall_weights rejected (with sum named)
  • Exact-1.0 recall_weights accepted (boundary)
  • Zero l1_lora_layers rejected (with field named)
  • Zero any tier_size rejected (loop over all fields — catches future additions)
  • cadence_multiplier < 1.0 rejected (with field + value)
  • cadence_multiplier = 1.0 accepted (boundary)
  • into_governor_policy composes correctly
  • load_policy_file reads valid file (I/O smoke)
  • Nonexistent path → Io err
  • Invalid TOML → Toml err
  • PolicyFileError Display + Error trait
  • From<io::Error> + From<toml::de::Error>
  • SpeculationLevel kebab-case strings parse (off/conservative/balanced/aggressive)
  • ConsolidationSchedule kebab-case strings parse (always/idle/idle-plugged-in/manual)
  • Full pipeline: hw_probe → classify_hardware → parse_policy_text → into_governor_policy

VDD evidence

N/A — pure parser + validator. Evidence with PR-3 when governor reads policy in production.

Stack

When #1345 merges, this PR rebases cleanly onto canary.

Per GENOME-FOUNDRY-SENTINEL #1327 Part 11 'Policy File Format'.
Stacks on #1345 (PR-1 governor-types).

What ships in src/workers/continuum-core/src/governor/policy_file.rs:

- PolicyFile + file-format sibling structs (TierSizesFile,
  CadenceMultipliersFile, ConcurrencyCapsFile, FederationCadenceFile,
  RecallScoreWeightsFile, SpeculationFileSection, ConsolidationFileSection)
  — snake_case for TOML idiom, separate from wire-format camelCase
  types in types.rs
- parse_policy_text(text) — pure parser (no I/O), testable with
  embedded TOML strings
- load_policy_file(path) — thin file-opener wrapping parse_policy_text
- validate() — enforces semantic invariants:
  * recall_weights sum to 1.0 within RECALL_WEIGHTS_TOLERANCE (0.01)
  * tier_sizes all > 0 (zero would disable a tier; not supported)
  * cadence_multipliers >= 1.0 (< 1.0 would speed up cadence; typo)
- into_governor_policy(file, hw_class, ts) — composes file + caller-
  supplied HardwareClass + timestamp into the published GovernorPolicy
- PolicyFileError typed enum with Display + Error + From for io::Error
  + toml::de::Error

Failure-mode discipline:

- imbalanced recall_weights returns RecallWeightsImbalanced { sum,
  tolerance } — not silently rescaled. Operator sees what they typed.
- zero tier_size returns InvalidTierSize { field, value } per-field.
- cadence_multiplier < 1.0 returns InvalidCadenceMultiplier { field,
  value }.
- TOML syntax errors propagate as PolicyFileError::Toml.
- Missing file returns PolicyFileError::Io with the path named.

Tests: 17 passing on cargo test --lib --features metal,accelerate
governor::policy_file::

- canonical M-Air policy parses + validates (from spec)
- canonical Blackwell 5090 policy parses + validates (same schema,
  larger numbers — pins scaling)
- imbalanced recall_weights rejected (with sum named)
- exact-1.0 recall_weights accepted (boundary)
- zero l1_lora_layers rejected (with field named)
- zero any tier_size rejected (loop over all fields)
- cadence_multiplier < 1.0 rejected (with field + value)
- cadence_multiplier = 1.0 accepted (boundary)
- into_governor_policy composes correctly with hw_class
- load_policy_file reads valid file (I/O smoke)
- load_policy_file nonexistent → Io err
- load_policy_file invalid TOML → Toml err
- PolicyFileError Display + Error trait
- From<io::Error> + From<toml::de::Error>
- SpeculationLevel kebab-case strings parse (off/conservative/balanced/aggressive)
- ConsolidationSchedule kebab-case strings parse (always/idle/idle-plugged-in/manual)
- full pipeline: hw_probe → classify_hardware → parse_policy_text →
  into_governor_policy

Stack:
- #1335 hw_probe (MERGED)
- #1345 PR-1 governor-types (OPEN)
- This PR (PR-2): TOML loader + validator
- Future PR-3: file watcher (notify crate) + policy selection by
  HardwareClass fingerprint + cascade state machine + LocalSubstrateGovernor
  reference impl + arc_swap publish
- Future PR-4: PressureBroker → governor wiring

VDD evidence N/A — pure parser + validator. Evidence with PR-3 when
governor reads policy in production.
@joelteply joelteply deleted the branch feat/substrate-governor-pr1-types May 16, 2026 23:00
@joelteply joelteply closed this May 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant