Skip to content

feat: Phase F3 declared snapshot/restore capability with measured-fork branching and honest refusal - #68

Open
txmed82 wants to merge 3 commits into
mainfrom
feat/phase-f3-declared-branching
Open

feat: Phase F3 declared snapshot/restore capability with measured-fork branching and honest refusal#68
txmed82 wants to merge 3 commits into
mainfrom
feat/phase-f3-declared-branching

Conversation

@txmed82

@txmed82 txmed82 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

What

Phase F3 from docs/NEXT_STATUS.md: a declared snapshot/restore capability for simulator branching with honest branch-equivalence refusal. The kernel ships no verified snapshot/restore primitive (the bridges' snapshot() methods are read-only reports, not restorable checkpoints), so this PR:

  • adds the BranchingSupport ladder (unspecified < declared < prefix_replay < measured_fork; not_supported is a refusal state, never a requestable mechanism) on WorldCapabilities, riding the same positional task-vs-adapter cross-check as every other eligibility gate via gates() — a task can't grant itself a fork mechanism the installed adapter withholds;
  • measure_branch_support: two independent forked rollouts of one seed grade the ladder — measured_fork only from byte-equal canonical digests, prefix_replay only inside an explicit finite tolerance, otherwise not_supported with the first difference recorded. BranchEvidence keeps claimed separate from support so a loud declaration can't overwrite a weak measurement;
  • branch_from: every arm re-runs the shared prefix on a fresh instance and is rolled twice; a prefix that fails to replay is refused with zero candidate comparisons — a seed alone must not be asserted to restore a full mutable state;
  • require_branch_support: fail-closed gate — a bare declared claim may drive arms (DRIVING_MECHANISM) but can never certify equality (FORK_EQUALITY_MECHANISM), which needs the measured rung;
  • declarations: lumen-gym declares prefix_replay (the one mechanism the pinned stack proves, tests/test_lumen_branch.py); no built-in claims measured_fork — that rung is earned per adapter at run time.

Tests

tests/test_branching.py (new): deterministic scripted fakes (no external stack) pin the four contract rules — declared-but-unmeasured rejection, measurement-earned acceptance, prefix-divergence honest refusal (zero arms emitted), candidate-divergence detection (candidate_diverged=False on identical plans means the intervention measured nothing), numpy-action content digests, tolerance-rung grading, fail-closed gate matrix, and the task-vs-adapter cross-check.

All existing capability/conformance/bridge suites pass unchanged: ruff check + ruff format --check clean, mypy clean (180 files), 248 tests green across test_branching / test_world_kinds / test_simulation_bridges / test_conformance / test_lumen_branch / test_multi_modality_contracts / test_eval_run / test_eval_p3 / test_eval_v03 / test_eval_contracts.

Out of scope (stated, not silently skipped)

conformance.py gains no new check here (no Tier change); F4 (closed-loop simulator branching) and the measured_fork declaration for non-Lumen bridges stay open until their forks are actually measured; LUMEN_REPLAY stays unspecified — its adapter semantics aren't verified in-repo, and declaring an unproven fork mechanism is exactly what this PR refuses.

Roadmap: docs/NEXT_STATUS.md F row (F3), plan §F lines 422-427.

…k branching and honest refusal

- BranchingSupport ladder (unspecified < declared < prefix_replay < measured_fork;
  not_supported is a refusal state, never a requestable mechanism) on WorldCapabilities,
  riding the same positional task-vs-adapter cross-check as every other gate via gates().
- measure_branch_support: two independent forked rollouts of one seed grade the ladder;
  measured_fork only from byte-equal canonical digests, prefix_replay only within an
  explicit finite tolerance, otherwise not_supported with the first difference.
- branch_from: every arm re-runs the shared prefix on a fresh instance; a prefix that
  fails to replay is refused with zero candidate comparisons - a seed alone must not be
  asserted to restore a full mutable state. BranchEvidence keeps claimed separate from
  support so a loud declaration cannot overwrite a weak measurement.
- require_branch_support: fail-closed gate; a bare declaration may drive arms but can
  never certify equality; lumen-gym declares prefix_replay (the one mechanism the pinned
  stack proves), no built-in claims measured_fork.
@factory-droid

factory-droid Bot commented Sep 11, 2026

Copy link
Copy Markdown

Droid encountered an error —— View job

Droid Exec exited with code 1:
Tool "Edit" skipped (GPT-5.2 uses ApplyPatch instead).
Tool "Create" skipped (GPT-5.2 uses ApplyPatch instead).

Security Review

@txmed82

txmed82 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@droid review

@factory-droid

factory-droid Bot commented Sep 11, 2026

Copy link
Copy Markdown

Droid encountered an error —— View job

Droid Exec exited with code 1:
Tool "Edit" skipped (GPT-5.2 uses ApplyPatch instead).
Tool "Create" skipped (GPT-5.2 uses ApplyPatch instead).

@txmed82

txmed82 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@droid review

@factory-droid

factory-droid Bot commented Sep 11, 2026

Copy link
Copy Markdown

Droid encountered an error —— View job

Droid Exec exited with code 1:
Tool "Edit" skipped (GPT-5.2 uses ApplyPatch instead).
Tool "Create" skipped (GPT-5.2 uses ApplyPatch instead).

…rtification hole

_rollout discarded both observations, so a world whose nondeterminism lives only in
the policy-visible stream digested byte-equal and was certified measured_fork; two
different candidate plans whose reward/info are action-independent constants could
also digest equal, so a branch comparison could claim candidate divergence the
policy could refute. The reset record and every step entry now carry
jsonable(observation), and _first_difference names the diverging channel so a
refusal says which stream failed to replay.

Regression envs pin all three holes: observation-only jitter cannot certify a fork
(and is refused outright as an arm replay), and identical plans on worlds that
differ only in observation must digest apart. Verified counterfactually: with the
observation stripped from the trace each new test fails, with it recorded each
passes.
@txmed82

txmed82 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@droid review

@factory-droid

factory-droid Bot commented Sep 11, 2026

Copy link
Copy Markdown

Droid encountered an error —— View job

Droid Exec exited with code 1:
Tool "Edit" skipped (GPT-5.2 uses ApplyPatch instead).
Tool "Create" skipped (GPT-5.2 uses ApplyPatch instead).

@txmed82

txmed82 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@droid review

@factory-droid

factory-droid Bot commented Sep 11, 2026

Copy link
Copy Markdown

Droid finished @txmed82's task —— View job


Droid review complete

The branching ladder, fail-closed gate, and task-vs-adapter cross-check are carefully designed and well tested, with strong honest-refusal semantics. One high-confidence correctness bug: the tolerance branch in _compare_fork_pair grants prefix_replay for purely categorical/structural twin divergence because _float_delta contributes 0.0 for non-numeric leaves, contradicting the module's own stated contract. Tests, docs, exports, and the lumen-gym prefix_replay declaration otherwise check out against the implementation.

Posted 1 inline comment.

candidate_diverged=False,
max_float_delta=0.0,
)
if delta <= tolerance and len(first) == len(second):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Zero/finite tolerance excuses categorical and structural divergence, granting prefix_replay to worlds whose twins differ in non-float channels

_compare_fork_pair grades PREFIX_REPLAY whenever delta <= tolerance and trace lengths match, but _float_delta returns 0.0 for every non-numeric leaf: strings, bools (explicitly skipped), mismatched dict keys (only left.keys() & right.keys() is walked), and list elements past the shorter length all contribute nothing. So with the default tolerance=0.0 on both measure_branch_support and branch_from, two twin rollouts that diverge purely categorically — e.g. an engine that stamps a run id or timestamp string into info/reset_info, or flips terminated on the final step so lengths stay equal — produce delta == 0.0 <= 0.0 and are graded prefix_replay with prefix_equal=True, and the recorded first_difference even claims the digests 'differ within tolerance 0'. This directly contradicts _float_delta's own stated contract ('a structural difference is caught by the digest comparison, which is stricter than any tolerance can be') — the digest mismatch is detected but then explicitly excused by the tolerance branch. The trigger is realistic (any engine logging non-numeric run metadata) and the observable result is wrong: measure_branch_support certifies prefix replay for a world whose twins categorically diverged, one rung above the not_supported it should report, which then lets require_branch_support(FORK_EQUALITY_MECHANISM)-adjacent consumers trust a replay guarantee the engine never provided. The tolerance check should only excuse differences that are actually float deltas; a byte-different pair with any non-numeric divergence should fall through to not_supported regardless of tolerance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants