Skip to content

Gate completion on active-solution validation and restrict full rewrites during recovery#285

Merged
mmprotest merged 1 commit into
mmprotest/implement-shell-normalization-for-command-executionfrom
mmprotest/implement-validation-based-completion-gating
Apr 11, 2026
Merged

Gate completion on active-solution validation and restrict full rewrites during recovery#285
mmprotest merged 1 commit into
mmprotest/implement-shell-normalization-for-command-executionfrom
mmprotest/implement-validation-based-completion-gating

Conversation

@mmprotest

Copy link
Copy Markdown
Owner

Motivation

  • The runner could mark a mission as completed even when the active solution file still failed validation, and it could perform broad full-file rewrites of that active file during recovery; this change makes completion gating and edit discipline conservative and surgical.
  • Keep behavior minimal and generic: track the most recent validation of the active solution file, block mission success on failed/unknown validation, and prevent one-shot full rewrites of that file during recovery while allowing bounded patches and helper-file edits.

Description

  • Added active-solution validation state to the runtime (_active_solution_last_validation_ok, _active_solution_last_validation_summary) and initialize/reset them with run state (in Runner.__init__ and run reset). (file: villani_code/state.py)
  • Gate final completion on that state by checking it in the run-time completion guard (_budget_reason) and funneling final textual/no-tool completion branches through _finish_bounded so the termination reason reflects blocked completion when validation is missing/failed. (file: villani_code/state.py, function: Runner.run internals / _budget_reason / completion paths)
  • Enhance verification flow to detect whether a validation/run command targeted the active solution file and mark _active_solution_last_validation_ok = False whenever the command shows a hard failure signal (non-zero exit or crash tokens like SyntaxError, NameError, ImportError, ModuleNotFoundError, traceback/unhandled exception); set it back to True on clean validation. Added helper utilities _command_targets_active_solution and _has_hard_failure_signal. (file: villani_code/state_runtime.py, function: run_verification + helpers)
  • Tighten recovery edit guard: when _recovery_mode is active and the active solution file has a recent hard failure (_active_solution_last_validation_ok is False), block broad full-file Write replacements of the active file and return compact structured JSON feedback; allow bounded Patch edits and edits to helper files. (file: villani_code/state_runtime.py, function: small_model_tool_guard)
  • Tests updated/added to validate behavior and discipline: assertions added to verification tests for active-solution validation state, added bounded execution tests to ensure completion is suppressed when validation is failing and allowed when validation is clean, and added mutation-policy tests to assert full-write blocking and bounded-patch allowance. (files: tests/test_state_runtime.py, tests/test_bounded_execution.py, tests/test_state_tooling_mutation_policy.py)

Testing

  • Ran the targeted test set: pytest -q tests/test_state_runtime.py tests/test_state_tooling_mutation_policy.py tests/test_bounded_execution.py and all tests passed (61 passed).
  • The modified tests exercise: failing validation detection (SyntaxError/NameError/ImportError), recovery-mode state transitions, completion gating when active-solution validation is false/true, and recovery write/patch blocking behavior; they all passed.

Codex Task

@mmprotest mmprotest merged commit 65a2038 into mmprotest/implement-shell-normalization-for-command-execution Apr 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant