fix(validation): reject contract-less plans at submit_plan#16
Merged
TueVNguyen merged 1 commit intoJul 3, 2026
Conversation
A task list of targetless work tasks over an empty contract/ directory previously passed every submit-time check vacuously: coverage has nothing to cover, and work tasks may legitimately have no targets. The mission then runs with no assertions, no validators, and no gates -- the terminal reviewer becomes the only line of defense, on a harness whose core claim is that done is structurally hard to fake. submit_plan also cannot rely on the contract_dir_missing parse error here, because it calls ensure_contract_dir() first, which creates the (empty) directory. Add an empty_contract check as the first submit-time validation, so a plan cannot be submitted before at least one contract/<ID>.md assertion exists. Reported before empty_task_list, since contract authoring precedes task authoring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
|
Thanks @Quigleybits, It looks good to me. |
jnorthrup
pushed a commit
to jnorthrup/zenith
that referenced
this pull request
Jul 7, 2026
…ce-nonempty-contract-at-submit-plan fix(validation): reject contract-less plans at submit_plan
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The hole
submit_plancurrently accepts a plan with zero contract assertions. This degenerate plan passes every submit-time check today:{"tasks": [{"id": "w1", "type": "work", "body": "do the mission", "targets": [], "skill": "s"}]}with
missions/<mid>/contract/empty. Each guard is individually correct but vacuous in combination:check_coverageiterates over contract ids — with none, there is nothing to be uncovered.worktasks may legitimately have emptytargets(setup/discovery/integration nodes), so shape checks pass.contract_dir_missingparse error can never fire fromsubmit_plan, because the controller callsensure_contract_dir()first, which creates the (empty) directory.Downstream, nothing recovers: the coordinator backfills contract state from the same empty directory, no validator or gate can exist (both require targets that must resolve to assertions), and
close_missionchecks only for runnable work before going straight to terminal review. The mission runs as a plain todo list — no assertions, no independent validators, no dissent-blocking gates — and the mission-blind terminal reviewer becomes the only line of defense for the property the whole runtime exists to enforce structurally.The orchestrator system prompt does mandate contract authoring before
submit_plan, but that is prompt discipline, not a runtime invariant — and Zenith is distributed for arbitrary host agents, not just the bundled prompt. This PR moves the smallest load-bearing slice of that prose into code, in the same spirit as the tech report's harness ladder (each rung turns an observed prompt-level failure mode into a mechanism).The fix
One new first check in
validate_task_list_submission:empty_contract— rejected when no contract assertion files exist for the mission.empty_task_list, since contract authoring precedes task authoring; the error text tells the orchestrator exactly what to do (write contract/<ID>.md files before submit_plan).submit_plantool description now names the invariant, so host orchestrators see it in the tool schema too.The patch path is deliberately untouched:
apply_patchre-runs the individual checks and patches can only add assertions, so a mission that entered running state can never regress to an empty contract.Tests
test_empty_contract_rejected— the exact degenerate plan above, red onmain(returns[]), green here.test_empty_contract_reported_before_empty_task_list— pins the error-ordering choice.test_contract_less_plan_rejected— controller-level:submit_planraisesinvalid_task_listwith theempty_contractdetail and the project stays inmission_planning.ruff check .,mypy src, andpytest -qall pass (199 passed, 7 ACP smoke tests skipped, on Linux/py3.12).🤖 Generated with Claude Code