Gate System

Gate System and Manifests

Every stage in CYPForge is a small program that produces evidence, and every transition between stages is governed by a gate that reads that evidence and decides whether the workflow continues. This page documents the gate semantics and the manifest schema.

Three gate outcomes

Outcome	Meaning	What the orchestrator does
PASS	All hard checks for this stage succeeded; no unresolved warnings of significance	proceed to the next stage
WARN	All hard checks succeeded but the stage flagged something a human should review	pause unless --auto-accept-warn was set at init; record the WARN in the manifest
FAIL	A hard check failed; the stage's output is not trustworthy	stop the workflow, mark the run as STOPPED_ON_FAIL, do not run downstream stages

The semantics are monotonic with respect to evidence: a stage that emits any FAIL is FAIL; a stage that emits no FAIL but at least one WARN is WARN; only a stage that emits no FAIL and no WARN is PASS. There is no "FAIL but overridable in code" — overrides exist only at the orchestrator level (--auto-accept-warn lets the workflow auto-continue past WARN, but never past FAIL).

What a manifest looks like

Every stage writes <stage_dir>/<stage_name>.manifest.json. The schema, in spirit:

{
  "stage": "core1_prepare_heme_cym",
  "version": "1.3.0",
  "started_at": "2026-06-24T14:01:32Z",
  "finished_at": "2026-06-24T14:01:38Z",
  "duration_seconds": 6.1,
  "inputs": {
    "pdb": "C:/cypforge_runs/my_run/inputs/protein_heme_ligand.pdb",
    "heme_state": "IC6",
    "axial_cys_resid": 442,
    ...
  },
  "outputs": {
    "prepared_pdb": "01_heme_only/prepared.pdb",
    "heme_mol2":   "01_heme_only/HEM.mol2",
    "cyp_mol2":    "01_heme_only/CYP.mol2",
    "frcmod":      "01_heme_only/IC6.frcmod"
  },
  "checks": [
    { "name": "fe_present",            "status": "PASS", "value": "FE @ (12.34, 5.67, -1.23)" },
    { "name": "axial_cys_renamed",     "status": "PASS", "value": "A:442 CYS→CYM" },
    { "name": "fe_sg_distance",        "status": "PASS", "value": "2.41 Å",   "spec": "2.0-3.0 Å" },
    { "name": "propionate_qc",         "status": "PASS", "value": "all three signed-distance products positive" },
    { "name": "axial_cys_uniqueness",  "status": "WARN", "value": "two CYS within 4 Å of Fe; chose closer one" }
  ],
  "gate": "WARN",
  "command": "cypforge module heme prepare --heme-state IC6 ...",
  "log": "logs/01_heme_only.log"
}

Three properties of this schema make the audit chain trustworthy:

1. Every check has a name and a status. The gate evaluator does not have to know what each check means; it counts FAIL, WARN, PASS and produces the stage outcome by the rules above.
2. Every check that produces a numeric value also records the spec. If a future you re-reads the manifest, you can see not only what was measured but what the criterion was at the time.
3. The exact subprocess command is logged. The manifest is reproducible from the inputs without ambiguity.

How the gates evaluator works

src/cypforge_core/orchestrator/gates.py implements GateChecker. Its contract:

def check(stage_dir: Path) -> GateResult:
    manifest = json.load(stage_dir / f"{stage_name}.manifest.json")
    statuses = [c["status"] for c in manifest["checks"]]
    if "FAIL" in statuses:
        return GateResult(gate="FAIL", reason="...")
    if "WARN" in statuses:
        return GateResult(gate="WARN", reason="...")
    return GateResult(gate="PASS")

The actual implementation also handles schema versioning, missing files, and the special case where a stage was skipped explicitly (SKIPPED). But the core rule is the three-line rule above.

Per-stage gating criteria (summary)

Stage	Hard gates (any FAIL stops the workflow)	Soft gates (WARN)
environment_check	tleap, pmemd.cuda, cpptraj, antechamber, parmchk2 resolvable; Multiwfn_noGUI present if RESP enabled	optional tools missing
core1_prepare_heme_cym	Fe present in PDB; axial Cys uniquely identifiable; Fe–S distance ∈ [2.0, 3.0] Å; propionate-side QC products all positive	secondary Cys within 4 Å of Fe; near-degenerate SVD plane
core2_prepare_ligand_resp_gaff2	strict graph isomorphism unique OR fallback returns unique/equivalent_ok; heavy-atom RMSD ≤ 0.05 Å; charge sum within 1e-4 of declared $Q_L$	equivalent_ok recorded; atom-name set mismatch with fallback merge
core3_finalize_protonation	protonation_decision.json present and parseable; every targeted residue resolved	warnings from external tools (PROPKA, etc.) if cited
core3_solvate_ionize	tleap exit 0; topology contains expected residues; net charge ≈ 0 after ionization	unusual box dimensions; high ion count
core3_render_pre_md	all 9 mdin files generated; references resolve	none typical
core3_run_pre_md	all 9 stages complete; energies finite; stage-09 NPT temperature/density stable	minor restart events; vacuum bubble warnings
global_audit	Fe still bonded to SG; heme planarity preserved; ligand still in pocket; ion balance	per-residue energy outliers
equilibration_decision	stage-09 ⟨RMSD⟩ within tolerance; temperature/density converged	drift near tolerance boundary
production_readiness_check	all upstream stages PASS; final topology + restart files present and parseable	none (this is the final summary)

The WARN boundary

WARN is not a degraded PASS. It is a checkpoint where a human (or a sufficiently capable agent) is expected to read the report and accept or reject the situation. The cases that produce a WARN are typically:

• a fallback path was taken (e.g. equivalent_ok in ligand mapping),
• a tool exited 0 with non-empty stderr that didn't match a known harmless pattern,
• a measurement landed near a tolerance boundary,
• a multi-CYS active site required the orchestrator to choose between candidates.

Theorem 11.1 in S3 explicitly treats WARN paths as audit boundaries — they are admitted as evidence but do not grant the unconditional uniqueness/consistency conclusion. The proof structure is honest about this.

Auto-accept WARN

If you pass --auto-accept-warn at init, the orchestrator continues past WARN automatically and records the acceptance in run_manifest.json. Use this for batch runs where a human cannot be in the loop and you have already validated that the WARN patterns this system produces are harmless.

We recommend not using --auto-accept-warn for first-time runs of a new system. Read each WARN once, decide what it means, then enable auto-accept once you understand the pattern.

The run manifest

run_manifest.json is the top-level state file for a run. It records:

{
  "run_name": "my_run",
  "run_root": "C:/cypforge_runs/my_run",
  "created_at": "2026-06-24T13:55:00Z",
  "config_hash": "sha256:...",
  "auto_accept_warn": false,
  "stages": [
    { "id": "00_environment_check",            "status": "PASS", "gate": "PASS", "manifest": "00_environment_check/manifest.json" },
    { "id": "01_heme_only",                    "status": "PASS", "gate": "WARN", "manifest": "01_heme_only/01_heme_only.manifest.json", "warn_accepted": false },
    { "id": "02_heme_mapping_leapin",          "status": "PENDING" },
    ...
  ],
  "workflow_state": "PAUSED_ON_WARN",
  "next_stage": "02_heme_mapping_leapin"
}

status is the orchestrator's view; gate is the stage's self-assessment. They normally agree, but the orchestrator can mark a stage STOPPED_ON_FAIL even when the stage itself produced WARN (e.g. because of a preceding FAIL).

What "audit trail" means concretely

If a reviewer asks "how did you decide the axial cysteine, the heme state, the ligand protonation, and where do I see the evidence?", you can hand them:

• run_config.json — every decision you made at init (heme state, axial Cys ID, ligand chain, formal charge, spin),
• 01_heme_only/*.manifest.json — every measurement the heme placement step made (Fe coordinates, Fe–S distance, propionate-QC products),
• 10_ligand_gpu4pyscf_esp/*.manifest.json — the SDF graph match, the RESP charge sum, the heavy-atom RMSD,
• 14_complex_protonation_finalize/*.manifest.json — every protonation decision applied, with the residue ID it touched and the source protonation_decision.json,
• 18_global_cyp450_audit/global_audit_report.md — the final geometric and chemical audit.

This is reproducibility in the strictest sense: not just "the code is available" but "every numeric decision the code made on this particular system is recorded with the spec it was checked against".

Next: FAQ for common errors and recovery.