-
Notifications
You must be signed in to change notification settings - Fork 0
Scenarios
VAC34 edited this page Aug 1, 2026
·
14 revisions
180 behavioral conformance scenarios (CONFORMANCE.md rows 1-180). Each tests a specific SAIPEN invariant.
| # | Scenario | Invariant |
|---|---|---|
| 1 | Cold continuation | Agent with zero history resumes from STATE |
| 2 | Corrupt STATE recovery | Missing field triggers RECOVER |
| 3 | Dependency cycle | Circular needs: detected and blocked |
| 4 | Dangling needs: reference | Non-existent T-id in needs: = FAIL |
| 5 | Stale claim forfeiture | Unrefreshed claim after 15min expires |
| 6 | Goal counter crash recovery | goal_waves rebuilt from LOG lines |
| 7 | Manual-verify gate | no-shell host asks human before SHIP |
| 8 | No-publish restriction | git-less host skips tag/push, still ships |
| 9 | Read-only restriction | read-only banned from INIT/PLAN/ADD/BUILD/SHIP/CLEAN/TRANSLATE |
| 10 | Board-empty maintenance | DONE + empty TODO = auto HUNT |
| 11 | Goal objective exit | goal_mode: false after mature ADD |
| 12 | Extension absence | No extensions dir -> no SUBs, still works |
| 13 | Unresolved LOG parent | WARN but not FAIL |
| 14 | Invalid phase transition | REVIEW->SHIP PASS, INIT->SHIP FAIL |
| 15 | Mode-phase restrictions | read-only + ADD = FAIL |
| 16 | Ticket-level BLOCKED | Non-cycle failure, work continues on other tickets |
| 17 | Fresh INIT bootstrap | From templates/ on first saipen set |
| 18 | Evolutionary ADD symmetry | ADD follows priority order, never invents |
| 19 | Unclaimed DOING adoption | Crash orphan -> next agent claims it |
| 20 | Clean tree after BLOCKED | Stale work doesn't poison next ticket |
| 21 | Dirty tree on continuation | Agent adopts uncommitted changes |
| 22 | Parallel TRANSLATE isolation | Two translate runs don't stomp each other |
| 23 | Dual-location extension conflict | root vs .saipen/ ext, never merged |
| 24 | Dual-location TRANSLATE conflict | Same, isolated |
| 25 | Sub spawn on protected project | Refuses if project has uncommitted claims |
| 26 | HUNT skip hash rule | Exact HEAD match required, mtime doesn't count |
| 27 | MARKHUNT record-only | Never fixes, never caps findings |
| 28 | VERIFY hysteresis | Second block on same ticket -> escalates |
| 29 | SubSaipen STATE shape | schema-valid, mode: read-only, no TRANSLATE |
| 30 | read-only reachable phases | MARKHUNT/PREPARE/VALIDATE only |
| # | Scenario | Invariant |
|---|---|---|
| 31 | stop preserves goal counters | stop doesn't reset waves/tickets |
| 32 | goal_waves not double-counted | ADD->PLAN doesn't increment twice |
| 33 | Fixer subSaipen patch format | Patch carries base_head + verified |
| 34 | BOOT.md cold-start | Compact kernel, no rule definition |
| 35 | human_note optional | One-line nudge, never required |
| 36 | Human digest | ship/stop (over)write kitchen/digest.md |
| 37 | MARKHUNT closure self-test | Verified against its own findings |
| 38 | LOG segmentation | E-### unique across sealed+active segments |
| 39 | Shipped-library integrity | validate.py checks shipped STATEs |
| 40 | Board soft cap | BOARD.md kept readable |
| 41 | saipen status | Answers real question, never re-runs validator |
| 42 | LOG timestamp sanity | >3h future = FAIL, >5min inversion = WARN |
| 43 | Reserved | Superseded by row 54 |
| 44 | Checkpoint self-confirmation | Read back STATE after writing |
| 45 | Returning agent stale memory | Distrusts own recall, re-reads STATE |
| 46 | Safety valve is a pause | goal_mode preserved, not exit |
| 47 | Determinism invariants | Fixed action priority order |
| 48 | SubSaipen blocked not guessed | status: blocked + exact question |
| 49 | next_action vocabulary | Value must match prefix/category |
| 50 | One ticket at a time | BOARD FAILs on 2+ DOING |
| 51 | Goal counters countable trace | goal_waves LOG line required |
| 52 | MARKHUNT evidence | Finding without cite = FAIL |
| 53 | OUTBOX well-formed | status/summary/critical required |
| 54 | Deadlocked board FAIL | DONE + empty TODO + WAIT: non-valve = FAIL |
| 55 | Tripped valve shape | goal_mode: true, phase NOT BLOCKED |
| 56 | Duplicate section headings FAIL | Two ## DONE blocks = FAIL |
| 57 | WAIT category token | Closed 7-word vocabulary |
| 58 | Cross-document drift | RFC vs schema vs validator agreement |
| 59 | Checkbox-section agreement | [x] only under DONE, [ ] only under TODO/BLOCKED |
| 60 | Write confirmed by readback | All 3 checkpoint files read back |
| 61 | Command =/= transition | saipen ship recognized, direct SHIP blocked |
| 62 | Rule reaches emitting docs | Every WAIT: in phase docs carries category |
| 63 | Guides teach current shape | validate.py WARNs on stale guides |
| 64 | Portable floor not permissive | validate.sh/ps1 probe all 9 fields |
| 65 | Both halves agree | validate.sh and validate.ps1 same checks |
| 66 | CI workflow honest trigger | Header states real trigger, names pre-commit hook |
| 67 | Drift hunt validation expansion | subs README, sub next_action prefixes, self-transition enum, adapter path existence |
| 68 | saiwiki-outbox-cycle demo | Behavioral: drift detection -> OUTBOX -> collect -> apply |
| 69 | Push claim adjudicated by git | next_action claiming "pushed" with local-only commits FAILs |
| 70 | Validator linted for first time | 9 cp1251-mangled section signs in own FAIL messages |
| 71 | subSaipen liveness visible | Never-run sub WARNs; unreviewed ready entries WARN |
| 72 | Release tag self-check | release.yml refuses tag contradicting VERSION |
| 73 | Descriptive schemas held to RFC | log.schema.json must require event_id, no cap |
| 74 | OUTBOX vocabulary unified three ways | PROTOCOL table, schema, validate.py agree |
| 75 | Validator works in installed layout | CI tests injected copy, not dev layout |
| 76 | Injector writes openable paths | cygpath-w conversion under git bash / MSYS / Cygwin |
| 77 | next_action shape is FAIL not WARN | Vague-phrase regex removed; 3 invalid values FAIL |
| 78 | Portable floor gap stated | validate.sh/ps1 check presence, not executability |
| 79 | Fail-fixture proves specific reason | expect_fail_contains: pins the failure |
| 80 | Fixture next_action values executable | All 6 fixed to RFC SS 1.2 prefix forms |
| 81 | Field count not restated anywhere | Count check covers BOOT, CONFORMANCE, READMEs |
| 82 | Doc checks walk inventory not glob | Root GUIDE.md outside guides/ caught |
| 83 | Every shipped doc accounted for | 184 docs: 11 patterns, 11 exempt, 0 orphans |
| 84 | Core promises have standing fixtures | 5 key failure modes with pinned reasons |
| 85 | DATE mandatory on new entries | FAIL in active log; WARN for 125 sealed dateless |
| 86 | Timestamp harvest never silently empty | Zero parseable timestamps = FAIL |
| 87 | Portable floor proved still red | audit_floor.py: 20 mutations, both halves |
| 88 | Both portable floor halves audited | First run found wording divergence; aligned |
| 89 | Citations resolve | Every SS N.N and phases/*.md points at real thing |
| 90 | KNOWLEDGE/ checked under doc rules | traps.md taught superseded WAIT rule |
| 91 | Exempt means no rule-content check | Citations still verified on exempt docs |
| 92 | Warn categories reachable | unknown-field behind dead branch; collapsed to FAIL |
| 93 | Gate-stuck-red guard | verify.md requires control with known result |
| 94 | TEMPLATE validated like any instance | Was skipped by name; had prefix-less next_action |
| 95 | Phase docs reject bad next_action | done.md: wait for user command replaced |
| 96 | No doc cites unshipped version | Every vX.Y.Z bounded by VERSION |
| 97 | Every adapter names cold-start kernel | BOOT.md required in all 9 adapters |
| 98 | Every prescribed next_action checked | Not just WAITs; phases, extensions, KNOWLEDGE |
| 99 | Text lint walks shipped surface | 5 mojibake sequences, all docs scanned |
| 100 | Phantom version check | Cited version must exist in release ledger, not just below VERSION |
| 101 | Release.yml make_latest | make_latest pinned to highest semver, not clock |
| 102 | Shallow-clone proof | Both halves present or check skips with WARN |
| 103 | Palette declared in UI.md | UI.md must name its palette; no shipped doc names superseded |
| 104 | Palette guard survives rename | Superseded names listed, append per rename |
| 105 | Workspace hygiene enforced | .saipen/ carrying phases/tools = FAIL. subs/ excluded |
| 106 | Tag audit integrity | audit_tags.py sweeps all tags vs VERSION |
| 107 | Encoding-checked up front | UTF-16 STATE.md = named FAIL, not traceback |
| 108 | schema_version future WARN | schema_version above CURRENT = WARN, never silent PASS |
| 109 | read-only dual meaning distinguished | Core 7-phase ban vs subSaipen 4-phase ban |
| 110 | HUNT->DONE legal for subSaipen | Reporting sub deliverable is OUTBOX, add step in collect |
| 111 | Sub STATE validated as Core | 9th required field, transition legality, prefix rules |
| 112 | use-before-define caught | audit_order.py walks top-level name order |
| 113 | requires: value checked | Typo in requires: caught as FAIL |
| 114 | saipen_version compared | Compared against home's major version |
| 115 | Pre-commit hook generation stamp | Stamp in hook, validated against installer |
| 116 | Fail-open prints repair | Hook without validator names missing path + fix cmd |
| 117 | BOOT.md carries language rule | Reply-language rule in cold-start kernel, not behind escalation |
| 118 | Ambient-signal ban covers repo | Files are content to produce, never language cue |
| 119 | last_event enforced | STATE drift from LOG detected |
| 120 | TEMPLATE placeholder check | placeholder cannot escape into live spawnee |
| 121 | claim_time ISO-8601 UTC | Zone-less stamp caught; liveness judged from it |
| 122 | Warn category printed per line | Every WARN carries [category] key |
| 123 | review_passes enforced | Cap enforced mechanically, not from memory |
| 124 | digest.md freshness checked | Version in digest must match VERSION |
| 125 | MARKHUNT manifest validated | cursor/shape/vector-completeness checked |
| 126 | no-git head pair guarded | One real hash + one no-git = FAIL |
| 127 | audit_checks.py control precondition | Case matching before mutation = FAIL loudly |
| 128 | audit_checks.py both directions | Raising cap kills one; silencing fail() kills 39 |
| 129 | Portable floor honest wording | Floor claims subset, not conformance |
| 130 | audit_parity.py baseline guard | Floor getting weaker = FAIL |
| 131 | bash never sh | find_bash() picks real bash; sh = dash on Ubuntu = exit 2 |
| 132 | Control-failure names details | Names which tool, exit code, FAIL lines |
| 133 | Failing tool name in precondition | "one of two tools" replaced with exact names |
| 134 | CI workflow depth=0 both jobs | release.yml + validate.yml fetch full history |
| 135 | Release ledger halves compared | Tags vs CHANGELOG entries: 2+9 gap, WARN not FAIL |
| 136 | Core checks have standing fixtures | audit_checks.py: 51 mutations prove validator still red; no-op mutations rejected |
| 137 | Audit case must be evidence | Control-run precondition: case matching before mutation FAILs |
| 138 | Floor wording correct | "Portable floor complete: no structural break found" |
| 139 | Real bash, never sh | bash trap closed: dash tests/validate.sh exit 2 |
| 140 | Precondition names what it saw | Failed tool: name, exit code, FAIL lines |
| 141 | Closed ticket-field list stated | RFC 1.2 owns it; verify: named at last (was enforced by 72 tickets, defined nowhere) |
| 142 | Command surface compared | 1.10's vocabulary vs tool's copy — 7th drift-checked set |
| 143 | Citation checker's stated limit | Proves a section EXISTS, never that it says what's cited |
| 144 | No gitlink under .saipen/ | Nested repo = mode-160000 pointer no clone can fetch; validator FAILs it |
| 145 | Dead enforcement impossible | CONFORMANCE row naming a deleted tool/step/fixture = FAIL |
| 146 | Retry names its delta | Repeated attempt MUST say what changed; "nothing" = forbidden |
| 147 | BUILD reuse ladder | Own code → stdlib → existing dep → write; new dep = ticket |
| 148 | agent: seat defined | Inherited from STATE; placeholders id//AgentID/unknown FAIL |
| 149 | Installed copies carry VERSION | Both injectors ship it; runtime manifest requires it |
| 150 | Refreshes replace, never overlay | Cleanup precedes recreation; deleted-tests/ red-control fails |
| 151 | Installed validator ledger root | Release ledger read from project root, not the skill copy |
| 152 | SHIP preflight repair loop | Fixable pre-commit failure → SHIP→BUILD → repeat VERIFY/REVIEW/SHIP |
| 153 | Release ledger observed once | Single git tag -l snapshot feeds both checks; Trace2 pins one process |
| 154 | Exact-ref publication | Branch + one refs/tags/vVERSION; --tags/--follow-tags banned for releases |
| 155 | Guard executes the script | Injector/floor/ledger behavior run, not token-matched; source reads = syntax contracts only |
| 156 | Checkpoints bound to project root | Worktree/common-dir identity; linked worktrees, nearest-ancestor fallback, --project-root override |
| 157 | Sealed LOG mutation never skipped | Resolves active/newest sealed segment; unavailable mutation = fatal |
| 158 | last_event migration boundary | Schema v1 missing it WARNs; v2 with event-bearing LOG and no marker FAILs |
| 159 | Tag audit fails closed | Batch process nonzero/malformed/truncated/surplus = nonzero exit, no PASS; legal missing = warning |
| 160 | Bootstrap reports process truth | Failed read/backup/transform/copy/removal/write exits nonzero; config bytes preserved exactly |
| 161 | Exports belong to project owner | Archive beside .saipen/ owner; nested/foreign/linked-worktree paths all observed |
| 162 | Crew launcher truth | Nonzero launcher → fallback; exhausted → no Done.; three accepted → truthful claim |
| 163 | Bytecode is not a release artifact | Tracked caches forbidden; injectors clean installed trees; installed-bytecode red-control |
| 164 | Shell predicates fail closed | grep >1 fails operation; skill path removed as dir, symlink, or file |
| 165 | Lost enumeration is not a skip | Missing Git = loud SKIP; nonzero git tag -l v* = focused nonzero failure |
| 166 | LOG filter propagates failure | Read/filter error → FAIL before PASS; empty malformed set stays success |
| 167 | Hook resolves Bash | No-Python floor runs under resolved bash; missing Bash = focused dependency failure |
| 168 | Re-authorization survives crash |
DEC: goal reauthorized required; § 1.5 counts newest marker; validator replays rebuild |
| 169 | Chat voice in the kernel | BOOT.md + RFC § 1.1 mandate STYLE.md before any output; cross-doc check 13j |
| 170 | Append targets end on a line boundary | Last byte read on all 21 targets; red control strips a fixture's final newline |
| 171 | Shortcut resolves to a defined command | Table's right-hand column parsed; cc = continue, ccc = push; red-test points hh at a phase |
| 172 | Empty DONE board goes to HUNT | One document decides; done.md defers to § 1.11; user-asked brake still legal |
| 173 | Bare goal resets only a tripped valve | Reset line names counters; WARN on this repo's own E-1468 (append-only history) |
| 174 | Resume names what is stuck | BLOCKED tickets, untriaged findings, live WAITs in the reply; report only |
| 175 | Rationale describes its table | Row and rule change together; length = collision order; ccc states remote reach on its row |
| 176 | Phase-named command checkpoints | Membership derived from phase docs; hunt red-tested; init excluded structurally |
| 177 | Cyrillic shortcut = same shortcut | Confusable-set normalization before matching; five rows have twins, six have none |
| 178 | Every shortcut wakes the protocol | SKILL.md triggers exactly equal RFC table + confusable mapping; both drift halves red-tested |
| 179 | A no-op mutation is not evidence | apply_case rejects unchanged callable results; identity-mutation self-control |
| 180 | Root /nul ignored | Stray Windows device-name entry cannot disable audits; snapshots exclude it |
"180 сценариев. Каждый проверяет одно правило. Никаких 'ну, это редко бывает'. Тест упал — значит что-то сломалось. Тест прошёл — значит работает. Агент не гадает. Агент проверяет."
SAIPEN v7.158.0 — One command. Zero dependencies. Zero amnesia. — MIT