docs: delivery summary for the M0 opening slice#6
Conversation
Closes the devague loop for this workforce run: planned versus actual, the mid-work decisions, the three recorded deviations, and delivery claims stated at the strength the evidence supports. Reports the run as PARTIAL (BLOCKED), not complete. 3 of 19 confirmed tasks delivered; 12 blocked behind a colleague release that does not exist yet; 4 belong to colleague's repository. Written by hand against the 19 confirmed tasks rather than from `devague summary`, which renders all 87 tasks — including 68 rejected during planning — as delivery rows (devague#88).
|
/agentic_review |
PR Summary by QodoDocs: add M0 delivery summary and bump version to 0.8.3
AI Description
Diagram
High-Level Assessment
Files changed (3)
|
Code Review by Qodo
1.
|
Qodo flagged README's environment table as claiming container isolation while the same README states no runner is built. It was right, and this is the exact class of overclaim tests/test_honesty.py exists to prevent — the guard missed it because _CLAIM only matches "sandbox" and "fully isolated", not a table cell reading "Execution isolation". The table itself is legitimate: the two-axis model is the spec and writing it down is the point. The problem was framing it so a reader takes it for shipped behaviour. So the Guarantee column is now "Intended guarantee", every row carries an explicit Built? cell (all No today), and the section opens by saying every row is a design target rather than a description. It closes with the honest one-word summary of what shell-cli isolates today: nothing. Guard the regression class rather than the instance: two new tests assert the section is framed as a design target and that EVERY runner row ends in a Built? cell of No/Yes. Verified non-vacuous — both fail against the table as previously shipped.
colleague_inventory.py is an enforcement gate: it exists so a NEW unmediated
subprocess path in colleague cannot land unnoticed. Every one of these bugs
attacks that guarantee, so each is fixed test-first.
1. Alias imports evaded the scanner (false negative — worst class of bug for
a gate). Matching only the literal dotted text meant `import subprocess as
sp; sp.run(...)`, `from subprocess import run; run(...)`, and
`from subprocess import run as r; r(...)` all scanned clean. A gate with a
one-line bypass is theatre. scan_file now pre-builds a binding map from the
module's own ast.Import/ast.ImportFrom nodes and resolves call targets
through it: an Attribute base through module aliases, a bare Name through
directly imported functions. Building the map from real import statements
only is what keeps a locally defined `def run(...)` out of the findings —
and it also removes an existing false positive, `import mything as os`
followed by `os.system(...)`, which the old literal match counted. Relative
imports are ignored; an unbound attribute base falls through as itself, so
the gate fails closed.
2. Scan failures were fail-open (reliability). A SyntaxError or
UnicodeDecodeError returned [], so an unreadable file was indistinguishable
from a spawn-free one: --check could pass and debt_remaining could look
clean while part of the checkout was never examined. Inventory now records
`skipped` (module -> reason), publishes it in both --json and the human
output, and --check fails with exit 2 when anything was skipped. Exit 2
("the scanner cannot be trusted") is deliberately distinct from exit 1
("the scan ran and found an unclassified path"), matching the convention
already used for a missing checkout. A gate that silently degrades to a
partial scan is worse than no gate.
3. ALLOWLIST lookups broke on Windows (correctness). `str(path.relative_to(
pkg))` renders with the host separator, so `resident\steward.py` could
never match the forward-slash key `resident/steward.py` — a classified
module reported as unclassified and CI failing for the wrong reason. A new
`module_key()` helper normalizes with .as_posix(); it takes PurePath so the
guarantee is tested against a PureWindowsPath rather than depending on the
host OS.
Verified against the pinned colleague checkout (28fee290, 1.51.0): counts are
unchanged at 21 spawn sites / 15 modules / debt_remaining 13, zero skipped,
exit 0. colleague uses plain `import subprocess` throughout, so bug 1 was a
latent bypass rather than an active under-report — no pinned figure in
CLAUDE.md, the plan, or the issue #1 comment needs correcting.
Tests: 17 -> 41 in tests/test_colleague_inventory.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D6ayjKDSGV7GhGeu4Goo61
An adversarial live test landed 30 executed evasions at exit 0 and 6 false positives against colleague_inventory.py (issue #7). The claim made in t73, in the issue #1 section 17 comment, and in the first draft of the delivery summary — that a new unclassified spawn path fails CI immediately — is false and is retracted. The architectural one: ALLOWLIST is keyed per MODULE, not per SITE. Three brand-new spawns added to tools.py, two of them shell=True running rm -rf and cat /etc/shadow, returned exit 0 with no signal. Fifteen modules are already allow-listed, so a new unmediated path in any existing file is invisible by design. Widening the detected-call set does not fix that; per-site pinning would. Also: subprocess.getoutput defeats it in one line, and one level of indirection (sp = subprocess; sp.run) defeats the alias hardening added for the qodo review. Relabel rather than overclaim-and-patch. The scanner docstring, the CI job comment, CLAUDE.md and the delivery summary now all say drift detector against a pinned baseline. The structural fixes are scoped in #7 as their own reviewed slice, because they change the pinned baseline and the tool's contract. The numbers themselves stand and are still mechanically derived: 21 spawn literals across 15 modules, 13 debt, reproduced exactly at the pinned SHA. What changed is the strength of the guarantee around them. One figure gains a caveat instead of a retraction — shell=True detection requires a literal ast.Constant, so "exactly two" describes spelling, not behaviour. The honest ceiling for a static AST scan is the posture this repo already commits to for the execution guard: it catches accidental and careless drift, not adversarial evasion. The tooling should promise no more than the product does.
Three qodo findings on the summary, all fair. "delivered" was doing too much work — t72/t73 read "PR #5, open and green", which was both ambiguous and, once #5 merged, stale. Replaced with a defined vocabulary: released (merged and published), submitted (green in a PR, not merged), blocked, not ours. t71-t73 are released; the corrections in #6 are submitted. The per-deviation "(acceptable)" labels read as operator verdicts. They are the classification field the AGENT proposed; all three deviations are still status: proposed and none was self-confirmed, because confirming is a user-only decision under devague's contract — specifically so an agent cannot ratify its own departure from the plan. Said so explicitly rather than leaving it inferable. The filename date is the plan's creation date, matching its siblings under docs/specs and docs/plans; the run happened a day later. Stated the convention instead of leaving two dates unexplained.
|
FIXED as a wording change, with a deliberate PUSHBACK on the remedy. You read the parenthetical correctly as ambiguous — but They will stay that way until a human runs So the fix is to say that plainly rather than to advance the state: each line now reads
|
|
FIXED. Both dates were real, neither was explained. The filename carries Rather than renaming the file and breaking that alignment, the header now states the convention explicitly and separates the two dates. Removed the bare
|
|
FIXED, and you caught something that had also gone stale — PR #5 merged after that text was written, so "open and green" was wrong as well as ambiguous. The table now defines its vocabulary up front:
One addition while fixing this:
|
colleague's third drive succeeded after two that completed zero steps, and found something the other two reviewers did not: the honesty guard added earlier in this branch selected table rows by matching the strings "Host" or "Container". Rename a runner and that filter matches nothing, asserts nothing, and reports green — a guard that quietly stops guarding. Rewritten to select rows structurally from the pipe-table header separator, and verified to fail against a table whose runner column reads Firecracker. Also corrected the "fail closed" comment on the unbound-attribute fall-through, which colleague flagged as contradicting a test. It does not — one is the bound case, the other unbound — but the comment invited reading over-reporting as a safety property when it is a known false-positive source (issue #7). Recorded d4, correcting d1. That deviation dropped colleague from the reviewer role on the evidence of two consecutive zero-step drives. The evidence was real but the sample was two, and the failure turns out to be intermittent rather than deterministic: roughly one drive in three returned usable work. The honest conclusion is not "unusable" but "not reliable enough for a critical path, worth asking when the request is read-only and failure costs only the wait" — which is what the ask-colleague skill already says about its read-only verbs.
|



Closure artifact for the Milestone 0 workforce run. Separate from #5 only
because #5 merged before this was written.
Records the run as partial (blocked), not complete:
t71,t72,t73)The critical path stops at
t74, which captures characterization fixtures froma fixed colleague. Capturing them from today's colleague would enshrine the
defective behaviour as the parity baseline — the exact outcome the
characterization step exists to prevent. So the block is respected rather than
worked around, and the summary names the single action that unblocks 12 tasks.
Delivery claims are stated at the strength the evidence supports and no higher,
including one row that claims nothing: "The extraction seam is proven — None.
Not claimed." No
Operationtype exists yet.Also records the three deviations (
d1–d3, human-readable in #4), mostnotably that colleague was dropped from the implementer and reviewer roles after
two consecutive drives completed zero steps.
Written by hand against the 19 confirmed tasks rather than generated:
devague summaryrenders all 87 tasks — including the 68 rejected during planning — asdelivery rows (devague#88).
66 tests, markdownlint 0 errors.