Bound component execution in the verification runner so a hang still emits the aggregate report

[alexeygrigorev]

Grooming decision

Filed as a separate issue, not folded into #195 or #197. #195 is about components that complete with a non-success result under-reporting the counts their own logs hold (ci/evidence.py::machine_output_claim, inside create_report); this is about a component that never completes preventing create_report from being reached at all — a different file, a different layer, and #195's non-goals pin its verdict/report semantics tight. #197 is origin attribution for local evidence, an unrelated mechanism. The three issues are adjacent faces of one principle — the harness must always produce a truthful machine-readable aggregate — and each keeps a small attributable diff.

Problem

When a verification component command hangs, make verification-run never emits a final verification-report.json. Per-component evidence accumulates, but there is no aggregate artifact, so the handoff cannot carry the report path/digest _docs/PROCESS.md requires, and the engineer must hand-assemble the account.

Root cause (verified): ci/runner.py executes each component via subprocess.run(command, ..., check=False, stdout=PIPE, stderr=STDOUT) with no timeout — a hung child blocks the runner indefinitely. The Makefile chain only reaches verification-report-check (which invokes ci.verification report to write verification-report.json) after the runner exits. Other CI modules already bound their waits (ci/screenshot_runtime.py uses process.wait(timeout=...) with a graceful terminate; ci/container_check.py and ci/history.py use HTTP timeouts) — the runner's component execution is the unbounded outlier.

This violates the principle #193's verdict-precedence fix established and #195 carries forward: reports must stay emittable on failure. A hang is a failure mode; today it produces no aggregate at all.

Observed incidents (from #198 engineering, #198 (comment))

• Attempt 1: disk-full failures plus a wedged playwright teardown, manually terminated — "No final aggregate report file was produced (verification-report-check not reached)."
• Attempt 2: six components green with envelopes retained, but playwright hangs forever in a driver-level deadlock (test_accessibility.py mobile-viewport core smoke; faulthandler shows the main thread inside the Playwright driver event loop), reproduced identically at clean base b487b1f via stash A/B, and the owner's main-checkout run of the same module stayed wedged for 12 hours. "Because the runner hangs inside this component, make verification-run cannot emit a final verification-report.json."

Scope

• Bound every runner component execution with an explicit wall-clock timeout: a documented default generous enough for the longest legitimate suite, plus an override (CLI flag and/or plan field — engineering's choice, recorded in the doc).
• On expiry: terminate and then kill the child, retain whatever partial output exists under the component's output artifact, and record the component's result envelope as non-success with an explicit timeout indication and the exact command.
• Continue to the remaining components and finish the run so verification-report-check emits verification-report.json with a failure verdict; the timed-out component is an executed rerun with a failure result, never a skip.
• Document the default and override in _docs/ci/change-selective-ci.md.

Non-goals

• No change to verdict semantics, bucket taxonomy, validate_report, or reuse policy — that is Failed test components' evidence envelopes must carry the counts their logs hold #195's territory; this issue only guarantees the aggregate is reached.
• No signal-handler partial-report emission for externally killed runner processes (Ctrl-C/SIGKILL); the in-process timeout is the guaranteed path, and manual terminations remain operator-disclosed as today.
• No fixing of the underlying hangs themselves: the Playwright driver deadlock and the local-versus-CI chromium geometry drift that Re-pin the refreshed homepage subtitle through the deploy contract and fail drift in Django CI #198's engineering documented are environment/test defects to be intaken separately, not runner behavior.
• No new dependency.

Acceptance criteria

• The runner bounds every component execution with an explicit timeout (documented default plus override); no component command can block the run indefinitely.
• A timed-out component gets terminate-then-kill handling, its partial output is retained as the component's output artifact, and its result envelope records a non-success result with an explicit timeout indication and the exact command.
• Remaining components still execute and the run reaches verification-report-check: a final verification-report.json is emitted with a failure verdict and no silently skipped component.
• Regression tests in tests_ci/: a never-terminating component command under a small configured timeout yields the timeout result envelope and an aggregate report with a failure verdict; a normal green run is unaffected by the timeout plumbing.
• _docs/ci/change-selective-ci.md records the default and override; make test-ci, lint, format-check, and typecheck green; verification plan per _docs/ci/change-selective-ci.md with all component dispositions recorded.
• Screenshots: not_applicable — internal CI/process change with no product-page render impact, per _docs/PROCESS.md.

Dependencies: none. Coordination note: after #195 lands, a timed-out component's zero-count fallback falls out naturally from its unparseable-log path; before it, the existing non-success envelope behavior applies — neither ordering blocks this issue.

Normative references: _docs/PROCESS.md (evidence origins, report requirements), _docs/ci/change-selective-ci.md.

[alexeygrigorev]

Ownership correction to the non-goals above: the underlying Playwright driver deadlock and local design-parity reds observed during #198's engineering are already owned by #193, not intake candidates — they are that issue's failure classes (the deterministic-screenshot settle parking on lazy/src-less figures, and the classes-2-6 reds), reconciled in cdc656a on #193's branch, whose push-run playwright verdict is the acceptance criterion. The clean-base A/B in #198's report only proved they pre-date #198's diff. If local reds persist after #193 lands, a residual gets groomed then, with post-#193 evidence. #199's scope is unchanged and stays runner-level: note that cdc656a adds wedge-guard timeouts at the CI job level (45/30 minutes), which complements — and at a different layer than — this issue's local runner bound.

[alexeygrigorev]

Software Engineer Report

Issue #199 — bound component execution in the verification runner so a hang still emits the aggregate report.

Base/head: b487b1fc44372d83f35a2edd09457e6535ad189a (base = head; worktree mode with uncommitted candidate changes)
Worktree: .tmp/website-199-engineer, branch issue-199-runner-component-timeout, all changes left uncommitted for tester review.
Graph digest: 4127118b4467a75b40b8dbadc4050b5b2dc3762b292aca93cd9309a4593bca19
Plan digest: 97a0f17c0f8f3595b5d3b38f6cab2be2f1fa7eae1d6f89c87c5725a8b96cb1da
Verification state digest: 7dee52ce447b682d26ad7004d935934c3a7e7961bb485b31ccf5feb85abd0b61
Profile full, browser profile core; direct nodes surface.ci, surface.configuration, surface.documentation; no downstream nodes; risk flags dependency_toolchain, deployment_runtime, test_infrastructure.

Changed files

• ci/runner.py — every component command now runs under an explicit wall-clock bound (DEFAULT_COMPONENT_TIMEOUT_SECONDS = 3600.0, documented). execute_component starts each command in its own session with combined output written straight to the component's output artifact (partial output survives expiry); on expiry _terminate_process_group sends SIGTERM to the whole group, honors a grace window, escalates to SIGKILL, and drains the group to extinction before returning so late grandchild writes cannot land after the evidence digests are frozen. The result record/envelope carries result: "timed_out", timed_out: true, exit code 124, and the exact command; the remaining components still execute; run_plan rejects non-positive and non-finite bounds; --component-timeout-seconds is the diagnostic override (the Makefile target always uses the default).
• tests_ci/test_runner_timeout.py (new, 9 tests) — partial-output retention on expiry, a group that ignores SIGTERM is killed, straggler grandchild drained before return, plain/success/failure/killed status mapping, non-positive and non-finite bound rejection, end-to-end: a hanging component is bounded, recorded timed_out with exit 124 and its partial log, the other components still run, and create_report still emits a failure verdict with an empty skipped bucket; plus a green run unaffected by the plumbing.
• tests_ci/test_runner.py — the tester-role recording test now fakes the new Popen-based execution path.
• Makefile — verification-run captures the runner's nonzero status and still runs verification-report-check, so a failing-but-completing runner cannot suppress the aggregate report (collaborating-thread fix, preserved and validated here).
• tests_ci/test_make_verification.py — test_verification_run_reaches_report_after_runner_failure pins that Makefile behavior (collaborating-thread, preserved).
• _docs/ci/change-selective-ci.md — documents the bound, the default, the terminate-then-kill and partial-output semantics, the timed_out/124 result, the guaranteed failure-verdict report, and the override.

No Django application code, templates, or browser tests were touched, so no separate full Django run was required by scope; the run's django component executed the full suite anyway under the new bound (2549 passed, 9 skipped, 2558 tests).

Four buckets (engineer report)

• rerun (6): selector success (always_fresh_control_plane), evidence_validation success (always_fresh_control_plane, 540/540), quality failure (fresh_full_regression), django success (fresh_full_regression), playwright timed_out (fresh_full_regression), container success (fresh_full_regression, 13 assertions).
• reused (0): none — every required component was rerun; no provenance or expiry reused.
• skipped (0): empty.
• not_applicable (3): compatibility (outside_impact_closure), content_invariants (outside_large_content_closure), screenshots (render_inputs_unchanged_without_reusable).

Commands and counts

• make terminology-check — pass.
• make test-ci — 540 passed in 29.13s (frozen tree; includes the two collaborating-thread tests).
• make lint / make format-check / make typecheck — pass (559 files formatted, 416 source files clean).
• make verification-run VERIFY_WORKTREE=issue-199-runner-component-timeout VERIFY_ISSUE=199 (isolated VERIFY_PLAN/VERIFY_OUTPUT_DIR/VERIFY_EVIDENCE_DIR/VERIFY_REPORT under .tmp/verification/issue-199-engineer/) — runner exited 1 (two failing components), report-check exited 0, and the aggregate report was emitted with verdict failure.
• make verification-evidence-check / make verification-report-check against that plan/report — both exit 0 (independently revalidated after the run).

Real-hang proof (small override bound)

Scratch plugin .tmp/hang-proof/hang_plugin.py sleeps 600 s only for the playwright component's pytest invocation; the runner was invoked directly with --component-timeout-seconds 480:

• playwright recorded timed_out, exit 124, counts 0, at exactly the 480 s bound (2026-08-19T12:05:07Z → 12:13:07Z); its partial log retains the plugin's own line and make: *** [Makefile:289: test-playwright-core] Terminated.
• container then executed and succeeded (12:13:07Z → 12:13:59Z); all other components green.
• Evidence directory validated; report emitted and validated with verdict failure; skipped bucket empty.
• Transcript and artifacts: .tmp/hang-proof/runner.log (632 lines, includes component playwright exceeded its 480s wall-clock bound; terminated the process group and recording a timed_out result), .tmp/hang-proof/verification-report.json, .tmp/hang-proof/evidence/.

The official run additionally proved the real default bound against the known local Playwright wedge (#198 signature at test_accessibility_core_smoke[chromium-viewport1-mobile]): playwright timed out at exactly 3600 s (14:24:14Z → 15:24:14Z, exit 124, 13,628-byte partial log retained including make's own teardown lines), and container succeeded afterwards (15:24:14Z → 15:24:51Z).

Artifacts

• Plan: .tmp/verification/issue-199-engineer/verification-plan.json
• Report: .tmp/verification/issue-199-engineer/verification-report.json
• Evidence: .tmp/verification/issue-199-engineer/evidence/
• Hang proof: .tmp/hang-proof/

Screenshot requirement: not_applicable — render inputs unchanged; internal CI-harness change with no user-facing render journey.

Limitations

1. quality failed on VulnerabilityScanError: dependency advisory scan found 1 finding(s) (pip-audit advisory drift, external to this change): the same lockfile passed the same scan earlier the same day (11:10Z, 12:01Z, and 13:21Z runs) before failing at 14:20:40Z; the scan fetches advisories live. Nothing in this diff touches dependencies.
2. The playwright partial log also records two genuine test_studio_foundation failures before the wedge — this machine's browser suite was already failing independently of this change; the timeout converted an indefinite wedge into a bounded, reported failure.
3. Collaborating-thread edits landed in this worktree while runs were in flight (the Makefile status capture, the non-finite-bound validation with post-kill polling, two tests, one doc sentence). They are preserved, validated by the gates and the 540 count, and the official evidence run used the post-edit tree. My earlier in-flight run (pre-edit Makefile recipe, plan digest 4720a56d…) was invalidated by those mid-run edits, emitted no report, and is superseded by the official run above.
4. The 3600 s default is a documented policy choice; any legitimate suite exceeding one hour would be recorded as timed_out (the override exists for diagnostics only — the Makefile target always uses the default).

Tester re-run

cd .tmp/website-199-engineer
make terminology-check
make test-ci
make verification-plan VERIFY_ISSUE=199 VERIFY_CONSUMER=tester
make verification-run VERIFY_WORKTREE=issue-199-runner-component-timeout VERIFY_ISSUE=199 \
  VERIFY_CONSUMER=tester VERIFY_PRODUCER_ROLE=tester VERIFY_PHASE=tester

(VERIFY_ISSUE=199 must stay explicit; the Makefile default of 113 is #197's known breakage.)

Worktree frozen as of this report; all changes uncommitted.

[alexeygrigorev]

QA Review

FAIL — independent tester review of the frozen engineer worktree.

Scope and frozen identity

• Worktree: /home/alexey/git/dtc-website/.tmp/website-199-engineer
• Branch: issue-199-runner-component-timeout
• Observed HEAD: b487b1fc44372d83f35a2edd09457e6535ad189a
• Exact plan range: base = head = b487b1fc44372d83f35a2edd09457e6535ad189a; source mode worktree
• Worktree stayed frozen/uncommitted: exactly 6 candidate paths (Makefile, _docs/ci/change-selective-ci.md, ci/runner.py, tests_ci/test_make_verification.py, tests_ci/test_runner.py, tests_ci/test_runner_timeout.py); git diff --check passed; no product files were changed.

Plan, graph, and provenance validation

Engineer artifacts:

• Plan: .tmp/verification/issue-199-engineer/verification-plan.json (file SHA-256 94b131ba39424d2042a2966e29da435c385152625b48e2ac1d450de38cd107c4; canonical plan digest 97a0f17c0f8f3595b5d3b38f6cab2be2f1fa7eae1d6f89c87c5725a8b96cb1da)
• Aggregate: .tmp/verification/issue-199-engineer/verification-report.json (file SHA-256 2a517fd9ef078485a716db3798895d42bf68afbb00fa7dfe52b5f7b4b6b8ec4e)
• Engineer evidence validation and engineer report validation both passed with make verification-evidence-check ... VERIFY_CONSUMER=engineer and make verification-report-check ... VERIFY_CONSUMER=engineer VERIFY_PHASE=engineer.

Tester recomputation:

• Plan: .tmp/verification/issue-199-tester/verification-plan.json (file SHA-256 a4074e7ca976f9ace62f6c79476007d84bf301111774968b632e319b9409f270; canonical plan digest 26460d75f83317f7c05347fea9202cbb439ccda203c3b3a57617692499e96d1d)
• Aggregate: .tmp/verification/issue-199-tester/verification-report.json (file SHA-256 25e51604f9e50d81aabcf8bd60bf5d0f782656443a891eea8d92f7923d30609c)
• Selection SHA-256 was identical for engineer/tester: b65c5131cc4f6911e2ac80d4742407aaa0a5bc8a3e922f24700b6736b8e7be6a.
• Graph digest: 4127118b4467a75b40b8dbadc4050b5b2dc3762b292aca93cd9309a4593bca19; policy v4; source manifest d755cf343a2611846e61d29506e6a08ad3ab62f1b411fcaa911ecdce4087724f; tree OID 2eabd4c45902aad0071c8b97bc3375a036552cab; verification state 7dee52ce447b682d26ad7004d935934c3a7e7961bb485b31ccf5feb85abd0b61.
• Deep comparison found no source, graph, policy, relevant-input, environment, selection, disposition, or render drift. The only engineer/tester plan difference is the expected created_at timestamp (engineer 2026-08-19T14:19:20Z, tester 2026-08-19T15:31:57Z), which changes the canonical plan digest.
• Render graph: impact=false, required_captures=[], render digest c018a20d3e15a6ea2bd986113eda7a3d783445a4696afb5edc9ea872aa7c5a5f.

Component input/environment digests from the tester plan:

Component	Disposition	Input SHA-256	Environment SHA-256
compatibility	not_applicable	39f089a901bc30846adc8affebc99c10039320702a48449f0f71f90efe196c7a	62fa21213247d65c01dfa68fc441baab2da54c2726913ff61d3a157c5e0326b5
container	rerun	eee8bd0f987a6796cbfc80092d0d4b14f7840a0d28957224ebf885efb0f9f93a	62fa21213247d65c01dfa68fc441baab2da54c2726913ff61d3a157c5e0326b5
content_invariants	not_applicable	08b918bc9514cb82e0a1f0ec03752798621eadbac1a498828b95e4801430682f	62fa21213247d65c01dfa68fc441baab2da54c2726913ff61d3a157c5e0326b5
django	rerun	120c7cd289d36d45905d98acd20d6390e02b6cd6a99a3e24fd09c4b5c7b218ad	0dfb8efeb2284ae35de13fdec79b63ec138fb834e7577a3d5e4ff0bd4aebc2d7
evidence_validation	rerun	0c7b672a5240d6730e68f478d0ca5dcd6f62ff1a0d43598aac6dff0846a03e8a	62fa21213247d65c01dfa68fc441baab2da54c2726913ff61d3a157c5e0326b5
playwright	rerun	557491364cf6615b095fd203e8ac690be5bc8e947fcab7607696099e19e5d192	96d89e56cd62ab21e1933895cd520122098ebad5efd7c9ab700a4a74b97f9222
quality	rerun	2ecc061de4c34fd1eefa1c552b633e500f84088e047da9eb922a78809acb4cc9	62fa21213247d65c01dfa68fc441baab2da54c2726913ff61d3a157c5e0326b5
screenshots	not_applicable	2d76a86aa30f321b557cb976cc02b135daf66eba59e90e64a2cc31bf02475f94	96d89e56cd62ab21e1933895cd520122098ebad5efd7c9ab700a4a74b97f9222
selector	rerun	0c7b672a5240d6730e68f478d0ca5dcd6f62ff1a0d43598aac6dff0846a03e8a	62fa21213247d65c01dfa68fc441baab2da54c2726913ff61d3a157c5e0326b5

Fresh tester commands and evidence

Focused issue regression:

• uv run --frozen pytest tests_ci/test_runner.py tests_ci/test_runner_timeout.py tests_ci/test_make_verification.py -q → 16 passed in 16.85s; log .tmp/verification/issue-199-tester/focused-tests.log.

Fresh planned run (exact tester variables):

make verification-run VERIFY_BASE_SHA=b487b1fc44372d83f35a2edd09457e6535ad189a VERIFY_HEAD_SHA=b487b1fc44372d83f35a2edd09457e6535ad189a VERIFY_PLAN=.tmp/verification/issue-199-tester/verification-plan.json VERIFY_OUTPUT_DIR=.tmp/verification/issue-199-tester VERIFY_EVIDENCE_DIR=.tmp/verification/issue-199-tester/evidence VERIFY_REPORT=.tmp/verification/issue-199-tester/verification-report.json VERIFY_CONSUMER=tester VERIFY_PHASE=tester VERIFY_ISSUE=199 VERIFY_WORKTREE=issue-199-runner-component-timeout VERIFY_PRODUCER_ROLE=tester

The run returned Make status 2 because the runner returned failure; its report-check stage still completed successfully and emitted the aggregate. The final tester evidence is under .tmp/verification/issue-199-tester/evidence/. All 18 recorded component artifacts (result, environment, and output artifacts) independently matched their envelope size/SHA-256 digests. Every envelope origin was {"issue":199,"kind":"local","producer_role":"tester","worktree":"issue-199-runner-component-timeout"} and every expiry was in the future at validation time.

Component	Exact command / result	Counts	Evidence ID / expiry	Output artifact SHA-256 (size)
selector	make verification-plan / success, exit 0	1 assertion	9dc509216c9fd740f526f89de819256546a0a9d2e51d77080b3b93c90bbd3f23 / 2026-08-26T15:35:12Z	selector-plan.json: a4074e7ca976f9ace62f6c79476007d84bf301111774968b632e319b9409f270 (1,639,608)
evidence_validation	make test-ci / success, exit 0	540 tests: 540 passed, 0 failed, 0 skipped	2e2b3e7fff6663c9d9770cae922c1d6530900b29976250795bdfd3ea7a7f4da9 / 2026-08-26T15:35:44Z	evidence_validation-output.log: 1be760b2589912eb1cdd13ea8cd694b0ab70581ec0b8a2809d1f0c26f2f85667 (874)
quality	uv run --frozen python -m ci.quality_contract --repository . / failure, exit 2	stopped at advisory; 0 tests recorded	294b8689cc48854ac41e8ac8914b146f2057709d37b4c0de92a3d9f881237f44 / 2026-08-26T15:35:47Z	quality-output.log: 1f1819b1254c1e6102b1c0d8ebd92b74e2b6e6ea496816910c3938d40705739d (1,518)
django	make test / success, exit 0	2,558 assertions: 2,549 passed, 9 skipped, 0 failed (compatibility stage 305 passed; Django stage 2,253 OK)	e531fdc2c45da52fd806fed41089597c99cf2ce8eca6ab9e70928d0955d4cf6b / 2026-08-26T15:39:26Z	django-output.log: 4e2a2d38eadec4ad01d62490630d92a5c5914da1dac42a0f8ab7bef7efab0076 (22,616)
playwright	make test-playwright-core / timed_out, exit 124 at the 3,600s bound	220 collected, 28 deselected, 192 selected; before the hang: 14 failed + 1 error; terminal test was mobile accessibility smoke at 47%; no final pytest count	3e14412daa1ae4bbe6c4d5627a86b60f9b08814909257022fa2ead48c3f51ab1 / 2026-08-20T16:39:26Z	playwright-output.log: ce4eca71cb3198072c7ae5fc9027a0124a27a97b08feee8e74ccbf4394463dee (13,552)
container	make verification-container / success, exit 0	13 assertions, 0 failed	8ad96cd7035d0c6b8505516a5af51779e329a04e260ec66f778851f3e0a0a237 / 2026-08-20T16:40:04Z	container-check.json: 8a316c915e06a1a1207c9910088b8f43eb47841f7789d60bce912e9dbe6ff453 (614)

Additional quality commands:

• make lint format-check typecheck migrations-check → exit 0: Ruff All checks passed!; format 559 files already formatted; mypy Success: no issues found in 416 source files; makemigrations --check --dry-run No changes detected; log .tmp/verification/issue-199-tester/quality-followup.log.
• uv run --frozen --with 'pip-audit>=2.9,<3' pip-audit --format json --progress-spinner off --desc off → exit 1; raw bounded output .tmp/verification/issue-199-tester/pip-audit.json.
• Final make verification-evidence-check VERIFY_PLAN=.tmp/verification/issue-199-tester/verification-plan.json VERIFY_EVIDENCE_DIR=.tmp/verification/issue-199-tester/evidence VERIFY_CONSUMER=tester → exit 0.
• Final make verification-report-check VERIFY_PLAN=.tmp/verification/issue-199-tester/verification-plan.json VERIFY_EVIDENCE_DIR=.tmp/verification/issue-199-tester/evidence VERIFY_REPORT=.tmp/verification/issue-199-tester/verification-report.json VERIFY_CONSUMER=tester VERIFY_PHASE=tester → exit 0.

Four-bucket tester-final report

.tmp/verification/issue-199-tester/verification-report.json has phase=tester, profile=full, verdict=failure, invalid_evidence=[], and classifies every component exactly once:

• rerun (6): selector success; evidence_validation success; quality failure; django success; playwright timed_out; container success.
• reused (0).
• skipped (0).
• not_applicable (3): compatibility (outside_impact_closure), content_invariants (outside_large_content_closure), and screenshots (render_inputs_unchanged_without_reusable_visual).

Screenshots are explicitly not_applicable, supported by the graph’s render=false and empty required_captures; this is an internal CI/process change with no product-page render impact.

Findings and acceptance criteria

1. Verified — every component has an explicit 3,600-second default bound, with documented direct CLI override; the implementation and focused tests cover timeout validation.
2. Verified — the fresh timed-out Playwright run retained partial output, recorded timed_out/exit 124 and the exact make test-playwright-core command, and the process-group terminate/kill regression tests passed.
3. Verified — after the timeout, the container component executed and the final .tmp/verification/issue-199-tester/verification-report.json was emitted with verdict=failure, with no skipped component.
4. Verified — the focused timeout/Makefile tests passed 16/16, including the synthetic hang/aggregate and normal green-run regressions.
5. FAIL — the required aggregate quality gate is not green. quality-contract-v1 stopped at one real advisory: Django 6.0.7, PYSEC-2026-3717 / CVE-2026-15830, fixed in 6.0.8. The frozen uv.lock SHA is 839a35927dfe09a74c0217baa6cf89f7a60e61197b17c265d499f8ac2d70e6e0, and Bound component execution in the verification runner so a hang still emits the aggregate report #199 changes neither pyproject.toml nor uv.lock; this is pre-existing, not introduced by Bound component execution in the verification runner so a hang still emits the aggregate report #199. The separately run lint/format/type/migration checks are green but do not waive the failed quality contract.
6. Verified — screenshots are graph-derived not_applicable for this non-rendering CI/process diff.

Findings against the diff:

• Pre-existing browser/environment failure, still blocking: the fresh tester log .tmp/verification/issue-199-tester/evidence/playwright-output.log reproduces the same 14 deterministic local Chromium failures and mobile accessibility driver deadlock documented by the independent clean-base A/B review in issue #198. Bound component execution in the verification runner so a hang still emits the aggregate report #199 changes no playwright_tests/**, template, static, JavaScript, fixture, dependency, or browser input. This establishes the 3,600-second Playwright timeout as pre-existing; it is still a required component failure and is not waived.
• Pre-existing dependency advisory, still blocking: .tmp/verification/issue-199-tester/pip-audit.json identifies only Django 6.0.7 / CVE-2026-15830 with fix 6.0.8. The lock and dependency declarations are unchanged by this diff.
• No implementation defect was found in the reviewed Bound component execution in the verification runner so a hang still emits the aggregate report #199 diff itself; the tester gate nevertheless remains FAIL because required command failures are failures under _docs/PROCESS.md, even when pre-existing.

Overall tester verdict: FAIL. Return to engineering for a clean required-gate result (or explicit remediation/decision on the pre-existing blockers); do not proceed to PM acceptance or commit/merge.

[alexeygrigorev]

QA Review

Verdict: PASS — independent tester verification of issue #199 (bound component execution in the verification runner).

Frozen state and applied diff

• Engineer worktree observed: .tmp/website-199-engineer, branch issue-199-runner-component-timeout, HEAD b487b1fc44372d83f35a2edd09457e6535ad189a, 6 uncommitted files (5 modified + 1 new), read-only throughout.
• My pristine base: git worktree add .tmp/tester-199-base --detach b487b1f; frozen patch generated read-only (git diff for tracked files + git diff --no-index /dev/null for the new file) and applied cleanly.
• Applied-diff sha256: 4442e3d2c5e0e682c92733f302d18133e0e2f9a917b61f6709b8532b18d075ea; all 6 files byte-compared (cmp) against the engineer tree — identical. Exactly the 6 claimed files: ci/runner.py, tests_ci/test_runner_timeout.py (new), tests_ci/test_runner.py, Makefile, tests_ci/test_make_verification.py, _docs/ci/change-selective-ci.md.

Structural review

ci/runner.py implements real session-group semantics, not a bare subprocess.run(timeout=…): Popen(..., start_new_session=True) per component, combined output streamed directly into the component's output artifact (partial output survives expiry), and on expiry _terminate_process_group SIGTERMs the whole group via os.killpg, honors a 10 s grace, escalates to SIGKILL, then polls the group to extinction before the evidence digests freeze (an uninterruptible straggler cannot reintroduce the hang). run_plan validates the bound (rejects 0/NaN/inf), records result: "timed_out", timed_out: true, exit 124, and the exact command, and keeps iterating RUN_ORDER so later components still run; --component-timeout-seconds is the diagnostic override and DEFAULT_COMPONENT_TIMEOUT_SECONDS = 3600.0 is pinned by test to the documented value. The Makefile verification-run captures the runner status, still runs verification-report-check, and propagates the runner's nonzero exit. Tests cover SIGTERM partial-output retention, SIGTERM-ignoring groups killed, grandchild straggler drained before return, status/exit mapping, non-positive/non-finite rejection, the end-to-end hang→bounded→report-emitted path, and an unaffected green run.

Nit (non-blocking): the handoff calls the new test file "9 tests"; it contains 8. Counts elsewhere are exact (make test-ci 540).

Unit and quality gates (my worktree)

• make test-ci — 540 passed in 30.50s (matches the handoff exactly).
• uv run --frozen pytest tests_ci/test_runner_timeout.py tests_ci/test_runner.py -q — 13 passed (8 + 5).
• make lint / make format-check / make typecheck — green (416 source files clean, 559 formatted).

Plan recompute and digest comparison

make verification-plan VERIFY_ISSUE=199 VERIFY_CONSUMER=tester in my worktree: graph digest 4127118b4467a75b40b8dbadc4050b5b2dc3762b292aca93cd9309a4593bca19 and verification-state digest 7dee52ce447b682d26ad7004d935934c3a7e7961bb485b31ccf5feb85abd0b61 both identical to the handoff. The plan-file digest differs only because created_at differs — a recursive field-diff of my plan against the engineer's frozen plan shows created_at as the sole difference. Dispositions identical: rerun 6 (selector, evidence_validation, quality, django, playwright, container), reused 0, skipped 0, not_applicable 3 (compatibility outside_impact_closure, content_invariants outside_large_content_closure, screenshots render_inputs_unchanged_without_reusable_visual).

Live bound reproduction — small override (independent)

Runner invoked directly in my worktree with --component-timeout-seconds 30 and a scratch PYTEST_ADDOPTS plugin hanging only the evidence_validation pytest invocation (.tmp/hangproof-tester/hang_proof_199.py, scratch only):

• evidence_validation bounded at exactly the override: 15:45:54Z → 15:46:24Z (30 s), result timed_out, exit 124, envelope carries the exact command make test-ci plus SHA-256 digests of the retained partial log.
• Partial log retains the plugin's own line and make's Error 143 (SIGTERM death of the group).
• Every remaining component still executed in sequence (quality completed failure; django/playwright/container each bounded at the same 30 s override — expected, they legitimately exceed 30 s), the runner exited 1, and make verification-report-check ... still emitted and validated the aggregate with verdict failure and an empty skipped bucket.
• No stray processes survived: the group drain reaped everything (verified via /proc cwd scan).

Live bound reproduction — default 3600 s on the real wedge (independent, official run)

make verification-run VERIFY_WORKTREE=tester-199-base VERIFY_ISSUE=199 VERIFY_CONSUMER=tester VERIFY_PRODUCER_ROLE=tester VERIFY_PHASE=tester in my worktree (explicit issue per #197):

• selector success; evidence_validation success 540/540; quality failure exit 2 (environmental, below); django success 2549 passed / 9 skipped / 2558 in 180 s (matches the engineer's counts exactly).
• playwright wedged at the known Re-pin the refreshed homepage subtitle through the deploy contract and fail drift in Django CI #198 signature test_accessibility.py::test_accessibility_core_smoke[chromium-viewport1-mobile] (log frozen 13,476 bytes for ~50 minutes) and was bounded at the default: 16:56:31Z → 17:56:32Z (3601 s), result timed_out, exit 124, 13,537-byte partial log retained including make's teardown lines.
• container still executed after the timeout and succeeded with 13 assertions (51 s) — matching the engineer's run shape.
• The aggregate report was emitted with verdict failure and validated: validate-report (tester phase, no pending) exit 0, validate-evidence-directory --consumer tester exit 0, every envelope artifact digest re-hashed and matched. make verification-run itself exited 1 — nonzero status propagated while the report-check still ran, exactly as claimed.

My tester-final four buckets: rerun 6 (selector success, evidence_validation success 540/540, quality failure, django success 2549/9/2558, playwright timed_out, container success 13 assertions), reused 0, skipped 0, not_applicable 3. No required skip; no pending screenshot evidence.

Environmental red attribution (quality)

My quality component failed in 5 s with VulnerabilityScanError: dependency advisory scan found 1 finding(s). uv.lock at this base pins django 6.0.7; issue #201 ("Bump the locked Django to 6.0.8 for PYSEC-2026-3717 (advisory-DB drift is blocking every landing)") owns exactly this advisory, and its lane proves zero findings at 6.0.8. The feed flaps live: my run failed at 16:53Z while three direct scans of the same lock at 15:54Z returned zero findings across 81 deps — the same flapping the engineer documented (clean 11:10Z/12:01Z/13:21Z, red 14:20:40Z). No dependency file is touched by this diff, so this red is #201's, not #199's. terminology-check and database-portability-check passed inside the same component before security-check failed.

Flake attribution (disclosed test_studio_foundation failures)

Ran the focused file standalone at a pristine base b487b1f without the diff applied (separate worktree): 3 failed, 2 passed in 28.70 s — test_audit_filter_detail_redaction_and_layout[chromium-viewport0], [chromium-viewport1], and test_revocation_prevents_back_cache_and_reload[chromium], failing on the horizontal-overflow assertion (scrollWidth <= innerWidth + 1, line 147). The same test ids appear in my run's partial playwright log and in the engineer's. So they are pre-existing on the base commit itself (local chromium geometry-drift class, #193/#198 follow-up territory per this issue's non-goals) — not #199's, and not even worktree-local noise.

The playwright wedge itself is likewise pre-existing at the clean base (engineer's stash A/B in the issue body), and an ~18-hour wedged main-checkout run of the same module was observable in the process table throughout my verification.

Reused provenance

Nothing was reused; every component was rerun in my own worktree. Cross-checks of the engineer's posted artifacts were read-only: their official-run envelopes (playwright 14:24:14Z → 15:24:14Z, exactly 3600 s, exit 124, 13,628-byte partial log; container success afterwards) and their 480 s hang-proof (only playwright timed out, all other components green, verdict failure, empty skipped bucket) both re-hash clean, and their plan/report validate in the engineer lane. For --consumer tester their local evidence is correctly rejected with untrusted_origin (producer_role engineer) — the #197 fail-closed origin rule working as designed, which is why my own tester-produced run above is the provenance of record.

Screenshots

not_applicable — internal CI/process change, no product-page render impact; the plan's screenshots disposition (render_inputs_unchanged_without_reusable_visual) agrees.

Anomaly disclosures

1. Unknown operator inside the engineer worktree: a tester-phase verification-run not belonging to the engineer, this verification, or the orchestrator was executing inside .tmp/website-199-engineer (output under .tmp/verification/issue-199-tester/, started ~15:35Z; runner and playwright processes confirmed via /proc cwd). I did not rely on, consume, or interfere with it, and none of its envelopes are evidence here.
2. Concurrent-edit disclosure accepted: the owner-session Makefile/test_make pair and hardening touches the engineer preserved were verified as part of the current frozen state (the applied diff above includes them).
3. Infra kill event during my verification: at ~16:51:33Z my first background verification-run was externally terminated 3 minutes before its in-process bound would have fired (16:54:14Z), and the same instant killed the unknown operator's run in the engineer worktree and the Bump the locked Django to 6.0.8 for PYSEC-2026-3717 (advisory-DB drift is blocking every landing) #201 tester lane's run, while the owner's ~18 h main-checkout wedge survived — i.e. an infrastructure-level termination sweeping active verification runs, not a defect of this change (an externally killed runner is this issue's documented non-goal; the in-process timeout remains the guaranteed path). I restarted my run fully detached and it completed end-to-end as reported above.

Artifacts (my worktree .tmp/tester-199-base/)

• Plan: .tmp/verification/verification-plan.json (state digest 7dee52ce…)
• Report: .tmp/verification/verification-report.json (verdict failure, validated)
• Evidence: .tmp/verification/evidence/ (all digests verified)
• 30 s override proof: .tmp/verification/hangproof-tester/ + .tmp/hangproof-tester/runner-invocation.log
• Full run log: .tmp/verification-run-tester2.log (bound message at line 487)

[alexeygrigorev]

Product Acceptance

Verdict: ACCEPT.

Accepted identity: the frozen uncommitted diff in .tmp/website-199-engineer (base = head = b487b1fc44372d83f35a2edd09457e6535ad189a, exactly the 6 claimed paths, git diff --check clean), sha256 4442e3d2c5e0e682c92733f302d18133e0e2f9a917b61f6709b8532b18d075ea. I reproduced that sha myself from the live worktree and it matches the tester's posted digest byte-for-byte.

Acceptance criteria — all six verified against live evidence

1. Explicit bound with documented default and override — verified. DEFAULT_COMPONENT_TIMEOUT_SECONDS = 3600.0 in ci/runner.py, pinned by test assertion and documented in _docs/ci/change-selective-ci.md; --component-timeout-seconds is the CLI override, and the doc states the Makefile target always uses the default. No component command can block the run indefinitely.
2. Terminate-then-kill, partial output, truthful envelope — verified. Popen(start_new_session=True) with output streamed directly into the component artifact; on expiry SIGTERM to the whole group, grace window, SIGKILL, then a group drain before evidence digests freeze. The tester-final playwright envelope records result: timed_out, timed_out: true, exit 124, and the exact command make test-playwright-core, with a 13,537-byte partial log retaining the wedge and make's Terminated line.
3. Run completes and the aggregate is emitted — verified. In the official default-bound run the container component executed after the timeout and succeeded; the final verification-report.json (phase tester, profile full) has verdict failure, invalid_evidence: [], and an empty skipped bucket; make verification-run propagated Error 1 while verification-report-check still ran.
4. Regression tests — verified. Eight tests in tests_ci/test_runner_timeout.py cover partial-output retention, SIGTERM-ignoring groups killed, straggler drain, status mapping, non-positive/non-finite rejection, the end-to-end hang→bounded→failure-report path, and an unaffected green run. I re-ran the focused set in the applied-diff worktree myself: 13 passed. (The handoff's "9 tests" for the new file is a count typo — it has 8; non-blocking.)
5. Docs and gates — verified. The doc section records the default, the override, the terminate/kill and partial-output semantics, and the guaranteed failure-verdict report. Tester-final: make test-ci 540 passed, lint/format-check/typecheck green, plan and report validated exit 0.
6. Screenshots not_applicable — appropriate. Internal CI/process change with no product-page render impact; the graph-derived disposition (render_inputs_unchanged_without_reusable_visual) agrees. Unrelated product screenshots would not be evidence here.

Four-bucket classification

Every component is classified exactly once: rerun 6 (selector success, evidence_validation success 540/540, quality failure, django success 2549/9/2558 — both counts reconcile from the retained logs: 305 + "Ran 2253 tests … OK (skipped=9)", playwright timed_out, container success 13 assertions), reused 0 (nothing reused, so no reuse justifications are due), skipped 0, not_applicable 3 (compatibility outside_impact_closure, content_invariants outside_large_content_closure, screenshots render_inputs_unchanged_without_reusable_visual — each with an explicit reason). No required skip, no pending screenshot evidence.

Environmental reds — correctly attributed, not this diff's

• quality: VulnerabilityScanError on the live advisory feed for django 6.0.7 / PYSEC-2026-3717. Bump the locked Django to 6.0.8 for PYSEC-2026-3717 (advisory-DB drift is blocking every landing) #201 owns exactly this and is closed; I confirmed origin/main now pins django 6.0.8, and this diff touches no dependency file. Post-merge runs take main's lock, so this red is already remediated upstream of landing.
• playwright: the mobile-accessibility wedge (bounded at exactly the 3600s default, 16:56:31Z→17:56:32Z) and the test_studio_foundation failures are pre-existing at pristine base b487b1f without the diff (tester's standalone run: 3 failed, 2 passed, same test ids). Fixing them is this issue's explicit non-goal (Main Playwright suite is red: design-5a batch landed without a browser verdict #193/Re-pin the refreshed homepage subtitle through the deploy contract and fail drift in Django CI #198 territory). The change did its job: an indefinite wedge became a bounded, reported, truthful failure.

Disclosures weighed

• Unknown operator inside the engineer worktree: observed and disclosed; none of its envelopes were consumed as evidence, and the tester-final provenance of record is the tester's own pristine-base worktree. The frozen 6-file identity I re-verified today is intact — no contamination of the diff under acceptance. (The .tmp/verification/issue-199-tester/ output directory is no longer present; no PASS evidence derives from it.)
• Concurrent owner edits: preserved by the engineer and included inside the frozen diff (Makefile status capture, non-finite-bound validation, two tests, doc sentence) — covered by the gates and the 540 count.
• Infra kill event (~16:51Z): externally killed runners are this issue's documented non-goal (no signal-handler partial-report emission; the in-process timeout is the guaranteed path). The tester restarted detached and completed end-to-end; the accepted evidence is from that completed run.

Operator perspective

The bound message is actionable ("component playwright exceeded its 3600s wall-clock bound; terminated the process group and recording a timed_out result"), the Make exit status stays truthful while the aggregate is still produced, a timed-out component reports zero counts rather than fabricated ones, and the doc makes clear the override is diagnostic-only. This is exactly the behavior the issue demanded: the harness now always produces a truthful machine-readable aggregate.

The engineer may now commit the frozen tree; Closes #199.