feat(harness): integrate safe feedback and diagnostics - #101
Merged
Conversation
* test(harness): bound review campaign closure * test(monitor): gate timeout on child readiness
* test(orchestration): expose stop uncertainty * fix(orchestration): retain unconfirmed stops Protocol-v2 stop acknowledgement does not prove quiescence or safe replay. Keep unresolved worker identities non-consumable and non-retryable, retain cancelled controllers, and migrate legacy interruption records conservatively. * test(orchestration): expose early stop overwrite * fix(orchestration): preserve early stop evidence Early stopped/aborted evidence must remain uncertain when later lifecycle messages arrive before spawn binding. Otherwise failure settlement can silently authorize another attempt. * test(orchestration): cover terminal consume recovery
* test(workflow): specify graph warning witnesses * feat(workflow): expose graph convergence warnings
* test(ui): separate ownership from execution * fix(ui): distinguish claims from execution
* test(harness): expose policy validation gaps * fix(harness): validate feasible review policies * test(ui): expose rejected cancellation severity * fix(ui): mark rejected cancellation as error * test(workflow): expose unrelated route witnesses * fix(workflow): narrow exhausted-route witnesses
There was a problem hiding this comment.
🔵 Needs a closer look
It changes core orchestration runtime/reducer/store deletion semantics and introduces a new public API surface, which warrants final human validation despite strong test coverage.
Pull request overview
Promotes the previously-reviewed harness feedback + diagnostics stack onto master, adding warning-only workflow-graph convergence diagnostics, tightening orchestration cancellation semantics around “unconfirmed termination”, and introducing a traceable harness boundary inventory + review-campaign coverage fixtures.
Changes:
- Add workflow graph diagnostics (
diagnoseWorkflowGraph) and surface warnings in workflow UI/tool summaries. - Refine orchestration status/presentation to distinguish local capacity reservations from provider-reported execution, and retain cancelled controllers when termination is unconfirmed.
- Add harness review-campaign policy fixtures + boundary inventory manifest, with accompanying unit/property/e2e regression coverage.
File summaries
| File | Description |
|---|---|
| test/workflow-presentation.test.ts | Updates workflow activity expectations (claimed/waiting). |
| test/workflow-graph.test.ts | Adds unit coverage for workflow graph warnings and formatting bounds. |
| test/widget.test.ts | Updates widget status text for claimed workflows and orchestration status/progress labels. |
| test/review-campaign.test.ts | Adds regression tests for bounded review campaign evaluation semantics. |
| test/property/workflow-graph.property.test.ts | Adds property test oracle for closed-component detection stability. |
| test/property/orchestration.property.test.ts | Adds property test ensuring cancelled work isn’t reopened/consumed under event permutations. |
| test/orchestration-tools.test.ts | Updates orchestration tool output expectations (active/reserved/pending). |
| test/orchestration-stop-safety.test.ts | Adds file-backed and migration coverage for stop/uncertainty retention. |
| test/orchestration-runtime.test.ts | Expands runtime tests for stop acknowledgement uncertainty + legacy consume reconciliation. |
| test/orchestration-presentation.test.ts | Updates orchestration presentation expectations and adds dispatch-observation labeling test. |
| test/notification-runtime.test.ts | Updates notification text assertions for new orchestration progress wording. |
| test/monitor-manager.test.ts | Makes the timeout test deterministic by gating timeout firing on child readiness. |
| test/loop-tools.test.ts | Updates loop tool cancellation contract and message expectations. |
| test/loop-command.test.ts | Updates command behavior/messages to honestly report orchestration retention/rejection. |
| test/index.test.ts | Updates integration wiring expectations for orchestration progress strings. |
| test/helpers/review-campaign.ts | Introduces deterministic review-campaign reference evaluator used by harness fixtures. |
| test/harness-boundaries.test.ts | Adds manifest validation for unique boundary IDs and referenced regressions. |
| test/fixtures/harness-boundaries.json | Adds traceable boundary inventory entries mapped to regression titles. |
| test/e2e/orchestration-conformance.mjs | Updates e2e conformance to expect retention of cancellation uncertainty. |
| src/workflow-graph.ts | Implements workflow convergence diagnostics + bounded formatting helper. |
| src/ui/workflow-presentation.ts | Renames workflow activity states and injects graph warnings into displays/inspection. |
| src/ui/orchestration-presentation.ts | Updates orchestration labels and adds cancellation messaging helper. |
| src/tools/workflow-tools.ts | Includes graph warning lines in workflow tool summaries. |
| src/tools/subagent-orchestration-tools.ts | Updates orchestration create tool details text (active/progress labels). |
| src/tools/loop-tools.ts | Changes orchestration cancellation hook to return a typed result + renders message/tone accordingly. |
| src/store.ts | Normalizes legacy orchestration stop states on load; blocks deletion with unconfirmed dispatches. |
| src/runtime/subagent-orchestration-runtime.ts | Changes stop handling to request stop without treating acknowledgement as quiescence; returns richer cancellation outcomes. |
| src/orchestration-reducer.ts | Adds stop normalization helpers; adds cleanup binding event; blocks dispatch/consume in unconfirmed-termination cases. |
| src/commands/loop-command.ts | Uses typed orchestration cancellation results and reports retention/rejection accurately in UI flows. |
| src/api.ts | Exports diagnoseWorkflowGraph / WorkflowGraphWarning as public API surface. |
| scripts/package-smoke.mjs | Verifies packaged API exports graph diagnostics and basic behavior. |
| package.json | Adds HARNESS_REVIEW_CAMPAIGNS doc to packaged file list. |
| docs/USAGE_GUIDE.md | Updates workflow/orchestration terminology and cancellation/uncertainty semantics. |
| docs/TESTING.md | Documents harness review-campaign fixtures and boundary inventory. |
| docs/REFERENCE.md | Updates reference semantics and documents graph warnings API. |
| docs/HARNESS_REVIEW_CAMPAIGNS.md | Adds dedicated documentation for bounded harness review campaigns and inventory intent. |
Review details
- Files reviewed: 36/36 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if (result === "deleted") return `Orchestration #${id} cancelled and deleted`; | ||
| if (result === "paused") return `Orchestration #${id} cancelled and paused`; | ||
| if (result === "retained") return `Orchestration #${id} cancellation recorded; retained and paused because worker termination is unconfirmed. No automatic retry or consume. Inspect OrchestrationGet; protocol v2 cannot prove safe cleanup.`; | ||
| if (result === "context_changed") return `Orchestration #${id} session changed during cancellation; inspect the original session before retrying. Deletion is not confirmed.`; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Promote the reviewed harness-feedback stack (#95–#98) and comment fixes (#100) from its integration branch to master before the next release.
Provenance and validation
#95–#98 previously merged onto
fix/controller-boundary-authority, not master. #100 merged there as 5fe7dd9 after Windows and Ubuntu CI passed. Master's tree exactly matched the original integration base fbb512c, so only the five reviewed commits were rebased with valid signatures onto master.The resulting tree is byte-identical to the validated follow-up head be506a6: 1042 tests, 18 properties, 96.52% function coverage, lint/typecheck/build/package/audit passed. No new implementation changes. This PR gets its own exact-head CI before merge.
User authorized CI-gated integration and a new release. Release-please follows only after this full stack reaches master. Provider quiescence/replay guarantees, scheduler election, and durable result receipts remain out of scope.