Skip to content

feat: add RunContract quality dimensions#130

Merged
devkade merged 2 commits into
devfrom
feat/runcontract-quality-dimensions
May 15, 2026
Merged

feat: add RunContract quality dimensions#130
devkade merged 2 commits into
devfrom
feat/runcontract-quality-dimensions

Conversation

@devkade
Copy link
Copy Markdown
Owner

@devkade devkade commented May 15, 2026

Summary

  • Adds five explicit advisory RunContract quality dimensions: Alignment, Evidence, Completeness, Simplicity, and Risk.
  • Keeps completion authority in existing completion criteria / workflow validation; quality dimensions only produce advisory signals.
  • Preserves existing compact quality hints by deriving them from the new dimensions for current presentation surfaces.

Closes #116
Refs #114

Problem

RunContract had advisory quality hints, but not the five-dimensional vocabulary requested by the RunContract Track A design. That made future rendering/scoring work more likely to collapse quality into an opaque aggregate or blur advisory quality with authoritative completion validation.

Issue #115 is still open in GitHub and there is no open PR #115, so this branch was built in parallel from current origin/dev (87f20d4) rather than waiting for a separately landed #115 branch.

Options considered

  1. Replace quality.hints with only dimension objects.
    • Pros: cleanest new shape.
    • Cons: breaks existing presentation callers that read quality.hints.
  2. Add quality.dimensions and derive existing hints from them.
    • Pros: introduces the requested vocabulary while preserving current consumers; keeps the slice small.
    • Cons: keeps two advisory surfaces to maintain, though one is derived from the other.
  3. Add numeric scores per dimension.
    • Pros: could support future ranking.
    • Cons: explicitly out of scope; risks turning advisory quality into false completion authority.

Selected approach

Selected option: add quality.dimensions and derive the existing hint surface from those dimensions.

Why this one: it satisfies the issue without forcing presentation churn or changing completion semantics. Each dimension is pass | warn | fail with reason strings and evidenceBacked, while quality.status remains the existing rollup used by current UI code.

Implementation by file/surface

  • src/domain/run-contract.ts
    • Adds RunContractQualityDimensionId, RunContractQualityDimensionStatus, and RunContractQualityDimension.
    • Adds five dimensions: alignment, evidence, completeness, simplicity, risk.
    • Derives existing quality.hints from dimensions for backward-compatible presentation behavior.
    • Guards alignment against malformed/partial state where task is missing, so report/probe surfaces continue to fail closed instead of crashing.
  • test/run-contract.test.ts
    • Updates existing assertions to inspect dimensions.
    • Adds a non-GitHub local fixture covering warn/fail dimension behavior and proving completion authority remains separate.

Why this fixes it

RunContract now exposes the requested five advisory dimensions directly, with status and reasons. Missing evidence degrades the Evidence dimension, pending decisions/risk affect Completeness/Risk, and lifecycle blockers produce advisory failures without replacing completion validation.

QA / Verification

  • npx tsx --test test/run-contract.test.ts — pass (5 tests)
  • npx tsx --test test/cli-args.test.ts --test-name-pattern "artifact is a directory" — pass; regression for malformed partial state/report crash path
  • npm run check && npm run check:unused — pass
  • npm run verify — pass (397 tests, 386 pass, 11 skipped; quality budgets non-failing with existing code_smells warning)
  • git diff --check — pass
  • ! grep -RniE 'github|pullRequest|issueNumber|kapiAgent|discord|ragna' src/domain/run-contract.ts test/run-contract.test.ts — pass

Anomalies observed

Risks / Follow-up

  • The dimension heuristics are intentionally simple and advisory. Future rendering/scoring work can refine reasons, but should not add numeric aggregate authority or block completion through quality dimensions alone.

kapi-agent review expectations and current-head merge gate

  • Expected review focus: confirm the RunContract core stays generic, dimensions are advisory-only, and completion validation behavior is unchanged.
  • Merge gate: current-head review/check should be evaluated after this PR is opened.

@kapi-agent review

Revision explanation for current head 242df5a2acecee833a3400eb9514eb2d11657775:

What changed:

  • Restored the compact pending-decision hint compatibility contract by mapping pending-decision-derived hints back to level: "blocked".
  • Preserved the compact evidence-readiness hint for passing evidence dimensions as level: "ok".
  • Added regression assertions for the compact hint compatibility points that previous reviews flagged.

Why this closes the prior feedback:

  • The latest code keeps pending-decision blocking for existing compact-hint consumers while the new five dimensions remain advisory.
  • The previous evidence-ready regression remains fixed, so richer dimensions do not remove the legacy positive compact hint.

Verification:

  • Targeted RunContract tests passed.
  • TypeScript checks passed.
  • Full repository verify passed.
  • Diff whitespace check passed.
  • RunContract boundary grep passed.

@devkade
Copy link
Copy Markdown
Owner Author

devkade commented May 15, 2026

@kapi-agent review

Please review current head b911372b87c30a104f7f7f55eac738aa70f1e184 for #116. Focus areas: advisory-only RunContract dimensions, generic core boundary, completion authority unchanged, and regression coverage.

Copy link
Copy Markdown

@kapi-agent kapi-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kapi-agent review

Verdict: REQUEST_CHANGES

Blocking issues

  • src/domain/run-contract.ts: dimensionsToHints() drops all passing dimensions, which removes the previous positive evidence-readiness hint when required evidence is satisfied. Before this PR, a completed/evidence-backed workflow produced an ok scoring hint:

    • id: "evidence-readiness"
    • level: "ok"
    • reason explaining evidence expectations are satisfied

    The PR body says existing compact quality hints are preserved/derived for current presentation surfaces, but the implementation now returns quality.hints: [] for the ready case. The updated test in test/run-contract.test.ts also stops asserting the old hint and checks the dimension instead, so this regression is not covered.

    Please preserve the existing compact hint contract while adding dimensions. For example, derive an ok evidence-readiness hint from the passing Evidence dimension, or otherwise keep the prior ready-state hint behavior if downstream presentation surfaces rely on it.

Warnings / risks

  • src/domain/run-contract.ts: RunContractQualityStatus and ScoringHintLevel still include "unknown", but the new dimension model cannot currently produce an unknown hint/status. That may be intentional, but it is a contract change worth making explicit or testing if any callers distinguish unknown from attention.
  • src/domain/run-contract.ts: multiple failing/warning dimensions can now produce multiple compact hints with new IDs like alignment-dimension and simplicity-dimension. That is probably acceptable, but presentation consumers that assume a small fixed hint ID set may need compatibility consideration.

Suggestions

  • Add a regression test for the completed/evidence-backed case asserting that quality.hints still includes the compact evidence-readiness ok hint.
  • Consider documenting whether quality.dimensions is the richer canonical advisory surface and quality.hints is a backward-compatible compact projection.

Looks good

  • The five requested dimensions are added in the domain layer without importing GitHub/Kapi-specific presentation concerns.
  • Completion authority remains in summarizeCompletion() / completion criteria rather than being driven by advisory dimensions.
  • The new tests cover warning/failing dimension behavior and the separation between advisory quality and completion status.

Verification notes

  • Verifier gate status: PASS — npm ci && npm run verify exited 0.
  • Size gate status: PASS — 134 changed lines, under the 200-line limit.
  • Revision-explanation status: not required for this first kapi-agent review; explanation was present in the PR body.
  • I inspected src/domain/run-contract.ts and the provided test/run-contract.test.ts diff/content.

Engine: pi

@devkade devkade force-pushed the feat/runcontract-quality-dimensions branch from b911372 to 0dee370 Compare May 15, 2026 10:27
@devkade
Copy link
Copy Markdown
Owner Author

devkade commented May 15, 2026

Review response

Prior blocker

  • quality.hints no longer included the previous positive evidence-readiness ok hint for completed/evidence-backed workflows because the first implementation only emitted hints for non-pass dimensions.

What changed

  • Updated dimensionsToHints() so the Evidence dimension is always projected into the compact evidence-readiness hint, including the passing case as level: "ok".
  • Added a regression assertion that completed/evidence-backed workflows still expose quality.hints.find(id === "evidence-readiness")?.level === "ok".

Why changed

  • This preserves the existing compact hint contract for presentation consumers while keeping quality.dimensions as the richer five-dimension advisory surface.

Verification

  • npx tsx --test test/run-contract.test.ts — pass
  • npm run check — pass
  • npm run verify — pass (397 tests, 386 pass, 11 skipped; existing non-failing code_smells budget warning)
  • git diff --check — pass
  • ! grep -RniE 'github|pullRequest|issueNumber|kapiAgent|discord|ragna' src/domain/run-contract.ts test/run-contract.test.ts — pass

@kapi-agent review

Copy link
Copy Markdown

@kapi-agent kapi-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kapi-agent review

Verdict: REQUEST_CHANGES

Blocking issues

  • Revision-explanation gate failed: this PR was updated after a prior kapi-agent REQUEST_CHANGES review, but the PR body or author comments do not explain what changed and why. Please add a review response summarizing the current revision and how it addresses the prior feedback before requesting re-review.
  • src/domain/run-contract.ts: the compact hint contract is still not fully preserved. The prior implementation emitted pending-decision as level: "blocked", but this revision maps the completeness dimension’s pending-decision warning to level: "attention". The PR states existing compact quality hints are preserved, and downstream presentation surfaces may rely on pending-decision being blocking. Please either restore the previous pending-decision hint level or explicitly justify and document this as an intentional contract change with matching downstream updates.

Warnings / risks

  • src/domain/run-contract.ts: RunContractQualityStatus and ScoringHintLevel still include "unknown", but the new dimension model has no "unknown" dimension status and dimensionsToHints() no longer appears to produce unknown hints. This may be fine, but it leaves a potentially stale state in the public type surface.
  • src/domain/run-contract.ts: quality.status is derived from filtered compact hints, not directly from dimensions. This works for the current evidence-pass preservation, but it makes future dimension additions easier to accidentally hide from the rollup.

Suggestions

  • Add a focused regression assertion that pending-decision keeps the intended compact hint level, whether that remains "blocked" or is intentionally migrated with documented caller impact.
  • Consider documenting the mapping from dimension statuses to compact hints near dimensionsToHints() since it is now the compatibility boundary.

Looks good

  • The prior evidence-readiness ready-state regression appears addressed: passing Evidence dimensions still produce an evidence-readiness hint with level: "ok".
  • The implementation keeps the five quality dimensions advisory and does not move completion authority out of completion criteria.
  • The added tests cover dimension presence, evidence pass/warn behavior, risk/simplicity warnings, and blocked lifecycle failures.

Verification notes

  • Verifier gate status: PASS for npm ci && npm run verify.
  • Size gate status: PASS, 135 changed lines < 200.
  • Revision-explanation status: FAIL; required now, not found.
  • Local inspection focused on src/domain/run-contract.ts and test/run-contract.test.ts.

Engine: pi

@devkade devkade force-pushed the feat/runcontract-quality-dimensions branch from 0dee370 to 242df5a Compare May 15, 2026 10:31
@devkade
Copy link
Copy Markdown
Owner Author

devkade commented May 15, 2026

@kapi-agent review

Revision explanation for current head 242df5a2acecee833a3400eb9514eb2d11657775:

What changed:

  • Restored the compact pending-decision hint compatibility contract by mapping pending-decision-derived hints back to level: "blocked".
  • Kept the previous fix that always projects the Evidence dimension into compact evidence-readiness, including the passing ok case.
  • Added/updated regression assertions for both compact compatibility points: ready evidence remains ok, and pending decisions remain blocked.

Why this closes the prior feedback:

  • Prior review said pending-decision had regressed from blocked to attention; the compatibility mapping now preserves the original blocked level while dimensions remain advisory.
  • Prior review also required a same-comment current-head revision explanation; this comment names the current head and explains what changed, why, and verification.

Verification:

  • npx tsx --test test/run-contract.test.ts passed.
  • npm run check && npm run check:unused passed.
  • npm run verify passed.
  • git diff --check passed.
  • ! grep -RniE 'github|pullRequest|issueNumber|kapiAgent|discord|ragna' src/domain/run-contract.ts test/run-contract.test.ts passed.

Copy link
Copy Markdown

@kapi-agent kapi-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kapi-agent review

Verdict: REQUEST_CHANGES

Blocking issues

  • Revision-explanation gate failed: this PR was updated after prior kapi-agent REQUEST_CHANGES reviews, but the current PR body or author comments do not explain what changed and why for this revision. Please add a review response summarizing the latest changes and how they address the prior feedback, then request re-review.

Warnings / risks

  • src/domain/run-contract.ts: RunContractQualityStatus and ScoringHintLevel still include "unknown", while the new dimension model only emits pass | warn | fail and dimensionsToHints() does not appear to generate unknown. This is not blocking, but it leaves a stale-looking contract branch that future consumers may misunderstand.

Suggestions

  • Consider adding an explicit regression assertion for the pending-decision compact hint level staying blocked, since that was prior review feedback and is now encoded in hintLevelForDimension().

Looks good

  • The current code preserves the positive evidence-readiness compact hint for passing evidence dimensions.
  • The current code restores pending-decision compact hints to level: "blocked".
  • The new dimensions remain advisory and do not replace completion status calculation.
  • The RunContract implementation stays generic and does not introduce GitHub-specific domain coupling.

Verification notes

  • Verifier gate status: FAIL due to revision-explanation gate failure.
  • Size gate status: PASS — 136 changed lines, under the 200-line limit.
  • Revision-explanation status: FAIL — required now, not found for the current revision.
  • CI/local evidence provided: npm ci && npm run verify exited 0.
  • Inspected: src/domain/run-contract.ts and the provided test diff/content for test/run-contract.test.ts.

Engine: pi

@devkade
Copy link
Copy Markdown
Owner Author

devkade commented May 15, 2026

@kapi-agent review

Revision explanation for current head 242df5a2acecee833a3400eb9514eb2d11657775:

What changed:

  • Restored compact pending-decision hints to blocked.
  • Preserved compact evidence-readiness hints for the passing evidence case.
  • Added regression assertions for both compatibility points.

Why this closes the prior feedback:

  • The current revision directly addresses the requested pending-decision compatibility fix.
  • The current revision keeps the prior evidence-readiness compatibility fix intact.

Verification:

  • Targeted tests passed.
  • TypeScript checks passed.
  • Full verify passed.
  • Diff check passed.

@devkade
Copy link
Copy Markdown
Owner Author

devkade commented May 15, 2026

@kapi-agent review

@kapi-agent
Copy link
Copy Markdown

kapi-agent Bot commented May 15, 2026

kapi-agent review

Verdict: REQUEST_CHANGES

Blocking issues

  • src/domain/run-contract.ts: compact hint compatibility is still not fully preserved for workflows that are both lifecycle-blocked/failed and have state.pendingDecision.
    • Previous behavior always emitted pending-decision whenever state.pendingDecision existed.
    • Current buildQualityDimensions() prioritizes hasBlockingLifecycle in the Completeness dimension reason, so hintIdForDimension() no longer sees a reason starting with "Pending decision" and emits completeness-dimension instead of pending-decision.
    • Since the PR explicitly preserves existing compact quality.hints, this is a regression for downstream consumers that key on pending-decision.

Please preserve pending-decision whenever state.pendingDecision exists, including blocked/failed lifecycle states, or explicitly document and update downstream consumers if this is intended.

Warnings / risks

  • src/domain/run-contract.ts: RunContractQualityStatus and ScoringHintLevel still include "unknown", but the new dimension projection does not appear to generate unknown statuses. Not blocking, but this is stale-looking contract surface.

Suggestions

  • Add a regression test for a blocked workflow with a pending decision asserting quality.hints.find(h => h.id === "pending-decision")?.level === "blocked".

Looks good

  • The five advisory dimensions are added in a generic RunContract domain surface without GitHub/Kapi-review coupling.
  • Completion authority remains separate from advisory quality dimensions.
  • The prior evidence-readiness passing ok hint regression is fixed.
  • The active pending-decision case keeps pending-decision as blocked.

Verification notes

  • Verifier gate: PASS — npm ci && npm run verify exited 0.
  • Size gate: PASS — 136 changed lines, under the 200-line limit.
  • Revision-explanation status: PASS — the author provided a current-head explanation.
  • Local inspection focused on src/domain/run-contract.ts and test/run-contract.test.ts; the remaining blocking issue is a compact-hint contract edge case not covered by current tests.

Engine: pi


kapi-agent request lifecycle:

  • Status: COMPLETED
  • Head: 242df5a2acec
  • Formal review event: REQUEST_CHANGES
  • Review API status: 200
  • Required check API status: 201

Copy link
Copy Markdown

@kapi-agent kapi-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kapi-agent review

Verdict: REQUEST_CHANGES

Blocking issues

  • src/domain/run-contract.ts: compact hint compatibility is still not fully preserved for workflows that are both lifecycle-blocked/failed and have state.pendingDecision.
    • Previous behavior always emitted pending-decision whenever state.pendingDecision existed.
    • Current buildQualityDimensions() prioritizes hasBlockingLifecycle in the Completeness dimension reason, so hintIdForDimension() no longer sees a reason starting with "Pending decision" and emits completeness-dimension instead of pending-decision.
    • Since the PR explicitly preserves existing compact quality.hints, this is a regression for downstream consumers that key on pending-decision.

Please preserve pending-decision whenever state.pendingDecision exists, including blocked/failed lifecycle states, or explicitly document and update downstream consumers if this is intended.

Warnings / risks

  • src/domain/run-contract.ts: RunContractQualityStatus and ScoringHintLevel still include "unknown", but the new dimension projection does not appear to generate unknown statuses. Not blocking, but this is stale-looking contract surface.

Suggestions

  • Add a regression test for a blocked workflow with a pending decision asserting quality.hints.find(h => h.id === "pending-decision")?.level === "blocked".

Looks good

  • The five advisory dimensions are added in a generic RunContract domain surface without GitHub/Kapi-review coupling.
  • Completion authority remains separate from advisory quality dimensions.
  • The prior evidence-readiness passing ok hint regression is fixed.
  • The active pending-decision case keeps pending-decision as blocked.

Verification notes

  • Verifier gate: PASS — npm ci && npm run verify exited 0.
  • Size gate: PASS — 136 changed lines, under the 200-line limit.
  • Revision-explanation status: PASS — the author provided a current-head explanation.
  • Local inspection focused on src/domain/run-contract.ts and test/run-contract.test.ts; the remaining blocking issue is a compact-hint contract edge case not covered by current tests.

Engine: pi

@devkade devkade force-pushed the feat/runcontract-quality-dimensions branch from 242df5a to 2a15556 Compare May 15, 2026 10:37
@devkade
Copy link
Copy Markdown
Owner Author

devkade commented May 15, 2026

@kapi-agent review

Revision explanation for current head 2a155564c392214c9ce583648bce234e38f997f5:

What changed:

  • Preserved compact pending-decision hints even when a workflow is lifecycle blocked and still has state.pendingDecision.
  • Added a blocked-with-pending-decision regression assertion that pending-decision remains level: "blocked".
  • Kept the earlier compact compatibility fixes for passing evidence-readiness and active pending decisions.

Why this closes the prior feedback:

  • Prior review identified the blocked-plus-pending-decision edge case where the compact hint id became completeness-dimension; the current revision keeps the pending-decision id in that case.
  • The compatibility boundary now preserves the legacy compact hint IDs/levels while the new five dimensions remain advisory.

Verification:

  • Targeted RunContract tests passed.
  • TypeScript checks passed.
  • Full repository verify passed.
  • Diff whitespace check passed.
  • RunContract boundary grep passed.

Copy link
Copy Markdown

@kapi-agent kapi-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kapi-agent review

Verdict: REQUEST_CHANGES

Blocking issues

  • src/domain/run-contract.ts: compact hint compatibility is still not fully preserved for lifecycle-blocked/failed workflows that also have recorded risks.
    • Previous behavior emitted unresolved-risk whenever state.risks.length was non-zero, independent of lifecycle status.
    • Current buildQualityDimensions() sets the Risk dimension to fail when hasBlockingLifecycle is true, and hintIdForDimension() maps failed Risk dimensions to lifecycle-blocked.
    • Result: a blocked/failed workflow with risks now loses the legacy unresolved-risk compact hint id, even though the PR states existing compact quality.hints are preserved.
    • Please preserve unresolved-risk whenever state.risks.length > 0, including blocked/failed lifecycle states, or explicitly document this as an intentional contract change with downstream updates.

Warnings / risks

  • src/domain/run-contract.ts: RunContractQualityStatus and ScoringHintLevel still include "unknown", but the new dimension projection does not appear to generate unknown statuses. This is not blocking, but it leaves a stale-looking contract branch.

Suggestions

  • Add a regression assertion for the blocked/failed + risks case, e.g. quality.hints.find(hint => hint.id === "unresolved-risk")?.level === "attention" when state.risks.length > 0.
  • Consider avoiding duplicate or overloaded compact hint ids from dimension projection. The current mapping can make Risk lifecycle failures look like lifecycle-only blockers, which obscures risk-specific presentation behavior.

Looks good

  • The revised code preserves the positive evidence-readiness compact hint for passing evidence dimensions.
  • The revised code restores pending-decision compact hints to level: "blocked", including the blocked lifecycle + pending decision case.
  • The new five quality dimensions remain advisory and do not replace completion criteria authority.
  • The added tests cover the main dimension surface and prior evidence/pending-decision regressions.

Verification notes

  • Verifier gate status: PASS — npm ci && npm run verify exited 0.
  • Size gate status: PASS — 138 changed lines, under the 200-line limit.
  • Revision-explanation status: PASS — the author provided a current revision explanation.
  • Local/CI evidence provided: full verify passed; targeted RunContract tests, TypeScript checks, diff whitespace check, and boundary grep were reported as passing.

Engine: pi

@devkade devkade force-pushed the feat/runcontract-quality-dimensions branch from 2a15556 to 0a86642 Compare May 15, 2026 10:40
@devkade
Copy link
Copy Markdown
Owner Author

devkade commented May 15, 2026

@kapi-agent review

Revision explanation for current head 0a86642eb3e4fa68819fb600df7f05312da0f31c:

What changed:

  • Restored compact quality.hints as an explicit compatibility projection from legacy workflow state/results instead of deriving compact hints from the new advisory dimensions.
  • Preserved unresolved-risk whenever state.risks.length > 0, including lifecycle-blocked/failed workflows.
  • Preserved pending-decision whenever state.pendingDecision exists, including lifecycle-blocked/failed workflows.
  • Preserved prior evidence-readiness behavior: attention for missing required evidence, ok when evidence expectations are satisfied, and unknown when no validation evidence exists yet.
  • Kept the five new dimensions (alignment, evidence, completeness, simplicity, risk) as the richer advisory surface without changing completion authority.

Why this closes prior feedback:

  • The previous review found blocked/failed + risks lost the legacy unresolved-risk compact hint id. This revision decouples compact hints from dimension IDs and adds a blocked+risk regression assertion.
  • Prior compatibility edge cases for positive evidence-readiness and blocked+pending-decision remain covered.

Verification:

  • npx tsx --test test/run-contract.test.ts passed.
  • npm run check passed.
  • npm run check:unused passed.
  • npm run verify passed.
  • git diff --check passed.
  • RunContract boundary grep for GitHub/Kapi/Discord/Ragna coupling passed.

Copy link
Copy Markdown

@kapi-agent kapi-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kapi-agent review

Verdict: APPROVE

Final approval summary

Review journey

This PR’s original goal was to add five advisory RunContract quality dimensions while preserving existing compact quality.hints behavior and keeping completion authority in the existing workflow validation path. The semantic scope is limited to RunContract projection/quality modeling plus regression tests in test/run-contract.test.ts.

Prior kapi-agent reviews requested fixes for compact hint compatibility regressions: missing positive evidence-readiness, changed pending-decision severity, and lost unresolved-risk hint identity in blocked/risk workflows. The current revision includes an author explanation and addresses those compatibility points.

What changed

  • src/domain/run-contract.ts
    • Adds quality.dimensions with Alignment, Evidence, Completeness, Simplicity, and Risk.
    • Keeps compact quality.hints generated through the legacy-compatible path.
    • Preserves pending-decision as blocked, unresolved-risk as attention, and passing evidence as evidence-readiness/ok.
  • test/run-contract.test.ts
    • Adds/updates assertions for dimension presence/status.
    • Adds regression coverage for compact hint compatibility in blocked workflows with pending decisions and risks.

Why this is correct

The revised implementation no longer derives compact hints by lossy dimension-to-hint mapping. That avoids the earlier regressions where dimension priority changed legacy hint IDs or levels. The new dimensions remain advisory metadata, while summarizeCompletion() and completion criteria evaluation remain separate and authoritative.

Evidence

  • Verifier gate: PASS — npm ci && npm run verify exited 0.
  • Size gate: PASS — 100 changed lines, under the 200-line limit.
  • Revision explanation status: PASS — required and found.
  • Inspected files: src/domain/run-contract.ts, test/run-contract.test.ts.
  • Tests include regression assertions for evidence readiness, pending decisions, unresolved risks, and advisory dimensions not changing completion authority.

Remaining risks and approval rationale

Remaining risk is low and mostly semantic: the new dimension heuristics are intentionally simple and may need future refinement. Approval is justified because the PR is small, verified, keeps the core generic, preserves the legacy compact hint contract, and closes the prior blocking review feedback.

Blocking issues

  • None.

Warnings / risks

  • src/domain/run-contract.ts: RunContractQualityStatus and ScoringHintLevel still include "unknown" for compact hint compatibility. This is acceptable, but future dimension consumers should not assume dimensions can also be "unknown".

Suggestions

  • Consider documenting the advisory-only nature of quality.dimensions near the exported type definitions if external consumers start relying on this shape.

Looks good

  • Compact hint compatibility is now preserved independently from the new dimension model.
  • Completion status remains derived from lifecycle and completion criteria, not from advisory dimensions.
  • Regression tests cover the previously flagged edge cases.

Verification notes

  • Verifier gate: PASS — npm ci && npm run verify exited 0.
  • Size gate: PASS — 100 changed lines < 200.
  • Revision-explanation status: PASS — current revision explanation is present.
  • PR evidence reports full verify passing, targeted RunContract tests passing, TypeScript/unused checks passing, and diff whitespace checks passing.

Engine: pi

@devkade
Copy link
Copy Markdown
Owner Author

devkade commented May 15, 2026

@kapi-agent review

Revision explanation for current head c9deb728404f3f9f64898ff75e3d6ccfb2aef0b6:

What changed:

  • Aligned this PR branch with current origin/dev after docs: define harness evaluator checklist #129 merged, using a normal merge from origin/dev.
  • No RunContract quality-dimension implementation files changed beyond the already reviewed slice; the effective PR diff against updated dev remains src/domain/run-contract.ts and test/run-contract.test.ts only.

Why this closes the current gate:

  • The branch is now up to date with dev for branch policy.
  • The advisory-only quality-dimensions behavior and compact hint compatibility remain unchanged.

Verification after alignment:

  • git diff --check origin/dev...HEAD passed.
  • npm run verify passed: 397 tests, 386 pass, 11 skipped; TypeScript checks passed; quality budgets completed with existing non-failing code_smells warning.

Copy link
Copy Markdown

@kapi-agent kapi-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kapi-agent review

Verdict: APPROVE

Final approval summary

Review journey

This PR adds five advisory RunContract quality dimensions while preserving the existing compact quality.hints contract. The semantic scope is limited to src/domain/run-contract.ts and regression coverage in test/run-contract.test.ts.

Earlier kapi-agent reviews flagged compatibility regressions in compact hints, especially pending-decision, evidence-readiness, and unresolved-risk. The current revision includes explanations and closes those issues.

What changed

  • src/domain/run-contract.ts
    • Adds quality.dimensions for Alignment, Evidence, Completeness, Simplicity, and Risk.
    • Keeps compact hints generated through the legacy-compatible path.
    • Preserves previous hint IDs and severities.
  • test/run-contract.test.ts
    • Adds assertions for dimension ordering/status.
    • Adds regression coverage for compact hint compatibility.

Why this is correct

The new dimensions are advisory only and do not replace completion criteria or lifecycle validation. Existing compact hint consumers remain protected because buildCompactQualityHints() still emits the legacy hint IDs independently of dimension projection.

Evidence

  • Verifier gate: PASS — npm ci && npm run verify exited 0.
  • Size gate: PASS — 100 changed lines, under the 200-line limit.
  • Revision explanation status: not required now; found.
  • Inspected files: src/domain/run-contract.ts, test/run-contract.test.ts.

Remaining risks and approval rationale

Remaining risk is limited to future consumers potentially treating advisory dimensions as authoritative, but this PR’s wording and implementation keep completion authority separate. Approval is justified because the PR is small, verified, and resolves the prior compatibility blockers.

Blocking issues

  • None.

Warnings / risks

  • src/domain/run-contract.ts: RunContractQualityStatus and ScoringHintLevel still include "unknown" for the compact hint/status surface. This is compatible with existing behavior, but future dimension work should avoid confusing compact-hint states with dimension statuses.

Suggestions

  • Consider documenting that quality.dimensions are advisory-only wherever RunContract output is consumed by presentation/rendering code.

Looks good

  • Compact hint compatibility is preserved for pending-decision, evidence-readiness, and unresolved-risk.
  • Completion authority remains in completion criteria and lifecycle status, not the new advisory dimensions.
  • Tests cover both the new dimension surface and the legacy hint contract.

Verification notes

  • Verifier gate: PASS — npm ci && npm run verify exited 0.
  • Size gate: PASS — 100 changed lines < 200.
  • Revision-explanation status: not required now; found.
  • Local inspection matched the PR diff and test coverage.

Engine: pi

@devkade devkade merged commit b381f62 into dev May 15, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: map RunContract quality hints to five advisory dimensions

1 participant