Skip to content

test: repair verified CLI evidence gates - #777

Merged
stevenobiajulu merged 2 commits into
mainfrom
tweak-verified-cli-ci-gates
Jul 30, 2026
Merged

test: repair verified CLI evidence gates#777
stevenobiajulu merged 2 commits into
mainfrom
tweak-verified-cli-ci-gates

Conversation

@stevenobiajulu

Copy link
Copy Markdown
Member

Summary

  • give the new CLI OpenSpec tests a deterministic Allure feature ID
  • place CLI scenarios under the mcp-server delta so strict package coverage maps all 5/5 scenarios
  • update the standalone protocol-v7 drift fixture with the required proof that its selected ZIP part is a regular file

Why this follow-up exists

PR #776 squash-merged while allure-labels and lean-build were non-blocking failures. This PR contains only the corrections for those two evidence gates; the product implementation is already on main.

Local evidence

  • npm run check:allure-labels
  • strict MCP OpenSpec coverage: 5/5
  • openspec validate add-verified-comparison-cli --strict
  • lake env lean ProtocolV7ProjectionDriftWitnesses.lean: 19 witnesses passed

Refs #775 and #776

Give the new OpenSpec traceability tests a deterministic feature identity so the Allure label policy can map them. Place the CLI requirements under the mcp-server capability while retaining the formal ZIP semantics under docx-comparison, allowing both strict coverage validators to evaluate the correct package boundary.\n\nRef: #775
The production extraction evidence now requires a proof that the selected ZIP entry is not a directory. The standalone protocol-v7 drift witness is compiled explicitly by CI rather than through the default lake target, so supply the same fail-closed regular-entry guard there.\n\nRef: #775
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
site Ready Ready Preview Jul 30, 2026 6:02pm

Request Review

@usejunior-llm-gate

Copy link
Copy Markdown

Aggregate and post review

Overall: PASS (7 pass, 0 warn, 0 error, 10 skipped, 17 total)

Check Verdict
read_file response metadata parity SKIPPED paths not touched by this PR
Live DOM namespace-safe OOXML writes SKIPPED paths not touched by this PR
Deleted field markup keeps w:fldChar outside w:del PASS The PR does not modify field atomization, 'validateFieldStructure', or deleted field markup, touching only CLI specs, a comparison command test in packages/docx-mcp/src/cli/commands/compare.test.ts, and verification/lean/ProtocolV7ProjectionDriftWitnesses.lean.
Field validation per story, not global SKIPPED paths not touched by this PR
Revision IDs seeded from all revision-bearing side parts SKIPPED paths not touched by this PR
Accept/reject sweep side parts and caches PASS The PR only modifies OpenSpec verification requirements, CLI comparison test labels, and Lean verifier witnesses, and does not touch DocxDocument.acceptChanges, DocxDocument.rejectChanges, REVISION_STORY_PART_PATHS, or revision-bearing stories.
DocumentViewNode.heading stays canonical SKIPPED paths not touched by this PR
AI-author parity across entry points PASS Every entry point constructing a SessionManager in docx-mcp, including the MCP server (server.ts:296) and CLI runners (tool_runner.ts:27, edit.ts:132, grep.ts:140), symmetrically resolves SAFE_DOCX_AI_AUTHOR with the exact same three-way semantics via resolveCliAiAuthor() (tool_runner.ts:17).
Property-change wrapper discipline SKIPPED paths not touched by this PR
SUPPORT.md Table A drift vs. implementation SKIPPED paths not touched by this PR
Table A / Table B boundary on side-part revisions SKIPPED paths not touched by this PR
Canonical-emission surface completeness SKIPPED paths not touched by this PR
Lean predicate drift against engine semantics (asymmetric) SKIPPED paths not touched by this PR
Unit-test quality (avoid tautological / change-detector tests) PASS The PR only updates the test reporting label metadata in packages/docx-mcp/src/cli/commands/compare.test.ts:23-24. The existing assertions are high-quality, independent of the SUT, and verify concrete semantic outcomes without being tautological.
Re-derived facts vs canonical sources PASS The PR only modifies specifications, a Lean proof file, and a test file label, and does not introduce or touch any TypeScript logic that computes or re-derives facts.
.openspec tag ↔ test-assertion drift PASS The PR only modifies the TEST_FEATURE label in review/packages/docx-mcp/src/cli/commands/compare.test.ts:22-23 and does not add, move, or change any .openspec(...) tags.
Library stays general (no downstream-domain leakage) PASS The PR only touches test code in review/packages/docx-mcp/src/cli/commands/compare.test.ts:23 and does not add or rename public APIs or identifiers with downstream-domain leakage.

@stevenobiajulu
stevenobiajulu merged commit ece5dfa into main Jul 30, 2026
26 of 27 checks passed
@stevenobiajulu
stevenobiajulu deleted the tweak-verified-cli-ci-gates branch July 30, 2026 18:12
@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

stevenobiajulu added a commit that referenced this pull request Jul 30, 2026
The verified CLI intentionally lowers the production verifier default from 120 seconds to 10 seconds, but the corpus-scale extraction-limit tests accidentally inherited that tighter runtime budget. GitHub's slower runner crossed the limit while constructing an expected failure certificate, making lean-build flaky even though the proof and checker were correct.\n\nGive only the compiled real-checker test helpers their historical 120-second allowance. Dedicated timeout tests continue to set short explicit budgets, while the CLI and library production default remain 10 seconds.\n\nRef: #777
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant