Skip to content

feat: support MCP-as-CLI progress messages on stderr#28109

Merged
pelikhan merged 3 commits intomainfrom
copilot/update-mcp-as-cli-progress-messages
Apr 23, 2026
Merged

feat: support MCP-as-CLI progress messages on stderr#28109
pelikhan merged 3 commits intomainfrom
copilot/update-mcp-as-cli-progress-messages

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 23, 2026

Summary

  • update actions/setup/js/mcp_cli_bridge.cjs to parse streamed JSON-RPC/SSE message payloads
  • detect MCP notifications/progress events and emit progress output to stderr
  • preserve existing final result/error rendering by selecting terminal result/error messages from the stream
  • add targeted unit tests for message-based and numeric progress notifications

Validation

  • cd /home/runner/work/gh-aw/gh-aw/actions/setup/js && npm run lint:cjs
  • cd /home/runner/work/gh-aw/gh-aw/actions/setup/js && npx vitest run mcp_cli_bridge.test.cjs
  • cd /home/runner/work/gh-aw/gh-aw && make agent-finish (fails in existing unrelated pkg/workflow tests: TestCopilotDetectionDefaultModel, TestWasmGolden_CompileFixtures)
  • parallel_validation (Code Review: no comments; CodeQL: no alerts reported)

Copilot AI and others added 3 commits April 23, 2026 14:21
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/bc53aa29-4a55-4a7f-9bb8-6c5b09a9a271

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/bc53aa29-4a55-4a7f-9bb8-6c5b09a9a271

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan marked this pull request as ready for review April 23, 2026 14:46
Copilot AI review requested due to automatic review settings April 23, 2026 14:46
Copilot AI requested a review from pelikhan April 23, 2026 14:49
@pelikhan pelikhan merged commit 9cf106c into main Apr 23, 2026
33 of 35 checks passed
@pelikhan pelikhan deleted the copilot/update-mcp-as-cli-progress-messages branch April 23, 2026 14:51
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 85/100

Excellent test quality

Metric Value
New/modified tests analyzed 2
✅ Design tests (behavioral contracts) 2 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 1 (50%)
Duplicate test clusters 0
Test inflation detected No
🚨 Coding-guideline violations None

Test Classification Details

Test File Classification Issues Detected
prints progress notifications to stderr and final text result to stdout for SSE responses actions/setup/js/mcp_cli_bridge.test.cjs ✅ Design None
prints numeric progress to stderr when progress notification has no message actions/setup/js/mcp_cli_bridge.test.cjs ✅ Design (edge case) None

i️ The Go change in pkg/cli/spec_test.go is purely a struct field alignment/rename fix — no new test functions were added, so it is not scored.


Test Analysis Notes

prints progress notifications to stderr and final text result to stdout for SSE responses
Verifies the behavioral contract for SSE progress notifications: that a notifications/progress message with a message field is written to stderr, and the final tool result text goes to stdout with exit code 0. This is a direct behavioral test of the new feature.

prints numeric progress to stderr when progress notification has no message
Covers the edge case where a progress notification omits the message field — the system should fall back to "Progress: N/M" formatting. Verifies both the fallback formatting on stderr and that the result still routes correctly to stdout. Good edge case coverage.


Scoring Breakdown

Component Score
Behavioral Coverage (2/2 design tests × 40) 40/40
Error/Edge Case Coverage (1/2 tests × 30) 15/30
Low Duplication (0 duplicate clusters) 20/20
Proportional Growth (test +24 lines, prod +118 lines → ratio 0.2:1) 10/10
Total 85/100

Language Support

Tests analyzed:

  • 🟨 JavaScript (*.test.cjs): 2 tests (vitest) — mcp_cli_bridge.test.cjs
  • 🐹 Go (*_test.go): 0 new tests (only struct formatting change in spec_test.go)

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). Both new tests verify observable behavioral contracts — stderr output content, stdout output content, and process exit codes — for the new MCP-as-CLI progress message feature.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

References: §24841780742

🧪 Test quality analysis by Test Quality Sentinel · ● 346.2K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 85/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). Both new JavaScript tests verify observable behavioral contracts for the MCP-as-CLI progress message feature.

@github-actions github-actions Bot mentioned this pull request Apr 23, 2026
Copilot AI review requested due to automatic review settings April 23, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants