Skip to content

Add golden test for gh aw --help output#5926

Closed
Copilot wants to merge 4 commits intomainfrom
copilot/add-golden-test-gh-aw-help
Closed

Add golden test for gh aw --help output#5926
Copilot wants to merge 4 commits intomainfrom
copilot/add-golden-test-gh-aw-help

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Dec 9, 2025

Adds snapshot testing to detect unintended changes to CLI help output.

Changes

  • cmd/gh-aw/help_output_test.go: Integration test comparing gh aw --help output against golden file

    • Captures actual output and diffs against baseline
    • UPDATE_GOLDEN=1 environment variable updates snapshot
    • Clear diff output on mismatch
  • cmd/gh-aw/testdata/help_output.golden.txt: Baseline snapshot

  • CONTRIBUTING.md: Documents golden file update workflow

Usage

Update golden files after intentional help text changes:

UPDATE_GOLDEN=1 go test -v -tags integration -run TestHelpOutputGolden ./cmd/gh-aw

Test fails in CI if help output drifts, showing exact differences.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/user
    • Triggering command: /usr/bin/gh gh api user --jq .login -unreachable=false /tmp/go-build4031923621/b255/vet--log-format /opt/hostedtoolcache/go/1.25.0/xjson (http block)
    • Triggering command: /usr/bin/gh gh api user --jq .login /ref/tags/v8 g/timeutil/format_test.go by/213d854ce4107514e8e0f3331ef6c834d0e6d04bd343d9469b481c3facb8da30/log.json (http block)
    • Triggering command: /usr/bin/gh gh api user --jq .login 56/sanitize_output.js c 3a14e89ff37ddbaf3357e37bebcb55cd07b/log.json - efer <8320933+mnapi (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Add golden test: gh aw --help</issue_title>
<issue_description>## Summary
Add a snapshot test to detect changes in gh aw --help output.

Implementation Notes

  • Capture baseline snapshot
  • Diff against snapshot in CI
  • Provide update instructions for contributors

Success Criteria

  • Help output drift is automatically caught in PRs
    </issue_description>

Comments on the Issue (you are @copilot in this section)

@pelikhan Sounds good assign to copilot.
  • Fixes githubnext/gh-aw-planning#10

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 3 commits December 9, 2025 06:20
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copilot AI changed the title [WIP] Add golden test for gh aw --help output Add golden test for gh aw --help output Dec 9, 2025
Copilot AI requested a review from mnkiefer December 9, 2025 06:55
@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Dec 9, 2025

This test will fail on every change to the help strings.

@pelikhan pelikhan closed this Dec 9, 2025
@pelikhan pelikhan deleted the copilot/add-golden-test-gh-aw-help branch December 15, 2025 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants