Skip to content

feat: add continuous live contract verification - #7

Merged
ygpark80 merged 1 commit into
mainfrom
fix/continuous-verification
Aug 5, 2026
Merged

feat: add continuous live contract verification#7
ygpark80 merged 1 commit into
mainfrom
fix/continuous-verification

Conversation

@ygpark80

@ygpark80 ygpark80 commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

User description

Changes

  • Add gh2 doctor to run read-only live contract probes and distinguish parser drift from GitHub browser or sudo reauthentication.
  • Add pull-request, main-branch, and weekly CI for source installation, type checking, tests, builds, and package inspection.
  • Gate releases on tests and verify npm, standalone, Debian, and Homebrew installations after publication.
  • Keep JSON output machine-readable for Support and fine-grained PAT dry runs.
  • Use 0.0.0 for development builds and skip release-binary downloads from Git checkouts.

Testing

  • bun install --frozen-lockfile
  • actionlint
  • bun run typecheck
  • bun test — 122 tests passed
  • bun run build
  • npm pack --dry-run
  • gh2 doctor --org circlesac --support --output json — 4/4 probes healthy
  • Full sudo-authenticated doctor run — 8/8 probes healthy, with no --yes submissions

Notes

  • GitHub Actions does not store browser cookies; the live web canary is designed for a trusted local scheduler.
  • Exit code 2 means reauthentication is needed, while exit code 1 means contract drift or another failure.

CodeAnt-AI Description

Add read-only live contract checks with clear authentication and parser-drift results

What Changed

  • Adds gh2 doctor to check GitHub App, repository, PAT policy, installation, Support, and fine-grained PAT surfaces without submitting changes
  • Reports each probe as healthy, reauthentication required, contract changed, or failed, with distinct exit codes for automation
  • Keeps JSON output machine-readable by suppressing status messages from dry-run and successful Support and PAT commands
  • Adds continuous pull-request, main-branch, scheduled, and release verification across supported installation methods
  • Documents trusted local scheduling and browser-session requirements for live checks

Impact

✅ Detects GitHub parser drift before administrative operations fail
✅ Separates expired sessions from contract changes
✅ Verifies published npm, standalone, Debian, and Homebrew installations

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

Summary by CodeRabbit

  • New Features
    • Added the read-only gh2 doctor command to verify GitHub integrations, authentication, and API responses.
    • Supports organization, App, installation, Support, PAT, and JSON-output checks.
  • Improvements
    • Dry-run and JSON output for support and PAT commands are now cleaner and consistently formatted.
    • Installers now skip execution in CI environments and Git checkouts.
  • Documentation
    • Added live verification guidance, scheduling recommendations, exit-code details, and troubleshooting instructions.
  • Quality
    • Expanded automated checks across supported platforms and installation methods.

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Reviewed your PR aba557a Aug 05, 2026 · 05:08 05:11

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d4a70a92-5ddb-4933-9613-bd2b822aaf2b

📥 Commits

Reviewing files that changed from the base of the PR and between 637a761 and aba557a.

📒 Files selected for processing (13)
  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • README.md
  • bin/install.js
  • docs/live-canary.md
  • package.json
  • skills/guide/SKILL.md
  • src/commands/doctor.ts
  • src/commands/pat/create.ts
  • src/commands/support/create.ts
  • src/commands/support/reply.ts
  • src/index.ts
  • tests/doctor.test.ts

📝 Walkthrough

Walkthrough

Added the read-only doctor command with GitHub contract probes, structured reports, documentation, and tests. Updated JSON output behavior and added CI and release workflows that verify builds and published installations.

Changes

Doctor command and verification

Layer / File(s) Summary
Doctor probe planning and validation
src/commands/doctor.ts
The command now builds and runs read-only organization, App, Support, installation, and PAT probes. It validates JSON responses and classifies failures.
Doctor reporting and CLI integration
src/commands/doctor.ts, src/index.ts, tests/doctor.test.ts
Probe results now produce reports and exit codes. The doctor subcommand is registered. Tests cover planning, classification, reauthentication, and contract changes.
Structured dry-run and creation output
src/commands/pat/create.ts, src/commands/support/create.ts, src/commands/support/reply.ts
JSON output no longer includes human-readable status messages. Support creation and reply commands emit consistent structured results.
Live canary documentation
README.md, docs/live-canary.md, skills/guide/SKILL.md
Documentation describes doctor options, exit statuses, authentication behavior, parser-drift detection, and scheduling guidance.
CI and release installation verification
.github/workflows/ci.yml, .github/workflows/release.yml, bin/install.js, package.json
CI now runs type checks, tests, builds, and package checks. Releases verify published installation methods across Ubuntu and macOS. The installer exits in CI and Git checkouts.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant doctorCommand
  participant runDoctorProbes
  participant GitHub
  Operator->>doctorCommand: invoke gh2 doctor
  doctorCommand->>runDoctorProbes: execute configured probes
  runDoctorProbes->>GitHub: run read-only contract checks
  GitHub-->>runDoctorProbes: return JSON responses or errors
  runDoctorProbes-->>doctorCommand: aggregate probe statuses
  doctorCommand-->>Operator: print table or JSON report and exit code
Loading
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/continuous-verification

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codeant-ai codeant-ai Bot added the size:XL This PR changes 500-999 lines, ignoring generated files label Aug 5, 2026
@ygpark80
ygpark80 merged commit fe3a46d into main Aug 5, 2026
2 checks passed
@ygpark80
ygpark80 deleted the fix/continuous-verification branch August 5, 2026 05:10
Comment thread bin/install.js
}

if (process.env.CI) process.exit(0);
if (process.env.CI || fs.existsSync(path.join(__dirname, "..", ".git"))) process.exit(0);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The installer exits whenever the package is installed from a Git checkout or while CI is set, even when bin/native/gh2 is missing. The launcher then imports this installer and immediately attempts to spawn that nonexistent binary, so gh2 fails to start for source-checkout installations and normal CI-installed packages. Only skip release downloads when a source-mode launcher is actually available, or make bin/gh2 fall back to the source entry point. [logic error]

Severity Level: Major ⚠️
- ⚠️ Normal non-CI registry installs remain unaffected.

Fix in Cursor Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** bin/install.js
**Line:** 35:35
**Comment:**
	*Logic Error: The installer exits whenever the package is installed from a Git checkout or while `CI` is set, even when `bin/native/gh2` is missing. The launcher then imports this installer and immediately attempts to spawn that nonexistent binary, so `gh2` fails to start for source-checkout installations and normal CI-installed packages. Only skip release downloads when a source-mode launcher is actually available, or make `bin/gh2` fall back to the source entry point.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

Comment thread src/commands/doctor.ts
Comment on lines +133 to +135
if (!data || typeof data !== "object" || Reflect.get(data, "app") !== options.app) {
return "App permission output did not identify the requested App.";
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The validator rejects an empty permission map as contract drift, but the permissions command intentionally omits permissions whose level is none, so an App with all permissions unset produces a valid current: {} response. This causes gh2 doctor to report healthy live data as parser drift. [incorrect condition logic]

Severity Level: Major ⚠️
- ❌ Doctor falsely reports valid App permissions as contract drift.
- ⚠️ Support diagnostics receive an incorrect failing probe.
- ⚠️ Exit code 1 can trigger false CI or scheduler alerts.

Fix in Cursor Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** src/commands/doctor.ts
**Line:** 133:135
**Comment:**
	*Incorrect Condition Logic: The validator rejects an empty permission map as contract drift, but the permissions command intentionally omits permissions whose level is `none`, so an App with all permissions unset produces a valid `current: {}` response. This causes `gh2 doctor` to report healthy live data as parser drift.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant