Add GitHub Models as an AI provider for intent evaluation

[clcollins]

Summary

• Extract shared prompt-building and response-parsing into pkg/ai/prompt/ so both providers share identical prompts and parse logic
• Add internal/ghmodels/ implementing ai.Client via the GitHub Models inference API (models.github.ai), with proper auth headers and vendor-prefixed model names
• Refactor internal/claudeai/ to use the shared prompt package
• Wire up provider selection in cmd/gort/main.go via GORT_AI_PROVIDER=github-models

Configuration

Env Var	Default	Description
GORT_AI_PROVIDER	claude	"claude" or "github-models"
GORT_GITHUB_MODELS_TOKEN	falls back to GORT_GITHUB_TOKEN	PAT with models:read scope
GORT_GITHUB_MODELS_MODEL	openai/gpt-4.1	Vendor-prefixed model name

Test plan

• go build compiles all changed packages
• go test ./internal/ghmodels/... ./internal/claudeai/... ./pkg/ai/prompt/... passes
• go vet clean on all changed packages
• Deploy with GORT_AI_PROVIDER=github-models and verify intent evaluation on a real push event

https://claude.ai/code/session_01PQkbNNyPpMiKE7ULAqTkhY

[codecov]

Codecov Report

❌ Patch coverage is 80.16529% with 48 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.30%. Comparing base (a1080a2) to head (4e795a2).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/ghmodels/client.go	71.01%	12 Missing and 8 partials ⚠️
cmd/gort/main.go	0.00%	19 Missing ⚠️
pkg/ai/prompt/prompt.go	93.91%	5 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main       #3      +/-   ##
==========================================
+ Coverage   51.25%   53.30%   +2.05%     
==========================================
  Files          10       12       +2     
  Lines         880      998     +118     
==========================================
+ Hits          451      532      +81     
- Misses        381      408      +27     
- Partials       48       58      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Adds a new AI provider (GitHub Models) for intent evaluation and refactors prompt construction/response parsing so multiple providers share the same logic.

Changes:

• Introduce pkg/ai/prompt for shared prompt-building and response-parsing functions, with unit tests.
• Add internal/ghmodels implementing ai.Client via the GitHub Models chat completions API, with tests.
• Update Claude provider and the main binary wiring to use the shared prompt package and select provider via GORT_AI_PROVIDER.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
pkg/ai/prompt/prompt.go	New shared prompt builders and response parsers used by all AI providers
pkg/ai/prompt/prompt_test.go	Tests covering prompt construction and basic parsing cases
internal/ghmodels/client.go	New GitHub Models-backed ai.Client implementation
internal/ghmodels/client_test.go	HTTP-level tests verifying headers, endpoint, and basic success/error flows
internal/claudeai/client.go	Refactor to use shared prompt parsing/building functions
cmd/gort/main.go	Provider selection and config wiring for claude vs github-models

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[clcollins]

@clcollins PR #3 has been fully qualified.

Qualification Summary:

• Loops: 5 iteration(s) of Phases 1–4
• New commits: 5 commit(s) added during qualification
• CI: All 7 checks passing (Go checks, Makefile lint, Markdown lint, Plan doc, Container build, Codecov patch, Codecov project)
• Copilot: 4 rounds of feedback addressed (12 total comments across rounds 1-4), clean review on round 5
• CodeRabbit: N/A (not enabled on this repo)
• Codecov: Passing (patch and project both SUCCESS)
• Merge: No conflicts (CLEAN, MERGEABLE)

Intent validation: The PR still fulfills its original intent — adding GitHub Models as an AI provider for intent evaluation with shared prompt logic. All changes made during qualification (parser robustness improvements, test assertions) are bug fixes and improvements to the code introduced by this PR. No functional drift from the original purpose.

Commits added during qualification:

1. 927f9c9 — fix: apply gofmt formatting to Go source files
2. 0af9045 — fix: use state-based file parsing instead of extension sniffing
3. d6e2e7b — fix: use trimmed values for header detection, skip --- as path
4. 0dbe4ce — fix: check inFile before headers to preserve file content verbatim
5. 4e795a2 — test: assert HTTP method and request body in ghmodels tests

This PR is ready to merge.

---

Qualified by Claude on behalf of @clcollins