Skip to content

feat: org-scoped coverage via org-owned tokens (#5) - #16

Merged
gkanitz merged 10 commits into
mainfrom
feat/5-org-scoped-coverage
Jun 12, 2026
Merged

feat: org-scoped coverage via org-owned tokens (#5)#16
gkanitz merged 10 commits into
mainfrom
feat/5-org-scoped-coverage

Conversation

@gkanitz

@gkanitz gkanitz commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Closes #5

Summary

One run with an org-scoped token now covers the subject's activity across every repository the token can see, aggregated into a single report, with a coverage stamp that makes the breadth explicit: full covered-repo list, time window, and token scope class ("covers N repos of org X, window Y–Z").

  • Repo enumeration (provider/github/repos.go, new): ListOrgRepos pages /orgs/{org}/repos; ListInstallationRepos pages /installation/repositories. Metadata only — repo names, never contents.
  • GitHub App installation tokens (provider/github/apptoken.go, new): AppAuth turns app ID + private key (PKCS#1 or PKCS#8 PEM) into a signed RS256 app JWT — stdlib only, zero new dependencies — auto-discovers the sole installation (or errors listing candidates when ambiguous), and mints the short-lived ghs_ token. Token acquisition stays pluggable: the adapter and pipeline consume "a token".
  • Token scope class (provider/github/scope.go, new): ClassifyToken maps token prefix + reported X-OAuth-Scopes to app-installation / fine-grained-pat / classic-pat / oauth-app / unknown, never echoing the token.
  • Coverage stamp (report): Coverage gains token_scope_class (set via new additive Build option WithTokenScopeClass); Validate now rejects an empty coverage repo list (permitted tightening).
  • CLI (cmd/coderepute): new additive flags -org, -app-id, -app-key, -installation-id; resolution order: explicit -repo-org enumeration → whole installation under an App token. New file orgscope.go holds the plumbing.
  • Render: coverage section shows "N repositories covered" and the scope class.

Parallel-work contract compliance

  • ActivitySet, the adapter's activity-fetch file (github.go), metrics package files, collaboration/cadence/verification sub-structs, CI YAML: untouched.
  • Shared files touched additively only: report/report.go (new field, variadic Build option — existing call sites compile unchanged, plus the explicitly permitted Validate repo-list tightening), cmd/coderepute/main.go (new flags + wiring), render/templates/sections/50-coverage.tmpl (my territory to enrich).
  • Cross-repo aggregation is proven at the adapter (fixtures: PRs in acme/widgets + acme/gadgets sum per-repo and in total) and end-to-end at the CLI (report sums 1 + 2 authored PRs across repos). Per the contract, no metrics files were modified; aggregation flows through the existing per-repo fetch loop and metrics.Compute.

Test output

$ go test -count=1 ./...
ok  	github.com/grkanitz/coderepute/cmd/coderepute	0.279s
ok  	github.com/grkanitz/coderepute/metrics	0.714s
?   	github.com/grkanitz/coderepute/provider	[no test files]
ok  	github.com/grkanitz/coderepute/provider/github	0.890s
ok  	github.com/grkanitz/coderepute/render	0.396s
ok  	github.com/grkanitz/coderepute/report	0.866s
$ go vet ./...   # clean

The token exchange itself is fixture-tested: the test server cryptographically verifies the app JWT (RS256 signature against the real public key, iss claim, exp > iat) before serving the installation token.

Manual checklist — App-token live validation (real org)

No real org credentials were used in development; run this against a real org before relying on App auth:

  • Create a GitHub App on the org (app-manifest flow or manually) with read-only permissions: Metadata: Read, Pull requests: Read; webhooks disabled.
  • Install the App on the org, granting All repositories.
  • Note the App ID; generate and download a private-key PEM.
  • coderepute -app-id <APP_ID> -app-key <key.pem> -subject <username> -out out/
  • Confirm out/report.json coverage.repos lists every org repo, including private ones, and nothing outside the installation.
  • Confirm coverage.token_scope_class is app-installation.
  • Spot-check cross-repo sums: pick two repos with known PR counts by the subject in the window and verify collaboration.pull_requests matches their sum.
  • If the App is installed on more than one account, confirm the run fails listing candidate installation IDs, and that -installation-id <ID> selects one.
  • Re-run to confirm a fresh token is minted each run (tokens expire after ~1 h; no caching exists by design).
  • Optionally re-run with -org <org> plus a classic PAT and diff the coverage stamps (classic-pat vs app-installation).

🤖 Generated with Claude Code

gkanitz and others added 10 commits June 12, 2026 12:15
Incomplete TDD state; resume from here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ListOrgRepos pages through /orgs/{org}/repos; ListInstallationRepos
pages through /installation/repositories. Both return owner/name lists
that feed the report's coverage stamp.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ClassifyToken maps token prefix (ghs_/github_pat_/ghp_/gho_) and the
reported X-OAuth-Scopes header to a scope class, never echoing the
token itself.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…o list

Coverage gains token_scope_class, set via the new additive Build
option WithTokenScopeClass. Validate now rejects reports whose
coverage stamp lists no repos, so an empty enumeration cannot pass
silently.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AppAuth turns app ID + private key into a signed RS256 app JWT
(stdlib only, no new deps), discovers the sole installation when none
is chosen, and mints the short-lived installation token. Fixture
tests verify the JWT signature server-side and pin the ambiguity
error when several installations exist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PRs authored in acme/widgets and acme/gadgets sum into one
ActivitySet: per-repo counts, total, merged count, and the repo list
that feeds the coverage stamp.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New flags: -org enumerates every repo of the org visible to the
token; -app-id/-app-key (+optional -installation-id) mint an App
installation token instead of -token; under an App token with no
-repo/-org the run covers the whole installation. One run aggregates
the subject's activity across all enumerated repos into a single
report, and the coverage stamp carries the token scope class.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The coverage section now reads "N repositories covered" and names the
credential class alongside the raw scopes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
After rebasing onto main (which merged PR #14 cadence and PR #15
collaboration metrics), two gaps surfaced:

1. provider/github now fetches /pulls/{n}/reviews and /pulls/comments
   for every PR in every repo; the multi-repo and org-scoped fixture
   servers were missing those routes, so FetchActivity returned 404s.
   Added empty-array handlers for all PR numbers exercised by those
   servers.

2. resolveRepos used a plain strings.Split for the -repo flag, silently
   passing leading/trailing whitespace into repo names. Ported main's
   splitRepos trimming logic into resolveRepos so TestRunTrimsRepoListWhitespace
   continues to pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gkanitz
gkanitz force-pushed the feat/5-org-scoped-coverage branch from b892f93 to f4bda38 Compare June 12, 2026 11:20
@gkanitz

gkanitz commented Jun 12, 2026

Copy link
Copy Markdown
Owner Author

Pre-review rebase

Rebased feat/5-org-scoped-coverage onto 1aa7990 (main after PRs #14 cadence and #15 collaboration metrics merged).

Conflicts resolved (keep-both)

cmd/coderepute/main.go — two conflict zones:

  • Activity fetch: kept this branch's resolveRepos(ctx, adapter, ...) call (replacing both HEAD's splitRepos(*repos) and the branch's context.Background()) to preserve org-scope enumeration with the ctx variable already set up earlier in the function.
  • Report build: merged main's &result.Cadence arg with this branch's report.WithTokenScopeClass(...) variadic option, giving report.Build(activity, &result.Collaboration, &result.Cadence, time.Now(), report.WithTokenScopeClass(...)).
  • Removed the splitRepos helper and strings import from main.go (both live in orgscope.go now).

Post-rebase fixups (separate commit f4bda38)

Two gaps surfaced that the conflict resolution exposed:

  1. Missing review/comment fixtures: PR feat: full GitHub collaboration metrics (#3) #15 extended the GitHub adapter to fetch /pulls/{n}/reviews and /pulls/comments for every PR. The multi-repo and org-scoped fixture servers in this branch were missing those routes. Added empty-array handlers for all exercised PR numbers in provider/github/aggregate_test.go and cmd/coderepute/orgscope_test.go.

  2. Whitespace trimming in resolveRepos: main had added whitespace trimming to splitRepos; this branch's resolveRepos used a plain strings.Split. Ported the trim-and-drop-empty logic so TestRunTrimsRepoListWhitespace passes.

Test results

ok  github.com/grkanitz/coderepute/cmd/coderepute      0.823s
ok  github.com/grkanitz/coderepute/metrics             1.133s
ok  github.com/grkanitz/coderepute/provider/github     1.255s
ok  github.com/grkanitz/coderepute/render              0.285s
ok  github.com/grkanitz/coderepute/report              0.424s

go build ./... and go vet ./... clean. New branch tip: f4bda381cf792c2ab31aed128eca20cc802c1f3d.

@gkanitz

gkanitz commented Jun 12, 2026

Copy link
Copy Markdown
Owner Author

Review round 1 — approved

Independently verified on a clean checkout of f4bda38 (merge-base = main tip 1aa7990): go build ./..., go vet ./..., go test -count=1 ./... all clean.

Acceptance criteria

  1. Org-scoped single runTestRunOrgScoped proves one run with -org enumerates both repos and emits one report; resolveRepos precedence (-repo-org → whole installation) is explicit and tested.
  2. Coverage stampcoverage.repos carries the full enumerated list, window, and new token_scope_class; Validate now rejects an empty repo list (pre-approved tightening); render shows count + class (TestHTMLShowsCoverageBreadth).
  3. App installation token flow — end-to-end: PEM parse (PKCS#1 + PKCS#8) → RS256 app JWT → installation discovery (sole / explicit / ambiguous-error listing candidates) → minted ghs_ token; live-org validation is a concrete manual checklist in the PR body, acceptable per the issue.
  4. Multi-repo fixtures + cross-repo sumsTestFetchActivityAggregatesAcrossRepos (2+2 PRs, 1+1 merged across acme/widgets + acme/gadgets, with org-repos pagination) and the CLI e2e (authored 3 = 1+2, merged 2 = 1+1 flowing through metrics.Compute). Metrics files untouched, per the contract.

Security scrutiny

  • JWT: RS256, iss=app ID, iat backdated 60 s, exp = now+9 min (within GitHub's 10-min cap). The fixture server genuinely verifies the signature against the real public key plus iss and exp > iat — not a stub.
  • Key/token hygiene: PEM and tokens never logged or echoed; ClassifyToken inspects only the well-known prefix; report carries only the class. e2e test asserts activity calls authenticate with the minted ghs_ token.
  • Metadata-only: repos.go constructs only /orgs/{org}/repos and /installation/repositories; the pre-existing endpoint-allowlist test is untouched (extending it would have meant editing the sibling-frozen github_test.go; the strict fixture muxes 404 anything else). Fixture repo descriptions seeded with "SECRET" never leave the adapter — only full_name is parsed.

Contract

Owned new files only (repos.go, apptoken.go, scope.go, orgscope.go + tests/fixtures); shared files additive (report.go variadic option + permitted tightening, main.go flags/wiring); ActivitySet, github.go, metrics, sibling sub-structs, CI YAML untouched. Rebase-induced touches (empty review/comment fixture routes, whitespace-trim port into resolveRepos) match the pre-review-rebase comment and stayed minimal.

Partial failure

A failing repo fetch aborts the whole run with an error — no silent skip, so the coverage stamp can never overstate what was covered. Honest behavior.

Non-blocking nits (no action required)

  • apptoken.go doJSON includes the raw response body in unexpected-status errors; in the pathological case of a token response with a non-201 status the minted token could reach stderr. Same pattern as the pre-existing getJSON; consider truncating/sanitizing both someday.
  • ListInstallationRepos pagination loop is shared with the tested org path but has no multi-page fixture of its own.

Merging.

@gkanitz
gkanitz merged commit a939594 into main Jun 12, 2026
1 check passed
@gkanitz
gkanitz deleted the feat/5-org-scoped-coverage branch June 12, 2026 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Org-scoped coverage via org-owned tokens with coverage stamping

1 participant