feat: GitHub repo-host vertical with live-tests CI#3
Merged
goldenwitch merged 16 commits intomainfrom May 4, 2026
Merged
Conversation
Phase 1+2+3 of the ADO/GHE work: * Rename `GitHubAdapter` trait -> `RepoHostAdapter`, `GitHubError` -> `RepoHostError`. The trait now exposes `host_id()` so downstream code can key on the forge instance. * New `host` module with `RepoHostKind` (`GitHub`, `AzureDevOps`) and `RepoHostId` (kind + api_base + browse host). Includes a permissive URL host classifier (github.com / ghe.* / dev.azure.com / *.visualstudio.com) and `from_browse_host` for URL-driven dispatch. * `GitHubAdapter` (concrete struct, was `LiveGitHubAdapter`) now takes a `RepoHostId` and threads `api_base` through every URL formatter, so the same impl serves github.com and any GitHub Enterprise Server install (https://<ghe>/api/v3). Convenience constructors: `github_com`, `ghe`, `from_env`, `new`. * New `AzureDevOpsAdapter` against ADO REST 7.0 with documented mappings: PR `status` -> `PrState`, reviewer vote integers -> `ReviewEvent`, and PR Status policy `state` -> `CheckRun.status`/`conclusion`. ADO's org/project/repo triple is encoded as owner="<org>/<project>", repo="<repo>". Auth via HTTP Basic with empty user + PAT. * Renamed `MockGitHubAdapter` -> `MockAdapter`; default host is github.com but `with_host` lets tests simulate any forge. * Daemon dispatch + tasks + tests rewired to the trait rename. `select_github_adapter` now reads `DEVDEV_REPO_HOST_ADAPTER`, with `DEVDEV_GITHUB_ADAPTER` retained as a legacy alias. Workspace `cargo build` + `cargo clippy --workspace --all-targets -- -D warnings` + `cargo test --workspace --tests` are green. The pre-existing WinFSP-related `live_workspace_cwd` test still fails with STATUS_DLL_NOT_FOUND on this machine (unchanged from main). Follow-ups on this branch: credential snapshot abstraction, daemon host registry, multi-host PR ref parsing, preferences `[[repo]]` entries, scenario S08.
Snapshot-once credential lifecycle for multi-host auth.
- credentials.rs: CredentialProvider trait + EnvVarProvider,
GhCliProvider, AzCliProvider built-ins; frozen CredentialStore
keyed by RepoHostId; RedactedString wrapper that never leaks via
Debug/Display.
- dispatch.rs: replace Arc<Mutex<AgentSecrets>> with Arc<CredentialStore>.
- mcp/provider.rs: DaemonToolProvider takes Arc<CredentialStore>;
ask() resolves github.com from the snapshot for token-bearing kinds.
- daemon_cli::run_up: build snapshot from EnvVar(GH_TOKEN) -> GhCli
chain, log captured source.
- secrets.rs deleted; e2e_pr_shepherding migrated.
Tests (11 new, all passing):
- redacted_string_does_not_leak_via_debug_or_display
- snapshot_records_token_for_each_host
- snapshot_falls_through_on_none_within_chain
- snapshot_first_provider_wins_when_both_have_token
- store_is_immutable_after_snapshot_env_mutation (lifecycle gate)
- env_var_provider_{unset,empty,records_source}
- with_entry_round_trips, empty_store_returns_none_and_is_clonable
- expires_at_hint_is_one_hour_after_sample
cargo clippy --workspace --all-targets -- -D warnings clean.
All workspace tests pass except pre-existing WinFSP
live_workspace_cwd failure (unchanged from main).
PrRef carries a RepoHostId so URLs to GitHub.com, GHE, dev.azure.com, and *.visualstudio.com all parse into a single host-tagged value. RepoWatchTask, MonitorPrTask, and DaemonEvent now track host_id end-to-end so identical (owner, repo, number) triples on different hosts cannot collide in dispatch maps, ledger keys, or the event bus. dispatch.rs threads host_id through repo/watch, repo/unwatch, task/add, and ensure_monitor_pr_task; the params.host field defaults to github.com for back-compat. Phase 5 will replace the default with registry-driven routing.
RepoHostRegistry maps RepoHostId to RepoHostAdapter so dispatch layers route to the correct API surface for github.com, GHE, and dev.azure.com hosts. for_url() classifies a browser-shaped URL or bare host via RepoHostId::from_browse_host and looks up the registered adapter. DispatchContext exposes a host_registry alongside the legacy github default; adapter_for(&host_id) prefers a registered adapter and falls back to github so single-host smoke flows keep working. AskRequest now carries an optional host field that selects which credential entry the response surfaces; unknown hosts are rejected hard. 10 new tests (8 registry + 2 ask routing).
docs/internals/spec-repo-hosts.md captures the multi-host architecture: RepoHostId/Adapter/Registry/CredentialStore seams, identity invariants, URL parsing matrix, the wire surface for repo/watch and devdev_ask host fields, and a test-landmark map. Scenario S08 exercises the full IPC surface end-to-end: same (owner, repo) on github.com vs ghe.acme.io must produce distinct task ids, idempotent re-watch returns the same id, unknown hosts hit -32602, and host-keyed unwatch never collapses across hosts.
New crate `devdev-test-env` (publish=false) reads a JSON manifest at `test-env/manifest.json` and reconciles GitHub.com + Azure DevOps fixtures via hand-rolled REST. Server-assigned ids land in a committed `manifest.lock.json`. Subcommands: apply (idempotent reconcile), verify (drift check), reset-comments (per-test cleanup; non-admin always swept, admin swept only if body carries the [devdev-live-test...] tag), print-env (emits DEVDEV_GH_PR_URL + ADO equivalents for the test runner), destroy (intentionally not implemented in first cut). CI: .github/workflows/live-tests.yml runs a three-job pipeline (provision -> live-tests -> cleanup). Admin tokens live in the live-tests-admin GitHub Environment and are visible only to provision and cleanup; consumer tokens live in live-tests-consumer and are visible only to the live-tests job. Triggered by workflow_dispatch, nightly cron, or PRs labeled `live-tests`. CODEOWNERS protects the workflow, manifest, and crate. Docs: docs/internals/live-test-fixtures.md (operational story, bootstrap, principal model, cost) and docs/internals/ghe-gap.md (deliberate GHE gap, what we still rely on, sponsorship invitation). Claim FIXTURE-MANIFEST-INTEGRITY covers the deterministic side (manifest validation + reset decision logic, 14 unit tests, validate.ps1 PASS). The fixture-state-matches-manifest side runs in CI only because it requires admin tokens.
Three new `#[ignore]`-gated live tests in devdev-cli, each with a separate env flag so CI can opt them in independently: * live_host_probe.rs: GitHubAdapter and AzureDevOpsAdapter round-trip the canonical fixture PR (get_pr + list_open_prs); asserts host_id stamp matches PrRef classification. Gate: DEVDEV_LIVE_HOSTS=1. * live_credential_chain.rs: GhCliProvider and AzCliProvider produce non-empty tokens from real signed-in CLIs; asserts TokenSource is GhCli/AzCli respectively. Gate: DEVDEV_LIVE_CRED_GH=1 / DEVDEV_LIVE_CRED_AZ=1. * live_ado_pr.rs: read path always (DEVDEV_LIVE_HOSTS=1); write path posts a tagged comment ([devdev-live-test:live_ado_pr:<nonce>]) and verifies it lands in the next list_pr_comments. Write path gated by DEVDEV_LIVE_WRITE=1; cleanup happens via devdev-test-env reset-comments. #[serial] to avoid nonce collisions. Wired into .github/workflows/live-tests.yml: the live-tests job now sets DEVDEV_LIVE_HOSTS=1 and DEVDEV_LIVE_CRED_GH=1 by default; DEVDEV_LIVE_CRED_AZ stays empty until the workflow seeds az login. Five new claims in claims.toml, surfaced in VALIDATION.md. README.md gains a pointer to the live-test fixture docs.
- Wrap GithubClient.token + AdoClient.auth_header in a Token newtype that redacts via Debug/Display; expose() is the only path to the raw value. - Rewrite .github/workflows/live-tests.yml to mint installation tokens via actions/create-github-app-token@v1 from environment-scoped vars + secrets. ADO portions are env-gated (DEVDEV_LIVE_ADO_ENABLED) until federated creds land. - Add scripts/devdev-secrets.ps1 (vault helper), scripts/verify-gh-apps.ps1, scripts/verify-gh-e2e.ps1 (raw-REST E2E), scripts/seed-ci-secrets.ps1 (idempotent CI bootstrap from vault). - Rename canonical fixture to goldenwitch/devdev-test-environment; add test-env/manifest.lock.json captured from live apply.
…-bound, not a fixture test)
Discovered while dogfooding the new keyring chain. The daemon's startup capture step was correctly pulling the gh CLI token into the CredentialStore, but select_github_adapter was still calling GitHubAdapter::from_env(), which only consults GH_TOKEN. Result: 'credential captured (source: GhCli)' followed immediately by 'falling back to mock adapter'. Thread the CredentialStore into the selector so the live adapter uses whichever provider won (env var or gh CLI).
The CLI used to scatter agent-launch logic across three places: acp_backend.rs called rewrite_copilot_invocation inline, realpath_shim.rs owned the Windows-only Copilot SEA bypass, and the live_daemon_fs_write test rolled its own PATHEXT search. Bare 'copilot' on Windows then went straight to Command::new (which doesn't apply PATHEXT for extensionless names), failing with 'program not found' even though the index.js was sitting right next to the .cmd shim and the rewrite would have worked if we'd resolved the launcher first. Introduce devdev_cli::agent_command::prepare(program, args) as the single canonical entry-point: PATH+PATHEXT resolution -> Copilot SEA-bypass rewrite -> spawn-ready (program, args). acp_backend now calls prepare() once; the live test reuses resolve_on_path. Verified end-to-end: 'devdev send pong' now spawns Copilot, gets a reply, and emits exactly one INFO line announcing the chosen launch path.
cleanup ran with 'needs: live-tests' + 'if: always()' but unconditionally downloaded the manifest-lock artifact. If gate skipped the workflow or provision failed before upload, live-tests was skipped and cleanup still ran, then failed on the missing artifact -- turning a clean skip/failure into a confusing secondary cleanup failure. Make cleanup depend on provision and only run when provision succeeded (it's the only job that uploads manifest-lock and the only thing cleanup needs to undo).
Polls a GitHub repository for open PRs and asks a Copilot ACP agent to review each one as it appears or updates. Read-only; never posts to the PR. The point of this sample is to exercise the *library* surface of every DevDev crate without going through the daemon or its IPC. Each crate contributes exactly one thing: devdev-acp gives us AcpClient::connect_process, devdev-cli gives us agent_command::prepare (the canonical resolver+rewriter), devdev-daemon gives us the CredentialStore + provider chain, devdev-integrations gives us GitHubAdapter and pr_state_hash. If anything in this sample ends up doing the daemon's work by hand, that's a signal a daemon-internal helper should be promoted to a library API. Validated end-to-end: 'cargo run -p pr-reviewer -- goldenwitch/devdev --once' reviewed PR #3, found 3 substantive issues, and exited cleanly. Builds clean as a workspace member, so any future API breakage in the public surface fails 'cargo build --workspace' immediately.
goldenwitch
changed the title
This was referenced May 4, 2026
# Conflicts: # .github/workflows/live-tests.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the GitHub repo-host vertical end-to-end: the
RepoHostAdapterabstraction, a working github.com adapter, a fully validated live-test fixture pipeline running against a real repo via short-lived GitHub App installation tokens, and a canonical sample exercising the public library surface.GHE and Azure DevOps adapters compile and have unit tests, but are not wired into routing or PR monitoring. They ship as preview-only scaffolding so the next vertical can plug in without re-doing the abstraction. Tracked in follow-up issues #4, #5, #6.
What's working
RepoHostAdapterabstraction (Phases 4ΓÇô7)RepoHostAdaptertrait,RepoHostId, host-awarePrRef/events/ledger keys.CredentialStoreretiresAgentSecrets; daemon credentials reworked.docs/internals/spec-repo-hosts.md.github.com adapter ΓÇö fully wired
samples/pr-reviewer.96e29c7) and surfaced four real bugs that became the follow-up issues below.Live test fixtures (
devdev-test-envcrate)apply/verify/print-env/reset-comments.test-env/manifest.json, lockfile attest-env/manifest.lock.json.goldenwitch/devdev-test-environment(private).CI: live-tests workflow
gate→provision(admin App) →live-tests(consumer App) →cleanup(admin App, gated onprovision == success).actions/create-github-app-token@v1. No long-lived secrets in the repo.live-tests-admin/live-tests-consumer) gates blast radius.DEVDEV_LIVE_ADO_ENABLED; off until the multi-host follow-ups land.Credential leak hardening
RedactedString(daemon) andToken(test-env) newtypes redact viaDebug/Display;expose()is the only path to raw bytes.Agent-spawn consolidation (dogfood-driven)
devdev_cli::agent_command::prepare— single canonical entry-point: PATH+PATHEXT resolution → Copilot SEA-bypass rewrite → spawn-ready(program, args). Replaces three scattered call sites.devdev sendon Windows used to fail withprogram not foundbecause bareCommand::new("copilot")doesn't apply PATHEXT.samples/pr-reviewer— canonical library-surface samplecargo build --workspaceimmediately.What's preview-only (deferred)
RepoHostRegistryonly registers github.com;DispatchContext::adapter_forfalls back to it for any unregistered host. Multi-host routing is structurally incomplete. Follow-up: Multi-host routing: registry rejects unregistered hosts (no github.com fallback) #4.AzureDevOpsAdapter::get_pr_diffreturnsUnsupported, butMonitorPrTask::do_promptcalls it unconditionally. ADO PR review can't run end-to-end. Follow-up: ADO PR monitoring: get_pr_diff unimplemented + MonitorPrTask unconditionally calls it #5.RepoHostId::classify_hostonly matchesghe.*/github.*prefixes; corporate-named GHE hosts fail to classify. Follow-up: GHE host classification only matches ghe.*/github.* prefixes; custom corp hostnames fail #6.Test skips on hosted runners
live_workspace_cwd,cargo_builds_hello_world_inside_mountΓÇö host-bound (FUSE //home/agentperms), not fixture-related.