feat(copilot): GitHub Copilot CLI provider (preview) — forward + reverse, verified against the live CLI#117
Open
akesling wants to merge 18 commits into
Open
feat(copilot): GitHub Copilot CLI provider (preview) — forward + reverse, verified against the live CLI#117akesling wants to merge 18 commits into
akesling wants to merge 18 commits into
Conversation
Adds toolpath-copilot 0.1.0, a forward provider that derives Toolpath documents from GitHub Copilot CLI (@github/copilot) sessions under ~/.copilot/session-state/<id>/events.jsonl. - Tolerant events.jsonl parser (schema reverse-engineered, authored without first-hand samples): session/user/assistant/tool/subagent/skill/hook events -> ConversationView -> shared derive_path. Tool names classified into the ToolCategory ontology; file writes with full content get a raw diff. - workspace.yaml git context (root/repository/branch/revision) -> Path.base, via a tolerant key-scan parser (no YAML dependency). - CLI forward path only: path p import/list/show copilot (mirrors codex). - 8-doc on-disk format reference at docs/agents/formats/copilot-cli/, with every claim confidence-tagged (official / reverse-eng / inferred / unverified) and a verify-once-we-have-samples checklist. - Bumps path-cli 0.14.0 -> 0.15.0 (new dep + subcommands). Preview: no projector (so no path p export copilot / path resume / path share) and the events.jsonl schema is unverified against a real session -- see docs/agents/formats/copilot-cli/known-gaps-and-sourcing.md.
…ting The reverse-engineered events.jsonl schema never confirmed a correlation id on tool.execution_* events (sources report only name/args/success). The prior logic paired solely by id, so an id-less complete fell through to a synthesized carrier — double-counting every tool call and duplicating file mutations. attach_tool_result now treats an id as authoritative when present, and falls back to positional pairing (most-recent result-less invocation in the open turn, preferring the same tool name) when absent. Adds tests for the id-less case and a regression guard for the id-bearing case; documents the correlation uncertainty in events.md + the verification checklist.
….0.67)
Verified the reverse-engineered events.jsonl format against a first-hand
capture and corrected several guessed field locations that lost data:
- session.start: cwd + git context are under data.context.{cwd,gitRoot,
repository,branch,headCommit}, not top-level; CLI version is copilotVersion
(top-level `version` is an int schema version). Base now carries the commit.
- tool.execution_complete: result text is under data.result.content (an
object), not a top-level string -- tool results were previously dropped.
- assistant.message: capture reasoningText (-> Turn.thinking) and per-message
outputTokens (summed as the session output total when no session.shutdown).
- tool correlation is via toolCallId (confirmed); positional fallback retained.
Also: fix a flaky test (strict-mode reader test mutated a process-global env
var, racing the parallel non-strict test -- now threads `strict` explicitly);
reshape all fixtures to the real envelope; refresh the docs folder from
[inferred]/[unverified] to [observed, 1.0.67] with resolved-vs-open gaps.
Verified live: `path show/import copilot` against the real session yields base
{uri, ref=<commit>, branch}, bash/view tool results (1574/223 chars), and
thinking populated. Envelope + core event types confirmed; subagent/skill/hook/
abort/shutdown/compaction and checkpoints/ remain unverified (not exercised).
`path share` is a separate orchestrator from the `p import/list/show` plumbing and didn't know about Copilot, so Copilot sessions never appeared in it. Its forward direction (derive -> upload to Pathbase) needs no projector, so: - Add Harness::Copilot / HarnessArg::Copilot and thread it through name/symbol/ parse/from_arg/harness_to_arg, the HarnessBundle, the gather-sessions probe (collect_copilot), the not-found classifier, and the "no sessions" status summary. - derive_session dispatch -> new cmd_import::derive_copilot_session. - Resume targets: argv_for gets `--resume <id>`; project_into_harness bails with a clear "no projector yet" message (resume into Copilot can't work without the reverse mapping). Adds share unit tests (gather includes/filters Copilot) and updates the docs (CLAUDE.md, CHANGELOG, crate README) to reflect share support. Verified live: `path share --harness copilot --session <real>` against a dead URL derives the real session and fails only at upload (Connection refused) -- i.e. share now surfaces Copilot end-to-end. path-cli 296 unit + 69 integration pass; clippy clean.
…ader
Adds the reverse path (ConversationView -> Copilot events.jsonl), wires Copilot
into resume/export and the cross-harness conformance matrix, and fixes the
first real-CLI resume rejection.
- New `CopilotProjector` (project.rs): emits the observed events.jsonl shape
(session.start context, user/assistant.* with reasoningText + per-message
tokens, tool.execution_* with result.content, subagent.*), remapping foreign
tool names via a new `native_name`. Envelope `id`/`parentId` are UUID strings
— the real `copilot --resume` rejects non-UUID ids ("invalid session event
envelope: `id` must be a UUID string"), reported from a live run.
- Forward provider carries per-turn token_usage (summed for the total) for
cross-harness token survival, and assigns position-stable turn ids so the
parent graph is idempotent.
- path-cli: `path resume` / `path p export copilot` via `project_copilot`
(writes session-state/<id>/{events.jsonl,workspace.yaml} + a session-store.db
`sessions` row, fresh uuid, INSERT-only). Fixes the resume picker (Copilot was
missing from ALL_HARNESSES; infer_source_harness now knows "copilot"). uuid v4.
- Cross-harness conformance: `CopilotHarness` in cross_harness_matrix
(test-fixtures/copilot/) — all cells pass. New resume integration test.
⚠️ Real `copilot --resume` acceptance is still being verified against live runs;
the envelope-id fix clears the first reported error. Projector otherwise
validated by the matrix + round-trip.
Second live `copilot --resume` rejection: "`timestamp` must be an ISO 8601 date-time string with a timezone offset". The projector stamped session.start with an empty timestamp (omitted) and passed through turn timestamps unchecked. - Projector now stamps every event (session.start included) with a valid offset-bearing RFC 3339 timestamp: picks a base (first offset-bearing turn ts, else the view's started_at) and normalizes each turn's timestamp against it. - Document the discovered loader contract: new docs/agents/formats/copilot-cli/writing-compatible.md (UUID envelope ids + offset timestamps, with the verbatim rejection messages), linked from the folder README and known-gaps. Updated as new rejections surface. Verified: projected session.start now carries id=<uuid> and timestamp=2026-07-01T14:31:29.298Z. Crate 59 + matrix + resume tests green.
akesling
force-pushed
the
copilot-provider
branch
from
d84a724 to
2ad38c2
Compare
|
🔍 Preview deployed: https://aec6a98e.toolpath.pages.dev |
Third live `copilot --resume` rejection: "`parentId` must be a UUID string or null" — the root session.start omitted parentId entirely. The projector now always emits parentId: the previous event's UUID, or null on the root. - project.rs: parentId always present (UUID | null). - docs: writing-compatible.md req 3 upgraded to [observed, 1.0.67] with the verbatim error. Verified: projected session.start now has id=<uuid>, parentId=null, timestamp=<offset ISO>. Crate + matrix + resume tests green.
…el shape)
Fourth live `copilot --resume` rejection: "line 1: missing field `startTime`".
The projected session.start `data` omitted it.
- project.rs: session_start_data now emits `startTime` (offset ISO, same base
timestamp as the envelope) plus the rest of the observed 1.0.67 top-level set
(contextTier, alreadyInUse, remoteSteerable) and richer context
(hostType/repositoryHost/baseCommit) — matching the real session.start so the
loader's one-field-at-a-time check doesn't keep rejecting.
- docs: writing-compatible.md req 5 [observed, 1.0.67] with the verbatim error.
Verified: projected session.start data keys now =
{sessionId, version, producer, copilotVersion, startTime, contextTier, context,
alreadyInUse, remoteSteerable}. Crate 59 + matrix + resume tests green.
Fifth live `copilot --resume` rejection (line 6): "missing field `turnId`".
Copilot requires a `turnId` on assistant-turn-scoped events; the projector
omitted it.
- project.rs: push_assistant takes a per-assistant-turn index ("0","1",…) and
stamps `turnId` on assistant.turn_start/.message/.turn_end and every
tool.execution_start/_complete (and subagent.*). session.start/user.message
correctly carry none.
- docs: writing-compatible.md req 6 [observed, 1.0.67].
The error advanced from line 1 → line 6 as earlier envelope reqs were fixed;
this clears the turn events. Crate 59 + matrix + resume tests green.
…Id; verified in 1.0.67 Two more live `copilot --resume` requirements + the verified round-trip. - messageId (req 7): assistant.message and assistant.turn_end must carry a `messageId` (UUID); turn_end references the message it closes. - toolCallId non-empty (req 8): tool.execution_* and the toolRequests mirror need a NON-EMPTY toolCallId (empty reads as "missing field"). Forward treats empty-string ids as absent; projector synthesizes a stable id when empty. Verified: drove `copilot --resume` locally (isolated COPILOT_HOME + copied config.json) — a projected real session LOADS and resumes (full context). Flipped docs (crate README / CLAUDE.md / CHANGELOG / writing-compatible.md) to verified-in-1.0.67 with the 8-requirement loader contract (verbatim rejections). Crate 60 + matrix + resume tests green; clippy clean.
…ion resumes The user's failing doc was a Pathbase graph (resumed by URL) containing sub-agents. `copilot --resume` rejected `subagent.started` line-by-line for missing fields. Self-drove the loop locally (isolated COPILOT_HOME + copied config.json, real `copilot --resume -p`) and cleared them all: - subagent.started / subagent.completed now carry `toolCallId` (synthesized, stable per delegation), `agentName` + `agentDisplayName` (from the delegation id), and `agentDescription` — in addition to the existing `id`/`prompt`/ `result`/`turnId`. Verified end to end: the exact 5817-event Pathbase graph now LOADS and resumes in real copilot 1.0.67 (full context, ↑99.6k tokens). writing-compatible.md req 9 added; CLAUDE.md / CHANGELOG updated (contract now 9 requirements, verified on small + large sub-agent sessions). Crate 60 + matrix + resume tests green; clippy clean.
…s render
The user reported edit tool calls not displaying a diff in a resumed session.
Referencing a real native copilot `edit`/`create` tool call: Copilot renders
the change from `result.detailedContent` (a git-style unified diff), on tools
named `edit` (args {path, old_str, new_str}) and `create` (args {path,
file_text}).
- native_name(FileWrite) now returns `edit`/`create` (was edit_file/create_file).
- CopilotProjector detects FileWrite tool calls and re-emits them in the native
shape: reshapes args (Claude Edit/Write -> old_str/new_str/file_text), emits a
result with a summary `content` + a git-style `detailedContent` diff, and
always emits the complete (the diff is the point) even when the IR had none.
- docs: file-fidelity.md "Reverse (projection): making an edit render"; CHANGELOG.
Verified: the Pathbase graph's edits project to toolName=edit with a
`diff --git …` detailedContent, and the session still loads in real copilot
1.0.67. Crate 61 + matrix + resume tests green; clippy clean.
Follow-up: the projected edit showed a flat (uncolorized) diff vs the native edit's colorized one. The native `edit`/`create` complete carries a `toolTelemetry` block whose `codeBlocks` (stringified JSON with fileExt + languageId + line counts) tells Copilot's UI to render a colorized diff view; `metrics.linesAdded/linesRemoved` give the +N/-M summary. Mine omitted it. - file_write_projection now builds `toolTelemetry` (properties/metrics/ restrictedProperties) matching the observed 1.0.67 shape: languageId derived from the path extension, line counts from the diff. - docs: file-fidelity.md; test asserts the telemetry. Verified: projected edit complete now carries toolTelemetry.codeBlocks with the right languageId + line counts, and the session still loads in copilot 1.0.67. Crate 61 + clippy clean.
The projected edit diff still showed flat (uncolorized). Root cause: the `detailedContent` had a duplicate/empty header — `toolpath_convo::unified_diff` prepends its own `--- a/<path>` / `+++ b/<path>` on top of `similar`'s empty-filename `--- `/`+++ ` header, producing a malformed diff Copilot can't parse into a colorized view. - git_diff now builds the diff with `similar` directly (single header, correct `a/dev/null` for creates). Added `similar` dep to toolpath-copilot. - test asserts exactly one file header (no stray `--- `/`+++ ` lines). - docs: file-fidelity.md. Verified: projected edit detailedContent is now a clean single-header `diff --git …` and the session still loads in copilot 1.0.67. Crate 61 + clippy clean.
Diagnosed by capturing Copilot's actual TUI rendering through a pty harness (answering its terminal queries, auto-accepting folder trust, ctrl+o to expand the timeline) and comparing a resumed NATIVE session against the same session re-projected through our pipeline, byte for byte: - Our diffs were already driving the colorized machinery (green/red +N/-M counts, colorized hunk rows) — q7/ske/Y7 in the app bundle all accepted them. - The "flat diff text" was ONE case: a hunkless diff. Creating an EMPTY file produces a ""→"" text diff with headers but no @@ hunk; Copilot's diff view renders parsed hunk rows and hides headers, but with nothing to parse it dumps `diff --git`/`index`/mode lines as raw text. Fixes: - git_diff: an empty-file create now diffs to one added empty line (@@ -1,0 +1,1 @@ / +), matching the native tool's rendering (`+1`). - FileWrite.detailed is now Option: any other hunkless diff omits detailedContent entirely instead of leaking headers. - docs: file-fidelity.md — hunk requirement [observed via pty capture, 1.0.68] + note that the TUI recomputes +N/-M from the diff (arg-diff vs file-state nuance explains +1 vs +1 -1). Verified via pty capture of the re-projected session: 0 raw header lines, `Create foo.txt +1` / `Edit foo.txt +1` colorized, expanded body renders the colorized `1 +` rows. Crate 61 + matrix + resume green; clippy clean.
…diffs render
The colorized diff still didn't render for Claude-origin docs. Reproduced with
the user's exact Pathbase graph (sliced to its first edit, projected, and
captured through the pty TUI harness): the row rendered as generic "● edit"
with file_path/old_string args and a flat markdown diff.
Root cause: Copilot's timeline UI builds tool rows from the
assistant.message.toolRequests MIRROR, not tool.execution_start. The mirror
still carried raw IR args (Claude's file_path/old_string), so arguments.path
was absent and the editor-family row (title "Edit <path>", +N/−M counts, Y7
colorized diff body) never engaged — the generic row markdown-rendered the
diff as flat text ("basic syntax highlighting").
- New projected_tool(): one (name, arguments) remap per tool call, used by
BOTH the mirror and tool.execution_start (they must agree). File writes →
edit/create; file reads → view with path + view_range from offset/limit.
- Regression test: mirror carries remapped args (path present, file_path
absent; view_range mapped) and execution_start agrees.
- docs: file-fidelity.md — the mirror is what the UI renders from.
Verified via pty capture on the user's doc: colorized line-numbered hunk rows
(1 -, 1 +, 3 +) render; zero raw-header/generic-arg leaks. Crate 62 + matrix +
resume green; clippy clean.
…ings Audit found five gaps between the docs and what the last debugging arc established; close them: - project.rs module doc: stale "resume unverified" -> verified (1.0.67/1.0.68, incl. sub-agent session), pointing at the two contract docs. - events.md: new "Native tool vocabulary" section (bash/view/edit/create arg shapes + result strings, observed); the assistant.message row now warns that the resumed-timeline UI renders tool rows from the toolRequests MIRROR, so a writer must keep the mirror in native vocabulary. - known-gaps-and-sourcing.md: new "Verification methodology" section (the isolated-COPILOT_HOME loader loop + the pty TUI-capture technique with terminal-query replies, trust auto-accept, ctrl+o, and bundle grepping); loader contract + rendering contract + tool vocabulary moved from open questions to resolved. - writing-compatible.md: "loading is necessary but not sufficient" pointer to the rendering contract. - folder README: revision 2026-07-02; grounding now spans 1.0.67 capture + 1.0.67-1.0.68 live loader/rendering verification.
…atrix, live probe
Replaces synthetic coverage with real-capture-driven tests. Ran the
feature-elicit prompt through the live copilot CLI (isolated COPILOT_HOME) —
all 9 tasks incl. a real sub-agent. Sanitized capture is the matrix fixture +
crate tests/fixtures/real-session.jsonl. capture-elicit-fixtures.sh +
feature-elicit.md learned copilot.
Parser fixes surfaced by the capture (all tested):
- session.shutdown real shape: tokenDetails.{...}.tokenCount + model-keyed
modelMetrics (reverse-eng usage.inputTokens was wrong); output == Σ
per-message outputTokens (verified).
- subagent.* are thin markers sharing the task tool's toolCallId; delegations
pair by it and the projector preserves the id (was synthesizing — broke
idempotency and cross-harness survival).
- Codex-grade file fidelity: edit/create completes embed the real file-state
diff in result.detailedContent; forward upgrades FileMutation.raw_diff.
- toolpath-pi: decode zero wire `input` as None (same absence rule as cache).
Tests: real_fixture_roundtrip.rs (forward invariants, projection round-trip,
wire serde value-identity on all 64 lines); matrix on the REAL fixture with
tokens + sub-agent (all cells green); scripts/verify-copilot-live.sh (loader +
context probe — resumed model answered session-specific facts correctly).
Docs synced (events.md, file-fidelity.md, known-gaps, CLAUDE.md). Full
workspace: 61 test binaries green, clippy clean.
akesling
force-pushed
the
copilot-provider
branch
from
2ad38c2 to
e6021aa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
toolpath-copilot, a preview provider for the GitHub Copilot CLI(
@github/copilot), wired both directions and verified against the real CLIat
copilotVersion1.0.67–1.0.68.path p import / list / show copilot,path share.Reads the
~/.copilot/session-state/<id>/events.jsonlstream (root overridablevia
COPILOT_HOME) into aConversationView→Path.CopilotProjectorpowerspath p export copilotand
path resume, writing a session back to disk (+ asession-store.dbsessionsrow) that the realcopilot --resumeloads.Verified against the live CLI ✅
The
events.jsonlschema is undocumented, so the contract was mappedempirically and is documented with verbatim evidence in
docs/agents/formats/copilot-cli/:writing-compatible.md) — 9 requirements, each with itsverbatim
Session file is corrupted (line N: …)rejection: UUIDid/parentId,offset-ISO timestamps,
turnId/messageId, non-emptytoolCallId, fullsession.startshape, and thesubagent.*fields. Verified end-to-end on a27-event session and a 5817-event session with sub-agents.
file-fidelity.md) — found by capturing the real TUIthrough a pty harness and cross-checking the app bundle: the timeline renders
tool rows from the
assistant.message.toolRequestsmirror (so file/readtool calls are remapped to Copilot's native
edit/create/viewvocabulary),and colorized diffs require a single-header, ≥1-hunk unified diff in
result.detailedContent+ atoolTelemetry.codeBlocksblock. Result: resumedsessions show proper
Edit <path>rows with colorized+N −Mdiffs.Testing (S-tier, real-capture-driven)
copilotCLI(shell, create/edit/view, glob+grep, errored read, a real sub-agent, reasoning,
per-message + shutdown tokens) — sanitized and committed as both the crate
fixture and the cross-harness matrix fixture.
real_fixture_roundtrip.rs: forward count/category/token/delegation/mutationinvariants, full projection round-trip fidelity through the real reader, and
wire-level serde value-identity on all 64 real lines.
harnesses (idempotency + schema validation), on the real fixture with tokens
and sub-agent included — all cells green.
edit/createcompletes embed the realfile-state diff inline (
result.detailedContent); the forward path upgradesFileMutation.raw_diffwith it.scripts/verify-copilot-live.sh: isolated-home loader check + resumed-modelcontext probe (the probe answers session-specific questions correctly,
proving full context transfer).
Docs
Eight focused references under
docs/agents/formats/copilot-cli/(README,directory-layout, session-state, events, session-store-db, file-fidelity,
writing-compatible, known-gaps-and-sourcing), plus the standard version-bump
touchpoints (CLAUDE.md, README, CHANGELOG, site data, release script).
Full workspace: 61 test binaries green, clippy clean.