Skip to content

import: support cursor, pi, factory, codex, copilot, gemini#1540

Merged
computermode merged 11 commits into
mainfrom
additional-agents-import
Jun 29, 2026
Merged

import: support cursor, pi, factory, codex, copilot, gemini#1540
computermode merged 11 commits into
mainfrom
additional-agents-import

Conversation

@computermode

@computermode computermode commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

https://entire.io/gh/entireio/cli/trails/676

What

Extends entire import (PR #1527, Claude Code only) to the other supported agents: Cursor, Pi, Factory AI Droid, Codex, Copilot CLI, and Gemini CLI. Each entire import <agent> subcommand now surfaces automatically. Supports #1336.

OpenCode is intentionally out of scope: it has no on-disk transcript Entire can read (transcripts only materialize via opencode export), and its sessions aren't repo-attributable.

How

PR #1527 built the agent-agnostic orchestration (discovery loop, idempotent per-turn IDs, redaction, checkpoint write) behind an Importer seam. This adds one self-contained Importer per agent and registers them in the importers slice — no command wiring, since import_cmd.go iterates agentimport.All().

Each importer reuses its agent's existing transcript methods rather than re-encoding parsing:

Agent Granularity Tokens / Model
cursor per-turn none (nil tokens, empty model — records neither)
pi per-turn CalculateTokenUsage + ExtractModel (branch-aware)
factoryai-droid per-turn subagent-aware CalculateTotalTokenUsage; model from adjacent .settings.json
codex per-turn cumulative-usage delta; global store, filtered by session_meta.cwd
copilot-cli per-turn CalculateTokenUsage; flat store, filtered by session.start cwd/gitRoot
gemini per-session single JSON doc, message-index offsets → one checkpoint/session

Codex and Copilot share a repoMatches helper (equal-or-descendant, symlink-normalized) so their global/flat stores only import the current repo's sessions.

Imported checkpoints keep the merged Claude behavior: commit-less, Kind:"imported", [imported] label, not rewindable, idempotent per sha256(sessionID + "/" + turnUUID).

No changes to claude.go, Run, the Importer interface, or Turn.

Testing

  • Per-agent unit tests: Discover (lookback / --session filter / repo-match) and SplitTurns (turn boundaries, prompt/model/tokens; Gemini single-turn).
  • Registry test asserts all seven importers register with distinct names.
  • End-to-end Run test through the Cursor importer (nil tokens / empty model) confirms the generic pipeline + idempotency.
  • mise run lint (0 issues) and the import/agentimport test suites pass locally. Verified against sample fixtures: cursor 3 turns / pi 2 / factory 2 / codex 2 / copilot 2 / gemini 1.

🤖 Generated with Claude Code


Note

Medium Risk
Imports local transcripts into persistent checkpoints with format-specific parsing and repo attribution; mistakes could mis-attribute sessions or miscount tokens, but scope is additive with strong test coverage and no auth/data-path changes.

Overview
Extends entire import beyond Claude Code by registering six new Importer implementations (Cursor, Pi, Factory AI Droid, Codex, Copilot CLI, Gemini CLI) in the static importers slice—existing command wiring via agentimport.All() picks them up automatically.

Each agent gets its own Discover (30-day lookback, optional session filter, agent-specific paths) and SplitTurns logic, mostly delegating tokens/model to existing agent helpers. Codex and Copilot filter global/flat session stores with shared repoMatches (cwd/gitRoot under repo, symlink-normalized). Gemini differs: one checkpoint per session with message-index offsets, not per user turn. Cursor imports with nil tokens and empty model.

Adds per-agent unit tests, a registry test for all seven names, shared test fixtures, and a Run end-to-end test through the Cursor importer for idempotency and checkpoint writes without token/model data.

Reviewed by Cursor Bugbot for commit 2045376. Configure here.

Extends `entire import` beyond Claude Code to the other supported agents.
PR #1527 built the agent-agnostic orchestration (discovery loop, idempotent
per-turn IDs, redaction, checkpoint write) behind an Importer seam; this adds
one self-contained Importer per agent and registers them, so each
`entire import <agent>` subcommand surfaces automatically.

Each importer reuses its agent's existing transcript methods rather than
re-encoding parsing:
- cursor: shared transcript.ExtractUserContent; no native token/model data
- pi: CalculateTokenUsage + ExtractModel (branch-aware), <ts>_<uuid> filenames
- factoryai-droid: subagent-aware CalculateTotalTokenUsage; model from the
  adjacent .settings.json
- codex: walks the global sessions tree, filters by session_meta cwd, uses the
  cumulative-usage delta for per-turn tokens
- copilot-cli: flat session-state store, filters by session.start cwd/gitRoot
- gemini: single JSON document, imported per-session (message-index offsets)

Codex and Copilot share a repoMatches helper (equal-or-descendant, symlink
normalized) so their global/flat stores only import this repo's sessions.
Imported checkpoints keep the merged Claude behavior: commit-less,
Kind "imported", not rewindable, idempotent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: b66e383bcad8
Copilot AI review requested due to automatic review settings June 26, 2026 18:59

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 2045376. Configure here.

Comment thread cmd/entire/cli/agentimport/pi.go Outdated
Comment thread cmd/entire/cli/agentimport/pi.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the cmd/entire/cli/agentimport registry to support importing local transcripts for additional agents (Cursor, Pi, Factory AI Droid, Codex, Copilot CLI, Gemini CLI) through the existing entire import <agent> orchestration.

Changes:

  • Added six new per-agent Importer implementations (discover + turn splitting) for Cursor, Pi, Factory AI Droid, Codex, Copilot CLI, and Gemini CLI.
  • Added per-agent unit tests for Discover/SplitTurns, plus a registry test asserting all supported importers are present and uniquely named.
  • Added an end-to-end Run test exercising the generic pipeline via Cursor (nil tokens / empty model) and verifying idempotency.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
cmd/entire/cli/agentimport/agentimport.go Registers the six new importers in the static importer registry.
cmd/entire/cli/agentimport/agentimport_test.go Adds a “all supported agents” registry test and a Cursor-based Run end-to-end test.
cmd/entire/cli/agentimport/cursor.go Cursor transcript discovery + per-user-prompt turn splitting (nil tokens/model).
cmd/entire/cli/agentimport/cursor_test.go Tests Cursor discovery (flat + nested) and turn boundaries.
cmd/entire/cli/agentimport/pi.go Pi transcript discovery + per-user-prompt splitting with tokens/model extraction delegation.
cmd/entire/cli/agentimport/pi_test.go Tests Pi discovery, per-turn token scoping, and prompt detection rules.
cmd/entire/cli/agentimport/factory.go Factory AI Droid discovery + splitting; token usage delegated to agent; model from settings file.
cmd/entire/cli/agentimport/factory_test.go Tests Factory discovery and per-turn tokens/model from settings.
cmd/entire/cli/agentimport/codex.go Codex recursive discovery (repo-attribution via session_meta.cwd) + per-turn splitting and token deltas.
cmd/entire/cli/agentimport/codex_test.go Tests Codex repo filtering, recursive lookback, and token delta scoping.
cmd/entire/cli/agentimport/copilot.go Copilot global-store discovery with repo attribution from session.start, plus per-turn splitting.
cmd/entire/cli/agentimport/copilot_test.go Tests Copilot repo filtering + per-turn prompts/tokens/model behavior.
cmd/entire/cli/agentimport/gemini.go Gemini discovery and per-session (single turn) import behavior for JSON transcripts.
cmd/entire/cli/agentimport/gemini_test.go Tests Gemini lookback/filtering and one-checkpoint-per-session behavior.
cmd/entire/cli/agentimport/fixtures_test.go Shared test fixture constants to reduce duplication across importer tests.

Comment thread cmd/entire/cli/agentimport/pi.go Outdated
Comment thread cmd/entire/cli/agentimport/copilot.go Outdated
computermode and others added 3 commits June 26, 2026 12:10
- pi: extract the model over the [0,end) prefix (the same buffer token usage
  uses) instead of the [start,end) slice. Pi's branch-aware ExtractModel walks
  parentId back to the root, so the slice must keep the file's beginning;
  truncating only the end preserves those chains while still bounding the turn.
  Fixes wrong/empty per-turn model on branched sessions and when a turn's own
  assistant message omits the model. Adds a regression test.
- copilot: scan events.jsonl line-by-line and stop at the first session.start
  instead of reading the whole file and allocating via splitRawLines.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: 0b1a3d9f7f74
Replace the bare 10*1024*1024 scanner-buffer magic number in
copilotSessionInRepo with a documented maxCopilotLineBytes constant.
No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: b0226781fa86
Add discoverSessionFiles (discover.go), a single home for the discovery rules
every flat-directory importer repeated: resolver-rejected entries skipped,
session-ID filter, lookback-window cutoff by transcript modtime, and path
sort. Route claude, cursor, pi, factory, gemini, and copilot through it; each
Discover drops from a ~30-line loop to a ~10-line call with a small resolver
(jsonlSessionResolver for the <stem><ext> agents, a closure for cursor's
nested layout and copilot's per-repo session.start filter).

codex keeps its own Discover — it walks a recursive date-sharded tree and
filters by session_meta cwd, which the flat helper does not model.

Pure refactor: discovery behavior and results are unchanged (verified against
the importtest fixtures and the existing per-agent Discover tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: 01c03669fe4d
@computermode computermode marked this pull request as ready for review June 26, 2026 19:51
@computermode computermode requested a review from a team as a code owner June 26, 2026 19:51
Comment thread cmd/entire/cli/agentimport/factory.go Outdated
computermode and others added 4 commits June 26, 2026 14:56
Add splitLineTurns (linesplit.go): the per-turn scaffolding every JSONL
importer repeated — scan for user-prompt starts, bound each turn by the next,
truncate to the [0,end) buffer the agents' token helpers consume, and assemble
the Turn. Each importer now passes an isPrompt predicate and a build callback
that fills the agent-specific fields by calling that agent's own parsing
(CalculateTokenUsage/CalculateTotalTokenUsage, ExtractModel, ExtractUserContent,
pijsonl, etc.); the duplicated loops are gone.

Routes claude, cursor, pi, factory, codex, and copilot through it. Net ~125
fewer lines across the importers. gemini stays per-session (single JSON
document, not newline-delimited).

The per-format prompt detectors remain: per-turn import needs each prompt's
line offset (to bound token usage in the agent's native offset space), which no
agent method exposes — so detection still scans raw lines. Pure refactor;
behavior unchanged, verified against the importtest fixtures and existing tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: 873ca152414f
After merging main, the golangci config no longer flags Get's interface return
for ireturn, so the //nolint:ireturn directive is unused and nolintlint fails
CI. Remove it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: d651e65d45a9
ireturn does flag Get's interface return under the CI golangci config; the
earlier removal was based on a stale local lint cache that reported the
directive unused. Restore it so CI lint passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: e8bc62ae8f41
Droid envelopes carry no per-message timestamp (the agent stamps events with
time.Now() at hook time), so imported factory turns had a zero CreatedAt. Fall
back to the transcript file's modtime, matching the Gemini importer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: 7897bb6707d0
pfleidi
pfleidi previously approved these changes Jun 26, 2026
Factor the last cross-importer duplication into shared helpers:
- resolveDir (discover.go): the overridePath-or-GetSessionDir block every
  Discover repeated; each Discover now resolves the dir in one line.
- parseTimestamp (linesplit.go): the RFC3339-or-zero parse used by cursor, pi,
  copilot, claude, and codexLineTime.

With discoverSessionFiles, splitLineTurns, resolveDir, and parseTimestamp, all
the agent-agnostic plumbing is now shared; what remains per importer is the
genuinely format-specific part — prompt-line detection and which agent
token/model parsing to call (per-turn import needs each prompt's native-space
line offset, which no agent method exposes, so detection can't be reused from
the agent layer). Pure refactor; behavior unchanged, verified against the
importtest fixtures and existing tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: 5041f405df33
Soph
Soph previously approved these changes Jun 28, 2026
Comment thread cmd/entire/cli/agentimport/cursor.go Outdated
Comment thread cmd/entire/cli/agentimport/copilot.go Outdated
Cursor transcript lines carry only role + message — no per-turn uuid or
timestamp — so DeriveCheckpointID collided on an empty UUID and every turn
after the first was skipped as a duplicate, with CreatedAt always zero. Key
each turn by its line index (the Codex pattern) and fall back to the file
modtime for CreatedAt (the Factory/Gemini pattern). Fixtures now match the
real on-disk format.

Copilot timestamps may be numeric epoch-millis, not only RFC3339 strings.
Decoding into a string field made json.Unmarshal fail on numeric values,
hitting the defensive skip and importing zero turns for the session. Decode
the timestamp as json.RawMessage and parse it with the agent's dual-format
parser (exported as copilotcli.ParseTimestamp).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Entire-Checkpoint: 217723121355
@computermode computermode merged commit a24b73b into main Jun 29, 2026
9 checks passed
@computermode computermode deleted the additional-agents-import branch June 29, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants