Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ All notable changes to the Toolpath workspace are documented here.

- `toolpath-pi` 0.1.0: new crate — reads Pi (pi.dev) coding-agent session JSONL logs, implements `ConversationProvider`, and derives Toolpath `Path` documents via `toolpath-convo`'s shared derivation (`toolpath_convo::derive_path`). Reads from `~/.pi/agent/sessions/` by default; base directory is configurable. Preserves Pi's in-file conversation tree (id/parentId) as a DAG in the derived `Path`, and follows `parentSession` links across session files (bounded depth). CLI subcommands planned: `path derive pi` and `path list pi` (wiring may be merged separately).
- `toolpath-claude` 0.7.0: `ClaudeProjector` for projecting `ConversationView` back to Claude `Conversation`. Enriched derive: full text, tool invocation steps, `agent://` URNs, token usage, tool results via cross-entry assembly, `conversation.init` steps.
- `toolpath-cli` 0.3.1: `path project claude` and `path incept` commands for projecting toolpath documents into Claude sessions.
- `toolpath-gemini` 0.1.0: new crate — reads Gemini CLI conversation logs from `~/.gemini/tmp/<project>/chats/`, implements `ConversationProvider`, and derives Toolpath `Path` documents. `PathResolver` supports both friendly-name (`projects.json`) and SHA-256 hash-slot layouts. Sub-agent chat files (`kind: "subagent"`) are folded into `DelegatedWork` on the parent `task` tool invocation, with `turns` populated from the sub-agent's messages. Polling-based `ConversationWatcher` (feature `watcher`, default on) emits `Turn` / `TurnUpdated` / `Progress { kind: "subagent_started" | "subagent_complete" }` events. Guarantees round-trip fidelity at the `ChatFile` layer via `Option<Vec<T>>` for absent-vs-empty preservation, `GeminiRole::Other(String)` catch-all, `Option<Value>` on polymorphic `resultDisplay`, and `#[serde(flatten)] extra` at chat and message levels. 163 unit + 12 integration + 4 doc tests.
- `toolpath-cli` 0.3.1: `path project claude` and `path incept` commands for projecting toolpath documents into Claude sessions; `derive gemini --project PATH [--session UUID] [--all] [--include-thinking]` and `list gemini [--project PATH] [--json]` subcommands.
- `toolpath-desktop` 0.1.0: new crate — Tauri 2 desktop app for non-technical users. Source discovery for Claude Code + Pi + local git + GitHub PRs; interactive DAG preview (d3 + dagre-d3, Svelte 5 + TypeScript frontend); local `.path.json` export; stubbed Pathbase upload. GitHub PAT stored in the OS keychain under `dev.pathbase.toolpath-desktop`. Hot-reloading dev loop via `cargo tauri dev` (spawns Vite on port 1420).

## 0.3.0 — toolpath-cli
Expand Down
13 changes: 10 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ crates/
toolpath-git/ # derive from git repos (git2)
toolpath-github/ # derive from GitHub pull requests (REST API)
toolpath-claude/ # derive from Claude conversation logs
toolpath-gemini/ # derive from Gemini CLI conversation logs
toolpath-pi/ # derive from Pi (pi.dev) agent session logs
toolpath-dot/ # Graphviz DOT rendering
toolpath-md/ # Markdown rendering for LLM consumption
Expand All @@ -36,6 +37,7 @@ toolpath-cli (binary: path)
├── toolpath-git → toolpath
├── toolpath-github → toolpath
├── toolpath-claude → toolpath, toolpath-convo
├── toolpath-gemini → toolpath, toolpath-convo
├── toolpath-pi → toolpath, toolpath-convo
├── toolpath-dot → toolpath
└── toolpath-md → toolpath
Expand All @@ -44,7 +46,7 @@ toolpath-desktop (binary: toolpath-desktop, Tauri 2 app)
├── toolpath, toolpath-claude, toolpath-git, toolpath-github
```

Cross-dependencies between satellite crates: `toolpath-claude → toolpath-convo`, `toolpath-pi → toolpath-convo`. `toolpath-desktop` is a leaf — nothing depends on it.
Cross-dependencies between satellite crates: `toolpath-claude → toolpath-convo`, `toolpath-gemini → toolpath-convo`, `toolpath-pi → toolpath-convo`. `toolpath-desktop` is a leaf — nothing depends on it.

## Build and test

Expand All @@ -64,6 +66,7 @@ The binary is called `path` (package: `toolpath-cli`):
cargo run -p toolpath-cli -- derive git --repo . --branch main --pretty
cargo run -p toolpath-cli -- derive github --repo owner/repo --pr 42 --pretty
cargo run -p toolpath-cli -- derive claude --project /path/to/project
cargo run -p toolpath-cli -- derive gemini --project /path/to/project
cargo run -p toolpath-cli -- derive pi --project /path/to/project
cargo run -p toolpath-cli -- render dot --input doc.json
cargo run -p toolpath-cli -- render md --input doc.json --detail full
Expand Down Expand Up @@ -96,6 +99,7 @@ Tests live alongside the code (`#[cfg(test)] mod tests`), plus `toolpath-cli` ha
- `toolpath-git`: 33 unit + 3 doc tests (derive, branch detection, diffstat)
- `toolpath-github`: 28 unit + 2 doc tests (mapping, DAG construction, fixtures)
- `toolpath-claude`: 216 unit + 5 doc tests (path resolution, conversation reading, query, chaining, watcher, derive)
- `toolpath-gemini`: 163 unit + 12 integration + 4 doc tests (path resolution, chat-file parsing, query, watcher, derive, provider, round-trip fidelity)
- `toolpath-pi`: ~88 unit tests (types, paths, error, reader, io, provider)
- `toolpath-dot`: 30 unit + 2 doc tests (render, visual conventions, escaping)
- `toolpath-cli`: 126 unit + 24 integration tests (all commands, track sessions, merge, validate, roundtrip, render-md snapshots)
Expand All @@ -106,6 +110,7 @@ Validate example documents: `for f in examples/*.json; do cargo run -p toolpath-
## Feature flags

- `toolpath-claude` has a `watcher` feature (default: on) gating `notify`/`tokio` dependencies for filesystem watching
- `toolpath-gemini` has a `watcher` feature (default: on) gating the polling-based `ConversationWatcher` module

## toolpath-desktop

Expand Down Expand Up @@ -153,7 +158,7 @@ When changing a crate's public API (new types, new trait impls, new public metho

**Release script** (`scripts/release.sh`) publishes in dependency order:
- Tier 1: `toolpath` (no workspace deps)
- Tier 2: `toolpath-convo` (depends on `toolpath`); then `toolpath-git`, `toolpath-github`, `toolpath-dot`, `toolpath-md`, `toolpath-claude`, `toolpath-pi`
- Tier 2: `toolpath-convo` (depends on `toolpath`); then `toolpath-git`, `toolpath-github`, `toolpath-dot`, `toolpath-md`, `toolpath-claude`, `toolpath-gemini`, `toolpath-pi`
- Tier 3: `toolpath-cli` (depends on everything)

Build the site after changes: `cd site && pnpm run build` (should produce 7 pages).
Expand All @@ -165,6 +170,8 @@ Build the site after changes: `cd site && pnpm run build` (should produce 7 page
- The git derivation (`toolpath-git`) uses `git2` (libgit2 bindings), not shelling out to git
- Claude conversation data lives in `~/.claude/projects/` as JSONL files; `toolpath-claude` reads these directly
- `toolpath-claude` follows session chains by default — Claude Code rotates JSONL files on context overflow; `read_conversation` merges segments, `list_conversations` returns chain heads. `read_segment`/`list_segments` for single-file access. `ChainIndex` makes this incremental.
- Provider-specific extras convention: `Turn.extra` and `WatcherEvent::Progress.data` use provider-namespaced keys (e.g. `extra["claude"]`). `toolpath-claude` populates `Turn.extra["claude"]` from `ConversationEntry.extra` and `Progress.data["claude"]` from the full entry payload. This lets trait-only consumers access provider metadata (like `subtype` for state inference) without importing provider types.
- Gemini CLI conversation data lives in `~/.gemini/tmp/<project>/chats/`. Main sessions sit at the top (`session-<timestamp>-<short>.json`, `kind: "main"`); sub-agents live in sibling `<full-uuid>/` directories (`kind: "subagent"`). The `<project>` slot is either a friendly name from `~/.gemini/projects.json` or the SHA-256 hex of the absolute project path; `toolpath-gemini` resolves both.
- `toolpath-gemini` treats main file + sibling sub-agent UUID dir as one conversation. Sub-agent files are folded into `DelegatedWork` with populated `turns` (unlike `toolpath-claude`, whose sub-agent turns live in separate session files and stay empty). See `docs/agents/formats/gemini.md` for the full format reference.
- Provider-specific extras convention: `Turn.extra` and `WatcherEvent::Progress.data` use provider-namespaced keys (e.g. `extra["claude"]`, `extra["gemini"]`). `toolpath-claude` populates `Turn.extra["claude"]` from `ConversationEntry.extra`; `toolpath-gemini` populates `Turn.extra["gemini"]` with the full `tokens` struct, per-thought metadata, and tool-call status. This lets trait-only consumers access provider metadata without importing provider types.
- Shared derivation: `toolpath-convo` provides a provider-agnostic `ConversationView → Path` mapping via `toolpath_convo::derive_path`. New conversation providers should build on it rather than re-implementing the mapping.
- Pi provider: `toolpath-pi` reads Pi session JSONL from `~/.pi/agent/sessions/`. Sessions use a tree (id/parentId) in a single file, and may link to a parent file via `parentSession` in the header. The tree is preserved as a DAG in the derived `Path`.
24 changes: 20 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ members = [
"crates/toolpath-git",
"crates/toolpath-github",
"crates/toolpath-claude",
"crates/toolpath-gemini",
"crates/toolpath-dot",
"crates/toolpath-md",
"crates/toolpath-pi",
Expand All @@ -22,6 +23,7 @@ toolpath = { version = "0.1.5", path = "crates/toolpath" }
toolpath-convo = { version = "0.6.0", path = "crates/toolpath-convo" }
toolpath-git = { version = "0.1.3", path = "crates/toolpath-git" }
toolpath-claude = { version = "0.7.0", path = "crates/toolpath-claude", default-features = false }
toolpath-gemini = { version = "0.1.0", path = "crates/toolpath-gemini", default-features = false }
toolpath-github = { version = "0.2.0", path = "crates/toolpath-github" }
toolpath-dot = { version = "0.1.2", path = "crates/toolpath-dot" }
toolpath-md = { version = "0.2.0", path = "crates/toolpath-md" }
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ crates/
toolpath-git/ Derive from git repository history
toolpath-github/ Derive from GitHub pull requests
toolpath-claude/ Derive from Claude conversation logs
toolpath-gemini/ Derive from Gemini CLI conversation logs
toolpath-pi/ Derive from Pi (pi.dev) agent sessions
toolpath-dot/ Graphviz DOT visualization
toolpath-md/ Markdown rendering for LLM consumption
Expand Down Expand Up @@ -75,6 +76,9 @@ path derive github --repo owner/repo --pr 42 --pretty
# Derive from Claude conversation logs
path derive claude --project /path/to/project --pretty

# Derive from Gemini CLI conversation logs
path derive gemini --project /path/to/project --pretty

# Query for dead ends (abandoned approaches)
path query dead-ends --input doc.json

Expand All @@ -99,10 +103,12 @@ path
git [--repo PATH] [--remote NAME] [--json]
github --repo OWNER/REPO [--json]
claude [--project PATH] [--json]
gemini [--project PATH] [--json]
derive
git --repo PATH --branch NAME[:START] [--base COMMIT] [--remote NAME] [--title TEXT]
github --repo OWNER/REPO --pr NUMBER [--no-ci] [--no-comments]
claude --project PATH [--session ID] [--all]
gemini --project PATH [--session UUID] [--all] [--include-thinking]
query
ancestors --input FILE --step-id ID
dead-ends --input FILE
Expand Down
2 changes: 2 additions & 0 deletions crates/toolpath-cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,15 @@ rand = "0.9"

[target.'cfg(not(target_os = "emscripten"))'.dependencies]
toolpath-claude = { workspace = true, features = ["watcher"] }
toolpath-gemini = { workspace = true, features = ["watcher"] }
toolpath-pi = { workspace = true }
toolpath-convo = { workspace = true }
toolpath-github = { workspace = true }
git2 = { workspace = true }

[target.'cfg(target_os = "emscripten")'.dependencies]
toolpath-claude = { workspace = true }
toolpath-gemini = { workspace = true }
toolpath-pi = { workspace = true }

[dev-dependencies]
Expand Down
169 changes: 169 additions & 0 deletions crates/toolpath-cli/src/cmd_derive.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,24 @@ pub enum DeriveSource {
#[arg(long)]
all: bool,
},
/// Derive from Gemini CLI conversation logs
Gemini {
/// Project path (e.g., /Users/alex/myproject)
#[arg(short, long)]
project: String,

/// Specific session UUID (the directory name under chats/)
#[arg(short, long)]
session: Option<String>,

/// Process all sessions in the project
#[arg(long)]
all: bool,

/// Include thinking blocks in conversation.append text
#[arg(long)]
include_thinking: bool,
},
/// Derive from Pi (pi.dev) coding-agent session logs
Pi {
/// Project path (cwd the session ran in)
Expand Down Expand Up @@ -105,6 +123,12 @@ pub fn run(source: DeriveSource, pretty: bool) -> Result<()> {
session,
all,
} => run_claude(project, session, all, pretty),
DeriveSource::Gemini {
project,
session,
all,
include_thinking,
} => run_gemini(project, session, all, include_thinking, pretty),
DeriveSource::Pi {
project,
session,
Expand Down Expand Up @@ -222,6 +246,60 @@ fn run_claude(project: String, session: Option<String>, all: bool, pretty: bool)
run_claude_with_manager(&manager, project, session, all, pretty)
}

fn run_gemini(
project: String,
session: Option<String>,
all: bool,
include_thinking: bool,
pretty: bool,
) -> Result<()> {
let manager = toolpath_gemini::GeminiConvo::new();
run_gemini_with_manager(&manager, project, session, all, include_thinking, pretty)
}

fn run_gemini_with_manager(
manager: &toolpath_gemini::GeminiConvo,
project: String,
session: Option<String>,
all: bool,
include_thinking: bool,
pretty: bool,
) -> Result<()> {
let config = toolpath_gemini::derive::DeriveConfig {
project_path: Some(project.clone()),
include_thinking,
};

let docs: Vec<toolpath::v1::Path> = if let Some(session_uuid) = session {
let convo = manager
.read_conversation(&project, &session_uuid)
.map_err(|e| anyhow::anyhow!("{}", e))?;
vec![toolpath_gemini::derive::derive_path(&convo, &config)]
} else if all {
let convos = manager
.read_all_conversations(&project)
.map_err(|e| anyhow::anyhow!("{}", e))?;
toolpath_gemini::derive::derive_project(&convos, &config)
} else {
let convo = manager
.most_recent_conversation(&project)
.map_err(|e| anyhow::anyhow!("{}", e))?
.ok_or_else(|| anyhow::anyhow!("No conversations found for project: {}", project))?;
vec![toolpath_gemini::derive::derive_path(&convo, &config)]
};

for path in &docs {
let doc = toolpath::v1::Document::Path(path.clone());
let json = if pretty {
doc.to_json_pretty()?
} else {
doc.to_json()?
};
println!("{}", json);
}
Ok(())
}

fn run_claude_with_manager(
manager: &toolpath_claude::ClaudeConvo,
project: String,
Expand Down Expand Up @@ -581,4 +659,95 @@ mod tests {
.contains("No conversations found")
);
}

fn setup_gemini_manager() -> (tempfile::TempDir, toolpath_gemini::GeminiConvo) {
let temp = tempfile::tempdir().unwrap();
let gemini = temp.path().join(".gemini");
let session_dir = gemini.join("tmp/myrepo/chats/session-uuid");
std::fs::create_dir_all(&session_dir).unwrap();
std::fs::write(
gemini.join("projects.json"),
r#"{"projects":{"/abs/myrepo":"myrepo"}}"#,
)
.unwrap();
std::fs::write(
session_dir.join("main.json"),
r#"{"sessionId":"s","projectHash":"","startTime":"2026-04-17T10:00:00Z","lastUpdated":"2026-04-17T10:10:00Z","directories":["/abs/myrepo"],"messages":[
{"id":"u1","timestamp":"2026-04-17T10:00:00Z","type":"user","content":[{"text":"Hello"}]},
{"id":"a1","timestamp":"2026-04-17T10:00:01Z","type":"gemini","content":"Hi","model":"gemini-3-flash-preview"}
]}"#,
)
.unwrap();
let resolver = toolpath_gemini::PathResolver::new().with_gemini_dir(&gemini);
(temp, toolpath_gemini::GeminiConvo::with_resolver(resolver))
}

#[test]
fn test_run_gemini_session() {
let (_t, mgr) = setup_gemini_manager();
let result = run_gemini_with_manager(
&mgr,
"/abs/myrepo".to_string(),
Some("session-uuid".to_string()),
false,
false,
false,
);
assert!(result.is_ok());
}

#[test]
fn test_run_gemini_most_recent() {
let (_t, mgr) = setup_gemini_manager();
let result =
run_gemini_with_manager(&mgr, "/abs/myrepo".to_string(), None, false, false, true);
assert!(result.is_ok());
}

#[test]
fn test_run_gemini_all() {
let (_t, mgr) = setup_gemini_manager();
let result =
run_gemini_with_manager(&mgr, "/abs/myrepo".to_string(), None, true, false, false);
assert!(result.is_ok());
}

#[test]
fn test_run_gemini_no_conversations() {
let temp = tempfile::tempdir().unwrap();
let gemini = temp.path().join(".gemini");
std::fs::create_dir_all(gemini.join("tmp/empty")).unwrap();
let resolver = toolpath_gemini::PathResolver::new().with_gemini_dir(&gemini);
let mgr = toolpath_gemini::GeminiConvo::with_resolver(resolver);

let result = run_gemini_with_manager(
&mgr,
"/no/such/project".to_string(),
None,
false,
false,
false,
);
assert!(result.is_err());
assert!(
result
.unwrap_err()
.to_string()
.contains("No conversations found")
);
}

#[test]
fn test_run_gemini_include_thinking() {
let (_t, mgr) = setup_gemini_manager();
let result = run_gemini_with_manager(
&mgr,
"/abs/myrepo".to_string(),
Some("session-uuid".to_string()),
false,
true,
false,
);
assert!(result.is_ok());
}
}
Loading
Loading