Skip to content

feat: add conversation import system for extracting knowledge from prior AI agent history#252

Merged
BYK merged 1 commit into
mainfrom
feat/conversation-import
May 12, 2026
Merged

feat: add conversation import system for extracting knowledge from prior AI agent history#252
BYK merged 1 commit into
mainfrom
feat/conversation-import

Conversation

@BYK
Copy link
Copy Markdown
Owner

@BYK BYK commented May 12, 2026

Summary

  • Adds a pluggable conversation import system that detects and extracts long-term knowledge from 7 external AI coding agents: Claude Code, Codex, OpenCode, Cline, Continue, Pi, and Aider
  • Introduces lore import CLI command and auto-detection in lore run for fresh projects
  • Refactors curator parseOps/applyOps into shared helpers reused by the import extraction pipeline

Details

Architecture

The import system lives in packages/core/src/import/ with a clean layered design:

  1. Provider interface (AgentHistoryProvider) — each agent implements detect(projectPath) and readChunks(projectPath, sessionIds)
  2. Detection orchestrator — scans all registered providers, returns results sorted by richness
  3. Knowledge extraction — feeds conversation chunks to the curator LLM sequentially, with dedup via existing entries
  4. Idempotencyimport_history table (DB migration v19) tracks imported sources by hash to prevent re-importing

Providers

Provider Source Format Project Matching
Claude Code ~/.claude/projects/<mangled>/*.jsonl JSONL Path-mangled directory
Codex ~/.codex/sessions/YYYY/MM/DD/*.jsonl JSONL cwd in session_meta
OpenCode ~/.local/share/opencode/opencode.db SQLite (read-only) worktree in project table
Cline VS Code globalStorage /tasks/<id>/ JSON (Anthropic MessageParam[]) cwdOnTaskInitialization
Continue ~/.continue/sessions/<id>.json JSON workspaceDirectory
Pi ~/.pi/agent/sessions/<encoded-cwd>/*.jsonl JSONL (tree-structured) CWD-encoded directory
Aider <project>/.aider.chat.history.md Markdown Per-project file

CLI

lore import                          # Detect + import interactively
lore import --dry-run                # Show what would be imported
lore import --agent claude-code      # Import from specific agent
lore import --yes                    # Skip confirmation

Auto-detection (lore run)

On fresh projects (no existing knowledge/temporal data), lore run checks for prior conversations before launching the agent. If found, prompts the user once and runs import in the background. Declined imports are recorded so we don't ask again.

Curator refactor

Extracted parseOps() and applyOps() from curator.ts into exported shared helpers. Both the live curator and the import system use them — zero behavior change for existing code paths.

Testing

83 new tests across 10 files covering:

  • All 7 providers (detection, parsing, chunking, edge cases)
  • Detection orchestrator (aggregation, error handling, sorting)
  • Import history (idempotency, declined state, hash computation)
  • Knowledge extraction (mocked LLM, sequential processing, progress callbacks)
  • Curator ops (parseOps, applyOps create/update/delete/truncate/skipCreate)

Related

…ior AI agent history

Introduces a pluggable system that detects, reads, and extracts long-term
knowledge from conversation histories of 7 external AI coding agents:
Claude Code, Codex, OpenCode, Cline, Continue, Pi, and Aider.

Key components:
- Provider interface (AgentHistoryProvider) with detect/readChunks methods
- Detection orchestrator scanning all registered providers per project
- Knowledge extraction via curator LLM (reuses shared parseOps/applyOps)
- Idempotency tracking (import_history table, DB migration v19)
- `lore import` CLI command with dry-run, agent filter, and progress
- Auto-detection in `lore run` for fresh projects (background import)
- 83 tests covering all providers, detection, extraction, and history
@BYK BYK enabled auto-merge (squash) May 12, 2026 13:47
@BYK BYK disabled auto-merge May 12, 2026 13:48
@BYK BYK merged commit a2eab45 into main May 12, 2026
7 checks passed
@BYK BYK deleted the feat/conversation-import branch May 12, 2026 13:50
This was referenced May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant