Advanced import settings: choose the time window, batch size and limits for session import

[mthsmb]

Session import (Claude Code and Codex) has several fixed limits. They're sensible defaults, but there's no way to change them, and some can't be worked around by re-running import. I'd like an Advanced section for import, reachable from Settings after onboarding (see #10961), where I can choose how much to import.

Limits that currently decide what gets imported (from AgentSessionScanner.ts in 0.1.5):

What	Today	Suggested setting
Time window	30 days (RECENT_THREAD_WINDOW_MS)	Any number of days, or "all history"
Conversations per run, per project	100 (MAX_IMPORT_TRANSCRIPTS)	A custom number, or "continue automatically until done"
Input records per run	100,000 (MAX_IMPORT_RECORDS)	Custom, or automatic batching (#17484)
Messages kept per thread	200 (MAX_IMPORTED_MESSAGES)	A custom number, or full history
Scan metadata budget	64 MiB per source (MAX_METADATA_BYTES_PER_SOURCE)	Custom, or continuable (#27163)

Other options that would help:

• Sources and folders: pick which provider instances and config directories to scan, for example extra Claude config dirs (mentioned in [Bug]: Session import is unreachable after onboarding, so externally started sessions can never be imported #10961).
• Exclusions: choose whether to include non-git folders, the home directory, and sessions whose folder no longer exists, for example deleted worktrees, imported into a project you choose.
• Preview / dry run: show, per project, how many conversations and how much data will be imported or skipped, and why, before anything is written.
• Refresh already-imported threads whose transcript has grown since the last import (mentioned in [Bug]: Session import is unreachable after onboarding, so externally started sessions can never be imported #10961).
• Skip empty sessions quietly: sessions with no user messages are not counted as failures.

The current values could stay as the defaults, with a short warning about time and memory use when you raise them. Related: #17484, #21764, #11697, #10961.

[diogovalada]

This is really necessary. I want to import all conversations in Codex/Claude, and the lack of ability to do it is limiting.

[mthsmb]

I've built two parts of this idea and opened them as draft PRs, so the code and screenshots can be reviewed alongside the proposal. I'd like a maintainer's go-ahead on direction and scope before marking either one ready. Both keep today's behavior by default.

#32127, configurable import limits

• A new environment setting, agentSessionImport, uses today's constants as defaults. Each limit has a bounded range:
  • time window: 30 days, or all history;
  • conversations per run: 100;
  • records per conversation: 100,000, per transcript as of fix(server): import valid conversations within each transcript record limit #23192;
  • messages per thread: 200, or full history;
  • scan budget: 64 MiB.
• The scanner honors them for Claude Code, Codex and Pi. The 32 MiB history budget per conversation stays fixed.
• Settings → Threads gets a collapsed "Session import" section, with reset and a warning when a limit is raised.

#32128, import continues in batches

• This is the "continue automatically until done" part. The per-run budgets stay the same, but the wizard keeps going until nothing is left.
• It shows progress and a Stop button.
• The summary separates "could not be imported" from "not imported yet", and offers "Continue import" when work is left.

Neither PR adds an import entry point after onboarding. That stays with #10961 (#22411 and #23122), and both changes apply to whichever entry point lands.

Questions:

1. Is Settings → Threads the right place for the limits, and are the bounds reasonable, or would you rather expose fewer of them?
2. Should batching be automatic, or opt-in?

Happy to narrow either PR, or close it, depending on your answer.

@diogovalada, #32127's "All history" setting is the part that would let you import all of your Codex and Claude conversations.

[dominikzabcik]

Some data on the "messages kept per thread" row, from migrating my own sessions into T3 Code. I opened #33402 as a narrower alternative to making the limit configurable. It changes the default to full history and keeps every other limit as it is.

• How often the cap bites: in my last 30 days, 26 of 196 Claude and Codex sessions (13%) had more than 200 visible messages. The longest had 2,845. Those threads showed the first prompt plus the newest 199 messages.
• Write cost without the cap: each conversation is written in one eventSink.write transaction. On in-memory SQLite, 200 messages take 70 ms, 2,000 take 0.39 s, 10,000 take 1.8 s and 40,000 take 7.7 s. Thread history is paged, so the thread view is unaffected.
• A trap for any full-history option, including the one in feat: choose how much agent session history to import #32127: the cap also bounded Codex prompt-copy matching. That code scans back through the turn for every user record. Without the cap it is quadratic on a long run of user records: parsing a Codex transcript with 25,000 user records and no assistant reply took 3,019 ms. fix(server): imported conversations keep their full visible history #33402 replaces the scan with per-turn indexes, which brings it to 219 ms (main, keeping only 200 messages, takes 250 ms).

The open question for maintainers is the default: keep 200 and let people opt into more (#32127), or keep everything by default (#33402). I'm happy to close #33402 or narrow it to just the index fix, whichever direction you prefer.

[mthsmb]

I think keeping the full history by default is the right call. People importing their sessions want the whole conversation, and the cap quietly drops the middle of the longest ones, which tend to be the ones that matter. If you go with #33402, I'll rebase #32127 on it so its messages-per-thread setting defaults to full history and only serves people who want a cap.

On the write cost, I checked how the import is written. Reads go through worker threads, but every write goes through the single writer connection (node:sqlite, synchronous) on the server's main thread, behind one lock. Each imported conversation is one eventSink.write transaction, and that transaction keeps the writer for its whole duration. So with full history, a 40,000-message conversation holds the writer for the 7.7 s you measured: live turns and every other write wait behind it, and the main thread spends that time on synchronous statements. Paged thread history helps the reads, not this.

One way to fix it would be to write long conversations in chunks: the thread and its first chunk in one transaction, then the remaining messages appended in transactions of a few hundred, yielding between them. The transcript would be marked as imported only after the last chunk, so an interrupted import resumes where it stopped. I'm not adding that to #32127 for now, but I'm happy to open it as a separate PR if you want it.

[diogovalada]

Agree that importing the full history would make sense to be the default, as it would follow the Principle of Least Astonishment.