Codex Desktop startup rescans archived_sessions instead of incrementally updating session_index.jsonl

[nanlis]

Problem

Codex appears to rebuild session_index.jsonl by scanning files under archived_sessions during startup. On machines with large archived histories, this makes startup much slower than necessary.

Observed case:

• archived_sessions contained about 2.2GB of JSON session data.
• Codex startup rescanned archived files to rebuild the session index.
• Archived sessions were rescanned even when they had already been indexed and had not changed.
• Environment: Windows 11, Microsoft Store Codex App 26.506.3741.0, codex-cli 0.130.0.

Related issues

• Codex Desktop App becomes laggy because it scans all ~/.codex/sessions rollout files instead of respecting Desktop-visible session index/state #20864 reports Desktop lag from scanning the hot sessions tree.
• Desktop performance collapses in profiles with a few very large local conversation histories (typing, scrolling, thread list, random exits) #18693 reports broad Desktop performance collapse with very large local histories.
• This issue is the archived-history variant: even files already moved to archived_sessions can remain on the startup indexing hot path.

Expected behavior

Codex should treat the session index as a persistent cache and update it incrementally:

• Load the existing index immediately on startup.
• Only scan metadata for archived session files.
• Only parse JSON for new or modified files.
• Update the index when a session is archived, instead of waiting for the next startup.
• Provide an explicit rebuild command for repair/migration cases.

Suggested design

Add a v2 index with a manifest/state file:

session_index_v2/
  meta.json
  records.jsonl
  state.json
  journal.jsonl

Use per-file fingerprints in state.json:

{
  "source_path": "archived_sessions/2026/05/session.jsonl",
  "size": 123456,
  "mtime_ms": 1778760000000,
  "head_hash": "sha256:...",
  "tail_hash": "sha256:..."
}

Startup flow:

1. Load records.jsonl.
2. Display known sessions immediately.
3. Start a background reconcile job.
4. Reconcile compares file metadata against state.json.
5. Parse only changed or new files.
6. Write updates through an append-only journal and compact later.

Acceptance criteria

• If no archived files changed, startup does not parse archived JSON files.
• Adding one archived file causes only that file to be parsed.
• Modifying one archived file causes only that file to be reparsed.
• Deleting an archived file writes a tombstone or removes the index entry.
• Corrupt index/journal rows do not block startup.
• A manual rebuild-session-index command exists for repair.

Workaround

Move cold archived sessions out of archived_sessions so Codex does not see them during startup. This avoids the startup slowdown but does not fix the underlying indexing design.

[etraut-openai]

I don’t think this report matches the current implementation.

session_index.jsonl is used only as an append-only legacy name index and fallback lookup path. It is not the primary thread index used by Desktop/app-server, and I do not see any production code that rebuilds, truncates, compacts, or regenerates it from archived_sessions during startup.

A few relevant points from the current codebase:

• App/server thread listing goes through the thread store and SQLite state DB, not session_index.jsonl.
• Archiving updates SQLite metadata immediately when the DB is available.
• session_index.jsonl is written on thread-name updates and later consulted for fallback title/name resolution.
• I found no path that rescans archived rollout files in order to rebuild session_index.jsonl.

There is a separate startup/backfill mechanism for the SQLite state DB that can scan both sessions and archived_sessions when rollout metadata backfill has not completed yet. There are also thread/list scan-and-repair paths that may touch rollout files unless callers opt into useStateDbOnly. Those are worth discussing if they are causing repeated archived-history work.

But as written, the specific claim that Desktop startup “rescans archived_sessions to rebuild session_index.jsonl” does not appear to be supported by the current code. I’d suggest reframing this around observed repeated archived-rollout scanning, with logs or traces showing whether the startup backfill gate or thread/list repair path is actually responsible.

I'm going to close this issue. If you'd like to open a new bug report, please include all of the information requested in the bug report template including the version of the app you're using, the platform, etc. Also, focus on the symptoms and repro steps of the bug before providing your analysis of the root cause.