Skip to content

relayburn-ingest: port per-harness orchestration loops (#277)#297

Merged
willwashburn merged 4 commits intomainfrom
claude/close-issue-277-ingest-orchestration
May 5, 2026
Merged

relayburn-ingest: port per-harness orchestration loops (#277)#297
willwashburn merged 4 commits intomainfrom
claude/close-issue-277-ingest-orchestration

Conversation

@willwashburn
Copy link
Copy Markdown
Member

Summary

  • Fills in the bodies of ingest_claude_into, ingest_codex_into, ingest_opencode_into, and the ingest_claude_session fast-path that were stubbed in [Rust port] relayburn-ingest: discovery + watch loop (poll-based) #245. Each helper drives the matching reader parser incrementally with cursor + state carry-over, appends every record kind through the Ledger writer surface, and (for codex / opencode) folds matching pending-stamp manifests into freshly-discovered sessions. Content-mode is sourced from load_config().content.store ([Rust port] relayburn-ledger: list_content_session_ids + Config / loadConfig surface #279) and defaults to Full if the config layer errors.
  • Gap-warning and reingest_missing_content call sites are intentionally stubbed out with TODO(#278) markers — the sibling PR fills them in. Also exposes derive_codex_session_id from the public surface so [Rust port] relayburn-ingest: gap-warning state machine + reingest_missing_content #278's reingest_missing_content body has the codex skip-existing filter it needs.
  • Adds tests/orchestration.rs with 5 round-trip integration tests: per-harness session through the full parse-and-append path, an ingest_all cross-harness sweep, and a fast-path EOF-cursor test that asserts a follow-up ingest_all reports appendedTurns == 0. Each test pins all three IngestRoots under a temp dir and serializes on RELAYBURN_HOME via a module-level mutex so parallel cargo test runs can't race.

Test plan

  • cargo test -p relayburn-ingest (32 tests pass: 18 unit + 5 orchestration + 5 pending_stamps + 3 watch_loop + 1 doc).
  • cargo build --workspace --all-targets clean.
  • cargo test --workspace clean.
  • cargo clippy -p relayburn-ingest --all-targets clean.
  • pnpm run build && pnpm run test clean (873 TS tests pass).

Closes #277.

🤖 Generated with Claude Code

Fills in the bodies of `ingest_claude_into`, `ingest_codex_into`,
`ingest_opencode_into`, and the `ingest_claude_session` fast-path that
were stubbed in #245. Each helper drives the matching reader parser
incrementally with cursor + state carry-over, appends every record
kind through the `Ledger` writer surface, and (for codex / opencode)
folds matching pending-stamp manifests into the freshly-discovered
session via `resolve_pending_stamps_for_session`.

`ingest_claude_into` runs `reconcile_claude_session_relationships` once
at the end of the pass so cross-file fork / continuation rows are
emitted alongside per-file relationships.

`ingest_claude_session` (the per-session fast-path used by the
`burn run claude` adapter after the child exits) now runs the
synchronous `parse_claude_session`, appends, and persists a Claude
cursor at EOF so a follow-up `ingest_all` sweep skips the file.

Content-mode is sourced from `load_config().content.store` (#279) and
defaults to `Full` if the config layer errors. Gap-warning and
`reingest_missing_content` call sites are intentionally stubbed out
with `TODO(#278)` markers; the sibling PR fills them in.

Adds `tests/orchestration.rs` (5 tests):
- `ingest_claude_projects_round_trips_a_fixture_session`
- `ingest_codex_sessions_round_trips_a_fixture_session`
- `ingest_opencode_sessions_round_trips_a_fixture_session`
- `ingest_all_walks_each_harness_root_once`
- `ingest_claude_session_writes_eof_cursor_so_followup_skips_file`

Each test pins all three IngestRoots under a temp dir and serializes
on `RELAYBURN_HOME` via a module-level mutex so a parallel cargo test
run can't see a leaked env mutation. The follow-up sweep in the last
test asserts `appendedTurns == 0`, proving the EOF cursor wins.

Also exposes `derive_codex_session_id` from the public surface — #278's
`reingest_missing_content` body needs it for the codex skip-existing
filter.

cargo test -p relayburn-ingest: 32 tests pass (18 unit + 5
orchestration + 5 pending_stamps + 3 watch_loop + 1 doc).
cargo build --workspace + cargo test --workspace clean. pnpm run
build + pnpm run test still green (873 TS tests).

Closes #277.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 5, 2026

📝 Walkthrough

Walkthrough

This PR implements the per-harness ingestion orchestration in relayburn-ingest: resolves ContentStoreMode at runtime, threads it through ingestion entrypoints, adds a Claude fast-path, fills ingest_claude_into, ingest_codex_into, and ingest_opencode_into with cursor-driven incremental scans, adds Codex cursor↔resume-state conversions, filesystem metadata helpers, and end-to-end orchestration tests.

Changes

Orchestration and Cursor-Driven Ingestion

Layer / File(s) Summary
Runtime Configuration & Threading
crates/relayburn-ingest/src/ingest.rs
Adds resolve_content_mode() (loads config with fallback ContentStoreMode::Full) and threads content_mode through ingest_all and per-harness entrypoints.
Claude Fast Path
crates/relayburn-ingest/src/ingest.rs
Implements ingest_claude_session: parse JSONL with parse_claude_session (uses content_mode), append parsed turns/content/events/relationships/tool-results/user-turns, then persist ClaudeCursor at EOF (inode + offset + mtime + last_user_text).
Claude Orchestration with Rotation Detection
crates/relayburn-ingest/src/ingest.rs
Implements ingest_claude_into: directory walk, rotation detection via inode/mtime/offset, carry last_user_text across incremental parses, call parse_claude_session_incremental, append entities, persist updated ClaudeCursor, perform cross-file relationship reconciliation.
Codex Orchestration with Resume State
crates/relayburn-ingest/src/ingest.rs
Implements ingest_codex_into: session file walk, cursor↔resume-state conversions, resume incremental parsing, derive session ID when missing (with fallback), resolve pending stamps when turns exist, append entities, persist next CodexCursor.
Opencode Orchestration with Message Directory Tracking
crates/relayburn-ingest/src/ingest.rs
Implements ingest_opencode_into: walk ses_*.json, compute per-session message-dir mtime for rotation detection, carry seen_message_ids across parses, append entities, resolve pending stamps, persist OpencodeCursor.
Cursor & Resume State Serialization
crates/relayburn-ingest/src/ingest.rs
Adds helpers to convert CodexCursorCodexResumeState and persisted representations for turn contexts and last-completed-turn.
Filesystem & Metadata Helpers
crates/relayburn-ingest/src/ingest.rs
Adds list_dirs, list_jsonl_files, dir_mtime, mtime_ms, and file_inode (Unix inode and non-Unix fallback using file length).
End-to-End Tests
crates/relayburn-ingest/tests/orchestration.rs
New test module: isolated temp ledger/RELAYBURN_HOME via ENV_LOCK, fixture generation, per-harness round-trip tests for Claude/Codex/Opencode and ingest_all, cursor persistence checks, and EOF-cursor skip validation.

Sequence Diagram

sequenceDiagram
    participant Walk as Directory Walker
    participant FSMeta as Filesystem Meta
    participant Cursor as Cursor Store
    participant Parse as Parser (incremental)
    participant Ledger as Ledger

    loop For each discovered JSONL file
        Walk->>FSMeta: stat file (inode, mtime, size)
        FSMeta-->>Walk: metadata
        Walk->>Cursor: load cursor for file
        Cursor-->>Walk: last offset / inode / last_user_text

        alt Rotated or reset condition
            Walk->>Cursor: reset cursor state
        end

        Walk->>Parse: parse from offset with content_mode & carried state
        Parse-->>Walk: turns, content, compactions, relationships

        Walk->>Ledger: append parsed entities
        Ledger-->>Walk: ack

        Walk->>Cursor: persist updated cursor (offset, mtime, inode, carried state)
    end

    Walk->>Walk: reconcile cross-file relationships (Claude)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

  • #295 — The implemented per-harness orchestration helpers fulfill the TODOs for those functions and relate to gap/instrumentation work referenced in that issue.

Possibly related PRs

Poem

🐰 Hops through JSONL with a curious nose,

Cursors tucked in where the file-rot grows.
Claude, Codex, OpenCode—each takes a turn,
Ledger hums softly as new entries burn.
A rabbit applauds: incremental, precise, and terse.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'relayburn-ingest: port per-harness orchestration loops (#277)' clearly and specifically describes the main change: implementing the previously-stubbed per-harness orchestration loop functions in the relayburn-ingest module.
Description check ✅ Passed The description is directly related to the changeset, detailing the implementation of ingest orchestration helpers, cursor management, content-mode sourcing, test coverage, and test plan validation.
Linked Issues check ✅ Passed The PR implementation fully satisfies linked issue #277 scope: ingest_claude_into, ingest_codex_into, ingest_opencode_into, and ingest_claude_session bodies are implemented with cursor carry-over, record appending, and state management; orchestration tests cover per-harness round-trips, ingest_all integration, and EOF-cursor fast-path behavior.
Out of Scope Changes check ✅ Passed All changes in the PR align with the stated scope of issue #277: the ingest.rs modifications implement the orchestration loop bodies and expose derive_codex_session_id as noted, and orchestration.rs adds only related integration tests; gap-warning and reingest_missing_content are intentionally stubbed with TODO(#278) markers as documented.
Docstring Coverage ✅ Passed Docstring coverage is 97.56% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/close-issue-277-ingest-orchestration

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/relayburn-ingest/src/ingest.rs (1)

909-909: 💤 Low value

Unnecessary sort on already-sorted array.

The expected_dashes array [8, 13, 18, 23] is already in sorted order; the .sort() call has no effect.

♻️ Proposed fix
 fn is_uuid(s: &str) -> bool {
     let groups = [8usize, 4, 4, 4, 12];
-    let mut expected_dashes = [8usize, 13, 18, 23];
-    expected_dashes.sort();
+    let expected_dashes = [8usize, 13, 18, 23];
     if s.len() != groups.iter().sum::<usize>() + (groups.len() - 1) {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-ingest/src/ingest.rs` at line 909, The array assigned to
expected_dashes is already sorted ([8, 13, 18, 23]), so remove the redundant
.sort() call that follows its declaration to avoid an unnecessary operation;
locate the declaration of the variable expected_dashes in ingest.rs (the let mut
expected_dashes = [8usize, 13, 18, 23]; line) and delete the subsequent
expected_dashes.sort() invocation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/relayburn-ingest/src/ingest.rs`:
- Line 909: The array assigned to expected_dashes is already sorted ([8, 13, 18,
23]), so remove the redundant .sort() call that follows its declaration to avoid
an unnecessary operation; locate the declaration of the variable expected_dashes
in ingest.rs (the let mut expected_dashes = [8usize, 13, 18, 23]; line) and
delete the subsequent expected_dashes.sort() invocation.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f0af3980-721a-4cdd-9db4-ec0e9bf64bd6

📥 Commits

Reviewing files that changed from the base of the PR and between 146bf7b and 130ae73.

📒 Files selected for processing (3)
  • crates/relayburn-ingest/src/ingest.rs
  • crates/relayburn-ingest/src/lib.rs
  • crates/relayburn-ingest/tests/orchestration.rs

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 130ae73b4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

inode: file_inode(&meta),
offset_bytes: meta.len(),
mtime_ms: mtime_ms(&meta),
last_user_text: None,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve Claude resume context in fast-path cursor

ingest_claude_session always persists last_user_text: None in the Claude cursor, but later incremental ingests depend on that seed to classify resumed assistant completions correctly when the prompt was before the resume offset. After this fast-path runs, a follow-up ingest_all on a session that later appends a completion can parse the turn without its prior prompt context and misclassify activity/user-turn attribution. Persisting resume context (or using the incremental parser result for cursor fields) avoids this regression path.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skipping — this matches the existing TS behavior in ingestClaudeSession (packages/ingest/src/ingest.ts:285–291), which also writes the cursor with lastUserText unset:

const cursor: ClaudeCursor = {
  kind: 'claude',
  inode: st.ino,
  offsetBytes: st.size,
  mtimeMs: st.mtimeMs,
};

The fast path uses the non-incremental parse_claude_session, which doesn't expose last_user_text on its ParseResult — only parse_claude_session_incremental does. Threading that field through the full parser is a TS-side change (and a divergence from #277's "faithful port" scope). If the misclassification path the comment describes is real, it's an existing TS bug worth filing separately rather than fixing only on the Rust side here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@willwashburn
Copy link
Copy Markdown
Member Author

Addressed bot feedback on e4a3eae:

  • CodeRabbit nitpick (is_uuid redundant .sort()) — fixed. The [8, 13, 18, 23] array is already sorted, so the .sort() call was dead code; dropped it and the mut binding.
  • Codex P2 (Claude resume context in fast-path cursor) — skipping. The fast path writes the cursor with last_user_text: None to match the existing TS ingestClaudeSession behavior (packages/ingest/src/ingest.ts:285–291), where lastUserText is similarly omitted. The non-incremental parse_claude_session doesn't surface last_user_text on its ParseResult (only parse_claude_session_incremental does). Threading it through would diverge from [Rust port] relayburn-ingest: per-harness orchestration loops #277's faithful-port scope and would need to land on the TS side first if it's a real concern. Replied inline on the comment.

cargo test -p relayburn-ingest and cargo clippy -p relayburn-ingest --all-targets are clean.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/relayburn-ingest/src/ingest.rs`:
- Around line 377-385: When a file is skipped because start_offset >= size (EOF)
the code currently updates prior_claude.mtime_ms and stores it back into cursors
then continues, which prevents unchanged sessions from ever producing
ReconcileClaudeRelationshipsInput and breaks relationship linking (also see
similar block around the ingest_claude_session fast-path at 430-452). Update the
EOF-skip path so that when prior_claude exists you also emit or enqueue a
ReconcileClaudeRelationshipsInput (built from prior_claude) into the same
reconciliation collection used for parsed sessions (or mark the cursor so the
reconciler will load it later) instead of letting the file be permanently
skipped by the ingest_claude_session fast-path; keep updating mtime_ms and
storing FileCursor::Claude(c) as before but ensure the reconciler receives the
unchanged session’s relationship info so forks/continuations can be linked.
- Around line 552-566: The call to resolve_pending_stamps_for_session is
currently ignored which can drop pre-spawn manifests; change the code around
resolve_pending_stamps_for_session(...) (the call that builds
PendingStampSessionCandidate with PendingStampHarness::Codex and the analogous
block at lines 666-676) to check its Result and on Err either propagate the
error out of the enclosing function (use ? or map_err to return a meaningful
error) or, if you cannot return an error there, prevent advancing/committing the
session cursor (i.e., bail/return early or skip the commit) so the failed stamp
resolution will be retried later; apply the same Result-handling fix to the
other harness branch as well.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 206f5a0b-2b31-4937-b15e-5db1502b9d48

📥 Commits

Reviewing files that changed from the base of the PR and between 130ae73 and e4a3eae.

📒 Files selected for processing (1)
  • crates/relayburn-ingest/src/ingest.rs

Comment on lines +377 to +385
if !rotated && start_offset >= size {
// Nothing new; refresh mtime bookkeeping and skip parse +
// reconciliation evidence — `relationshipIdHash` dedup keeps
// re-emits idempotent.
if let Some(mut c) = prior_claude.clone() {
c.mtime_ms = mtime;
cursors.insert(key, FileCursor::Claude(c));
}
continue;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Claude reconciliation misses relationships to unchanged sessions.

EOF-skipped files never contribute ReconcileClaudeRelationshipsInput, so the final reconciliation only sees sessions parsed in this sweep. That means a newly discovered fork/continuation can miss its link to an older unchanged session, and the ingest_claude_session fast-path makes this permanent by parking that older file at EOF immediately.

Also applies to: 430-452

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-ingest/src/ingest.rs` around lines 377 - 385, When a file is
skipped because start_offset >= size (EOF) the code currently updates
prior_claude.mtime_ms and stores it back into cursors then continues, which
prevents unchanged sessions from ever producing
ReconcileClaudeRelationshipsInput and breaks relationship linking (also see
similar block around the ingest_claude_session fast-path at 430-452). Update the
EOF-skip path so that when prior_claude exists you also emit or enqueue a
ReconcileClaudeRelationshipsInput (built from prior_claude) into the same
reconciliation collection used for parsed sessions (or mark the cursor so the
reconciler will load it later) instead of letting the file be permanently
skipped by the ingest_claude_session fast-path; keep updating mtime_ms and
storing FileCursor::Claude(c) as before but ensure the reconciler receives the
unchanged session’s relationship info so forks/continuations can be linked.

Comment on lines +552 to +566
if !parsed.turns.is_empty() {
if let Some(sid) = &codex_session_id {
let cwd = next_resume
.session_cwd
.clone()
.or_else(|| parsed.turns.first().and_then(|t| t.project.clone()));
let candidate = PendingStampSessionCandidate {
harness: PendingStampHarness::Codex,
session_id: sid.clone(),
session_path: Some(file.clone()),
session_mtime_ms: Some(mtime),
cwd,
};
let _ = resolve_pending_stamps_for_session(ledger, &candidate);
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don’t discard pending-stamp resolution failures.

Both harnesses ignore resolve_pending_stamps_for_session(...). If that write fails, the session still appends turns and advances its cursor, so the pre-spawn manifest is silently dropped and won’t be retried on later sweeps. Propagate the error, or at least avoid committing the cursor when stamp resolution fails.

Also applies to: 666-676

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-ingest/src/ingest.rs` around lines 552 - 566, The call to
resolve_pending_stamps_for_session is currently ignored which can drop pre-spawn
manifests; change the code around resolve_pending_stamps_for_session(...) (the
call that builds PendingStampSessionCandidate with PendingStampHarness::Codex
and the analogous block at lines 666-676) to check its Result and on Err either
propagate the error out of the enclosing function (use ? or map_err to return a
meaningful error) or, if you cannot return an error there, prevent
advancing/committing the session cursor (i.e., bail/return early or skip the
commit) so the failed stamp resolution will be retried later; apply the same
Result-handling fix to the other harness branch as well.

…7-ingest-orchestration

# Conflicts:
#	crates/relayburn-ingest/src/ingest.rs
#	crates/relayburn-ingest/src/lib.rs
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
crates/relayburn-ingest/src/ingest.rs (2)

283-294: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Claude reconciliation still excludes unchanged sessions.

EOF-skipped files never contribute ReconcileClaudeRelationshipsInput, and ingest_claude_session() immediately parks fast-pathed files at EOF. That means a newly parsed fork/continuation can only reconcile against sessions re-parsed in this sweep, so links to older unchanged sessions are still missed.

Also applies to: 352-360, 423-427

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-ingest/src/ingest.rs` around lines 283 - 294, The current
logic persists an EOF cursor (building ClaudeCursor and calling
save_cursor_changes) before reconciliation, causing unchanged sessions to be
excluded from ReconcileClaudeRelationshipsInput and preventing links to older
sessions; modify ingest flow so either (A) defer persisting/updating the cursor
(the ClaudeCursor/FileCursor::Claude and save_cursor_changes call) until after
ingest_claude_session() and reconciliation have been run, or (B) if you must
fast-path and park EOF files immediately, still emit or queue a
ReconcileClaudeRelationshipsInput for that file using the existing session
record so unchanged sessions are included in reconciliation; update the code
paths around load_cursors, the ClaudeCursor creation, and save_cursor_changes to
implement one of these two fixes.

527-541: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don’t advance past failed pending-stamp resolution.

Both harnesses discard resolve_pending_stamps_for_session(...). If that write fails, turns still append and the cursor still advances, so the pre-spawn manifest is effectively dropped and won’t be retried on later sweeps.

Suggested fix
-                let _ = resolve_pending_stamps_for_session(ledger, &candidate);
+                resolve_pending_stamps_for_session(ledger, &candidate)?;

Apply the same change in the OpenCode branch as well.

Also applies to: 642-651

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-ingest/src/ingest.rs` around lines 527 - 541, The code
currently ignores the Result from resolve_pending_stamps_for_session, allowing
cursor advancement even if the pending-stamp write fails; change the call in
both the Codex branch (where PendingStampHarness::Codex and
PendingStampSessionCandidate are constructed) and the OpenCode branch to
propagate errors instead of discarding them — e.g. capture the Result and use ?
(or return Err) so that on failure you abort/return early and avoid appending
turns or advancing next_resume; keep the candidate construction (session_id,
session_path, session_mtime_ms, cwd) as-is and only change how
resolve_pending_stamps_for_session(ledger, &candidate) is handled.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/relayburn-ingest/src/ingest.rs`:
- Around line 241-248: The code takes offset_bytes from the initial
fs::metadata(&file) (meta) before calling parse_claude_session(), which can
cause replay if the file grows during parsing; after calling
parse_claude_session() (and before saving the cursor), re-stat the file (e.g.,
fs::metadata(&file) or file.metadata()) and derive offset_bytes from that fresh
metadata (replace uses of the pre-parse meta/offset_bytes with the post-parse
stat), and apply the same change for the other occurrences noted (the blocks
around lines 250-256 and 283-289) so the saved cursor always points to the true
EOF after parsing.

---

Duplicate comments:
In `@crates/relayburn-ingest/src/ingest.rs`:
- Around line 283-294: The current logic persists an EOF cursor (building
ClaudeCursor and calling save_cursor_changes) before reconciliation, causing
unchanged sessions to be excluded from ReconcileClaudeRelationshipsInput and
preventing links to older sessions; modify ingest flow so either (A) defer
persisting/updating the cursor (the ClaudeCursor/FileCursor::Claude and
save_cursor_changes call) until after ingest_claude_session() and reconciliation
have been run, or (B) if you must fast-path and park EOF files immediately,
still emit or queue a ReconcileClaudeRelationshipsInput for that file using the
existing session record so unchanged sessions are included in reconciliation;
update the code paths around load_cursors, the ClaudeCursor creation, and
save_cursor_changes to implement one of these two fixes.
- Around line 527-541: The code currently ignores the Result from
resolve_pending_stamps_for_session, allowing cursor advancement even if the
pending-stamp write fails; change the call in both the Codex branch (where
PendingStampHarness::Codex and PendingStampSessionCandidate are constructed) and
the OpenCode branch to propagate errors instead of discarding them — e.g.
capture the Result and use ? (or return Err) so that on failure you abort/return
early and avoid appending turns or advancing next_resume; keep the candidate
construction (session_id, session_path, session_mtime_ms, cwd) as-is and only
change how resolve_pending_stamps_for_session(ledger, &candidate) is handled.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d1570385-251a-426c-b803-5ff691cc99a0

📥 Commits

Reviewing files that changed from the base of the PR and between e4a3eae and 40f85e6.

📒 Files selected for processing (1)
  • crates/relayburn-ingest/src/ingest.rs

Comment thread crates/relayburn-ingest/src/ingest.rs Outdated
`ingest_claude_session` was building the EOF cursor from the pre-parse
fs::metadata, but `parse_claude_session` reads via `BufReader::lines()`
to actual EOF — so if the file grew during parse, the parser consumed
the new bytes but the saved cursor still pointed at the pre-parse size.
A follow-up `ingest_all` would replay that range and emit duplicate
turns. Re-stat after parse (and before saving the cursor) so the cursor
reflects what was actually read. Diverges from TS, which has the same
race window; cursor position is internal state, so this stays
parity-safe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/relayburn-ingest/src/ingest.rs (1)

711-716: 💤 Low value

Potential truncation when cumulative exceeds i64::MAX.

Casting u64 to i64 at lines 712-715 can overflow if cumulative token counts exceed 2^63. In practice this is unlikely, but a defensive .min(i64::MAX as u64) as i64 would prevent silent data corruption on extreme values.

♻️ Defensive casting
 CodexResumeState {
     cumulative: ReaderCumulativeUsage {
-        input: c.cumulative.input as i64,
-        output: c.cumulative.output as i64,
-        cache_read: c.cumulative.cache_read as i64,
-        reasoning: c.cumulative.reasoning as i64,
+        input: c.cumulative.input.min(i64::MAX as u64) as i64,
+        output: c.cumulative.output.min(i64::MAX as u64) as i64,
+        cache_read: c.cumulative.cache_read.min(i64::MAX as u64) as i64,
+        reasoning: c.cumulative.reasoning.min(i64::MAX as u64) as i64,
     },
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-ingest/src/ingest.rs` around lines 711 - 716, The current
casts of c.cumulative.* from u64 to i64 (used when constructing
ReaderCumulativeUsage in ingest.rs) can overflow; update each field (input,
output, cache_read, reasoning) to clamp the u64 to i64::MAX before casting (e.g.
replace direct as i64 with (.min(i64::MAX as u64) as i64)) so extreme counts
never silently wrap or truncate while keeping the same ReaderCumulativeUsage
field types.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/relayburn-ingest/src/ingest.rs`:
- Around line 711-716: The current casts of c.cumulative.* from u64 to i64 (used
when constructing ReaderCumulativeUsage in ingest.rs) can overflow; update each
field (input, output, cache_read, reasoning) to clamp the u64 to i64::MAX before
casting (e.g. replace direct as i64 with (.min(i64::MAX as u64) as i64)) so
extreme counts never silently wrap or truncate while keeping the same
ReaderCumulativeUsage field types.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 32661740-38e9-4e71-9f53-9e1487e9ae7b

📥 Commits

Reviewing files that changed from the base of the PR and between 40f85e6 and 066d219.

📒 Files selected for processing (1)
  • crates/relayburn-ingest/src/ingest.rs

@willwashburn willwashburn merged commit 852fbb8 into main May 5, 2026
3 checks passed
@willwashburn willwashburn deleted the claude/close-issue-277-ingest-orchestration branch May 5, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Rust port] relayburn-ingest: per-harness orchestration loops

1 participant