Skip to content

Implement Rust summary parity#367

Merged
willwashburn merged 2 commits intomainfrom
codex/summary-parity
May 7, 2026
Merged

Implement Rust summary parity#367
willwashburn merged 2 commits intomainfrom
codex/summary-parity

Conversation

@willwashburn
Copy link
Copy Markdown
Member

Summary

  • implements Rust burn summary parity for workflow/agent/provider filters
  • wires --by-tool, --by-subagent-type, --by-relationship, --subagent-tree, and --quality
  • adds SDK content querying and exports needed by the Rust CLI presenter
  • covers provider parsing, nested agent-session resolution, and by-tool byte-share attribution

Tests

  • cargo test -p relayburn-cli
  • cargo test -p relayburn-sdk
  • BURN_GOLDEN=1 cargo test -p relayburn-cli --test golden -- --nocapture

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

Review Change Stack

📝 Walkthrough

Walkthrough

This PR extends the Rust burn summary CLI from simple group-by-model/provider output into a multi-mode dispatcher supporting provider filtering, agent-tree traversal, by-subagent-type/relationship/tool aggregations, and optional quality metrics. Supporting SDK changes add in-Rust query filtering for content records and expand public type re-exports.

Changes

Multi-Mode Summary Expansion with Tool Cost Attribution

Layer / File(s) Summary
SDK Type Surface & Query
crates/relayburn-sdk/src/lib.rs, crates/relayburn-sdk/src/ledger/content.rs, crates/relayburn-sdk/src/ledger.rs
SDK re-exports now include ToolResultEventSource, ToolResultStatus, UserTurnBlock, UserTurnBlockKind, and UserTurnRecord. New content::query() deserializes JSON body rows and filters in-Rust via optional timestamp/session/source bounds. Ledger method signatures and export-kind formatting were reformatted for consistency only.
CLI Dispatcher Architecture
crates/relayburn-cli/src/commands/summary.rs (imports & main run flow)
Module imports expanded. run_inner rewritten as a dispatcher that parses provider CSV filters (case-insensitive), optionally resolves agent→session trees (BFS), queries enriched turns, applies agent/provider filters, and dispatches to renderers.
Query & Filtering Infrastructure
crates/relayburn-cli/src/commands/summary.rs (helpers & query building)
build_query now supports workflow enrichment. New helpers: parse_providers(), filter_turns_by_agent_and_provider(), collect_agent_session_tree(), and enrich_turns_to_turn_records().
Grouped & Tool-Attribution Rendering
crates/relayburn-cli/src/commands/summary.rs (grouped emission, by-tool mode)
emit_grouped() accepts optional quality. render_by_tool_mode() loads per-session user turns, attributes ingest cost to preceding tool calls via user-turn byte-share (or even-split fallback), labels attribution method, reports unattributed cost, and integrates replacement-savings when present.
Subagent & Relationship Rendering
crates/relayburn-cli/src/commands/summary.rs (subagent-type, relationship, tree modes)
render_subagent_type_mode() aggregates by subagent type. Relationship mode builds relationship→turn indices, computes percentiles/statistics, and supports subagent drilldown. Subagent-tree mode BFS collects connected relationships, loads turns across sessions, and renders an ASCII tree with node stats.
Quality Metrics Output
crates/relayburn-cli/src/commands/summary.rs (human rendering & quality section)
Human renderer appends a quality section when --quality is set and output is non-JSON. render_quality() formats outcome counts and computes the one-shot rate from quality metrics, handling n/a when no edit turns exist.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

  • AgentWorkforce/burn#315: Modifies the Summary command wiring and previously expanded the CLI summary features this PR further extends.
  • AgentWorkforce/burn#291: Implements per-tool attribution/byte-share logic that this PR reuses for by-tool cost attribution.
  • AgentWorkforce/burn#299: Adjusts SDK re-export surfaces that relate to the re-exports expanded in this change.

Poem

🐰 I hopped through queries, filters, and trees,
I counted tool bytes and shared them with ease,
I traced every agent and rendered the view,
A summary in modes, both JSON and new,
Carrots for quality—hoppy review!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.39% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Implement Rust summary parity' directly describes the main objective of the changeset, which implements parity for the Rust burn summary command.
Description check ✅ Passed The description clearly outlines the summary implementation with specific features, SDK additions, and testing approach, all directly related to the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/summary-parity

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a7e0435fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/relayburn-sdk/src/ledger/content.rs Outdated
Comment thread crates/relayburn-cli/src/commands/summary.rs Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/relayburn-cli/src/commands/summary.rs (1)

170-174: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Propagate --no-archive into LedgerOpenOptions.

SummaryArgs::no_archive is parsed, but this open path never reads it, so burn summary --no-archive still uses the default archive behavior. That makes the documented escape hatch a no-op.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-cli/src/commands/summary.rs` around lines 170 - 174,
SummaryArgs::no_archive is parsed but never applied when opening the ledger;
modify the LedgerOpenOptions you build before calling Ledger::open so it
reflects the parsed no_archive flag from SummaryArgs (e.g. after creating opts
via LedgerOpenOptions::with_home(h) or LedgerOpenOptions::default(), set
opts.no_archive = summary_args.no_archive or call the appropriate builder method
like opts.set_no_archive(summary_args.no_archive)), then pass that opts into
Ledger::open so --no-archive actually changes archive behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/relayburn-cli/src/commands/summary.rs`:
- Around line 196-203: The cost-attribution logic is running against the
filtered turn stream causing mis-attribution when turns are skipped; fix by
performing tool-cost attribution on the original unfiltered enriched list and
only filter afterward (or ensure render_by_tool_mode/attribute_cost_to_tools
iterate the full enriched sequence). Specifically, keep the initial enriched =
handle.raw().query_turns(&q)? result intact, call
attribute_cost_to_tools(enriched, ...) / render_by_tool_mode(enriched, ...) on
that full list, then call filter_enriched_turns(...) and
turns_from_enriched(...) to produce the displayed turns so indices like list[i -
1] reference real session predecessors rather than the post-filtered sequence.
- Around line 1539-1549: The BFS loop only queries rows with session_id == id,
missing child edges stored under related_session_id/agent_id; change the lookup
to fetch both kinds of relationships (e.g., perform two queries via
ledger.query_relationships: one with Query { session_id: Some(id),
..query_base.clone() } and another with Query { related_session_id: Some(id),
..query_base.clone() } ), iterate both result sets (deduping using the existing
out/seen_ids logic), and then continue using relationship_connected_ids(&r) and
inserting with relationship_instance_key(&r) as before so subagent children are
discovered.

In `@crates/relayburn-sdk/src/ledger/content.rs`:
- Around line 130-134: The query function currently eagerly collects all content
bodies into memory via query_map(...).collect(), causing memory spikes; change
it to stream rows from stmt.query_map(...) and iterate over the returned
iterator, extracting each row's body, applying the Query filter and only
deserializing/pushing matching entries into the result Vec<ContentRecord>.
Update logic in the query function (referencing conn, query, stmt, rows, Query,
ContentRecord) to loop over the iterator, handle rusqlite::Result for each row,
parse JSON (e.g., serde_json::from_str) and construct ContentRecord only when
the row satisfies the Query, avoiding the intermediate Vec<String>.

---

Outside diff comments:
In `@crates/relayburn-cli/src/commands/summary.rs`:
- Around line 170-174: SummaryArgs::no_archive is parsed but never applied when
opening the ledger; modify the LedgerOpenOptions you build before calling
Ledger::open so it reflects the parsed no_archive flag from SummaryArgs (e.g.
after creating opts via LedgerOpenOptions::with_home(h) or
LedgerOpenOptions::default(), set opts.no_archive = summary_args.no_archive or
call the appropriate builder method like
opts.set_no_archive(summary_args.no_archive)), then pass that opts into
Ledger::open so --no-archive actually changes archive behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ca8a8bad-6c98-4d9f-9e04-645f8d8ad617

📥 Commits

Reviewing files that changed from the base of the PR and between e6c38dd and 1a7e043.

📒 Files selected for processing (4)
  • crates/relayburn-cli/src/commands/summary.rs
  • crates/relayburn-sdk/src/ledger.rs
  • crates/relayburn-sdk/src/ledger/content.rs
  • crates/relayburn-sdk/src/lib.rs

Comment thread crates/relayburn-cli/src/commands/summary.rs
Comment thread crates/relayburn-cli/src/commands/summary.rs Outdated
Comment thread crates/relayburn-sdk/src/ledger/content.rs Outdated
@willwashburn
Copy link
Copy Markdown
Member Author

Addressed the outside-diff --no-archive comment in 2bccc01 as well: summary now installs the same RELAYBURN_ARCHIVE=0 guard pattern used by compare before opening/running the ledger path. Validation rerun: cargo test -p relayburn-cli; cargo test -p relayburn-sdk; BURN_GOLDEN=1 cargo test -p relayburn-cli --test golden -- --nocapture.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
crates/relayburn-cli/src/commands/summary.rs (2)

854-872: 💤 Low value

Consider filtering user turns by session at the query level if supported.

load_user_turns_for_by_tool loads all user turns via query_user_turns(&Query::default()) and filters in-memory by session ID. For large ledgers with many sessions, this could be inefficient. If the SDK's query_user_turns supports session_id filtering, consider passing the session IDs to reduce I/O.

If the SDK doesn't support this filter, the current approach is acceptable.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-cli/src/commands/summary.rs` around lines 854 - 872, The
function load_user_turns_for_by_tool currently calls
ledger.query_user_turns(&Query::default()) and then filters by session_id
in-memory, which is inefficient for large ledgers; update the call to pass a
Query that includes the session_ids (or a single session_id when iterating) if
the SDK supports session_id filtering (e.g., construct a Query with session_id
or a sessions list and call ledger.query_user_turns(&query)), falling back to
the existing in-memory filter only if the SDK lacks that filter; keep references
to load_user_turns_for_by_tool, ledger.query_user_turns, and Query::default when
making the change.

453-454: 💤 Low value

std::env::set_var/remove_var will require unsafe blocks in Rust Edition 2024.

These functions are unsafe in Edition 2024 and will require explicit unsafe blocks to compile. Currently set at lines 24–25 (in ArchiveOverride::activate) and lines 39–40 (in Drop impl) without unsafe wrapping. While the current Edition 2021 allows this, the pattern is unsound in multi-threaded contexts and will break on upgrade.

Consider refactoring to read the config once at startup and pass it through the call chain, or use a thread-safe configuration struct instead of relying on process-wide environment mutation.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/relayburn-cli/src/commands/summary.rs` around lines 453 - 454, The
code currently mutates the process environment with std::env::set_var/remove_var
in ArchiveOverride::activate and its Drop impl (and toggles RELAYBURN_ARCHIVE),
which will become unsafe in Rust 2024; instead refactor to stop mutating
process-wide env: change ArchiveOverride::activate to accept a configuration
flag (or a reference to a thread-safe config struct/OnceCell) passed in from
program startup, store that flag on the ArchiveOverride instance, and remove any
calls to std::env::set_var/remove_var from both ArchiveOverride::activate and
the Drop impl; alternatively implement a thread-local or atomic/shared config
that is initialized once at startup and read by the code paths that currently
depend on RELAYBURN_ARCHIVE, ensuring no direct process-wide env mutation
occurs.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/relayburn-cli/src/commands/summary.rs`:
- Around line 854-872: The function load_user_turns_for_by_tool currently calls
ledger.query_user_turns(&Query::default()) and then filters by session_id
in-memory, which is inefficient for large ledgers; update the call to pass a
Query that includes the session_ids (or a single session_id when iterating) if
the SDK supports session_id filtering (e.g., construct a Query with session_id
or a sessions list and call ledger.query_user_turns(&query)), falling back to
the existing in-memory filter only if the SDK lacks that filter; keep references
to load_user_turns_for_by_tool, ledger.query_user_turns, and Query::default when
making the change.
- Around line 453-454: The code currently mutates the process environment with
std::env::set_var/remove_var in ArchiveOverride::activate and its Drop impl (and
toggles RELAYBURN_ARCHIVE), which will become unsafe in Rust 2024; instead
refactor to stop mutating process-wide env: change ArchiveOverride::activate to
accept a configuration flag (or a reference to a thread-safe config
struct/OnceCell) passed in from program startup, store that flag on the
ArchiveOverride instance, and remove any calls to std::env::set_var/remove_var
from both ArchiveOverride::activate and the Drop impl; alternatively implement a
thread-local or atomic/shared config that is initialized once at startup and
read by the code paths that currently depend on RELAYBURN_ARCHIVE, ensuring no
direct process-wide env mutation occurs.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 32556da9-65fb-433d-ab4e-84a6efe2b674

📥 Commits

Reviewing files that changed from the base of the PR and between 1a7e043 and 2bccc01.

📒 Files selected for processing (2)
  • crates/relayburn-cli/src/commands/summary.rs
  • crates/relayburn-sdk/src/ledger/content.rs

@willwashburn willwashburn merged commit c6933d1 into main May 7, 2026
11 checks passed
@willwashburn willwashburn deleted the codex/summary-parity branch May 7, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant