relayburn-sdk: port the ingest verb#302
Conversation
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughReplaces placeholder ingest verb with full async implementation in the SDK. Introduces ChangesIngest Verb Implementation
Sequence DiagramsequenceDiagram
participant Client
participant SDK as SDK<br/>(LedgerHandle)
participant Ingest as relayburn_ingest
participant Ledger
Client->>SDK: ingest(options)
SDK->>SDK: Open ledger if needed<br/>(LedgerOpenOptions)
SDK->>SDK: Convert IngestOptions<br/>to RawIngestOptions
SDK->>Ingest: ingest_all(raw_opts, ledger)
Ingest->>Ledger: scan & ingest
Ingest-->>SDK: IngestReport
SDK-->>Client: Result<IngestReport>
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 458c862f0d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| fn into_raw(self) -> RawIngestOptions { | ||
| RawIngestOptions { | ||
| on_progress: self.on_progress, | ||
| on_warn: self.on_warn, | ||
| roots: self.roots, |
There was a problem hiding this comment.
Preserve ledger_home for ingest side effects
into_raw drops ledger_home, but ingest_all still performs home-scoped side effects via cleanup_stale_pending_stamps() and load_config() (in crates/relayburn-ingest/src/ingest.rs), both of which resolve paths from RELAYBURN_HOME. As a result, calling ingest with a non-default ledger_home can clean pending stamps and read content settings from the wrong ledger, violating the SDK contract for multi-ledger embeddings and potentially mutating another ledger’s state.
Useful? React with 👍 / 👎.
Fills in `crates/relayburn-sdk/src/ingest_verb.rs` with both forms of the SDK `ingest` verb: - `LedgerHandle::ingest(opts)` runs `relayburn_ingest::ingest_all` against an already-open handle. - Free `ingest(opts)` opens a fresh ledger via `Ledger::open` using `opts.ledger_home`, then delegates to the method. Defines a Rust-friendly `IngestOptions` (PathBuf + boxed Send+Sync sinks + `IngestRoots`) that maps onto `relayburn_ingest::IngestOptions` (re-exported from the SDK as `RawIngestOptions`). Re-exports `ProgressSink` / `WarnSink` aliases at the SDK surface since the underlying ingest crate keeps them as module-private types. Adds a `#[tokio::test]` that points `RELAYBURN_HOME` plus all three harness root dirs at empty TempDirs and asserts the verb returns an all-zero `IngestReport` without scanning the developer's home. Part of #246. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
458c862 to
6d3ecd8
Compare
Summary
Fill in `crates/relayburn-sdk/src/ingest_verb.rs` with the async `ingest` verb in both forms:
Both delegate to `relayburn_ingest::ingest_all`. The free function opens its own ledger via `Ledger::open(LedgerOpenOptions { home: opts.ledger_home, ..Default::default() })`. `IngestOptions` exposes `ledger_home`, `roots`, and the boxed `on_progress` / `on_warn` sinks the lower crate already supports.
The Rust port threads the ledger location through `Ledger::open` explicitly rather than swapping `RELAYBURN_HOME` (the TS sibling did the env-var dance), so embeddings can run multiple ledgers per process.
Per-harness verbs (`ingest_claude_projects`, etc.) intentionally not wrapped here — scoped to `ingest_all`; can be a follow-up.
Part of #246.
Test plan
Stacking
Stacked on #300 (module stubs) — base is `claude/sdk-module-stubs` and will rebase to `main` once that lands.
🤖 Generated with Claude Code