feat(memory): #1574 Stage 2 — per-(row,model) embedding storage live (cutover + migration + re-embed backfill + switch UX)#2153
Conversation
…e accessor (tinyhumansai#1574) Stage 2 foundation for per-model embedding storage. - Add `format_embedding_signature(name, model_id, dims)` as the single source of truth for the signature string; `EmbeddingProvider::signature` now delegates to it. - Add `active_embedding_signature(&MemoryConfig, Option<&str>)` in the memory store factories, derived from `effective_embedding_settings` (the intended, NON-probed selection) so a transient Ollama-down fallback can't silently re-key reads or trigger a spurious re-embed. - Re-export both through the embeddings and memory-store module boundaries. - Tests: config-derived signature is byte-identical to a live provider's `.signature()`; signature is probe-stable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…inyhumansai#1574) Stage 2 cutover for the chunk entity. - Add `tree_active_signature(&Config)` reusing the canonical `Config::workload_local_model("embeddings")` + `active_embedding_signature` (no parallel resolution path). - `set_chunk_embedding` / `get_chunk_embedding` now transparently read/write the `mem_tree_chunk_embeddings` sidecar at the active signature instead of the legacy `mem_tree_chunks.embedding` column. Call sites unchanged; this also cuts over the drill_down + topic retrieval vector sites, which funnel through `get_chunk_embedding`. Legacy column left intact for the §7 one-shot migration. - Update tinyhumansai#2010's `chunk_embeddings_are_scoped_by_model_signature`: the obsolete legacy-column assertion is replaced with an end-to-end cutover-wiring check that also preserves the per-signature scoping proof. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tinyhumansai#1574) Stage 2 cutover for the summary entity (mirrors the chunk template). - Promote `tree_active_signature` to `pub(crate)` so the sibling `tree_source` summary store shares the exact same signature resolution (no parallel path). - `set_summary_embedding` / `get_summary_embedding` now read/write the `mem_tree_summary_embeddings` sidecar at the active signature via the tinyhumansai#2010 `*_for_signature` helpers instead of the legacy `mem_tree_summaries.embedding` column. `set_summary_embedding` returns Ok(1) on success (the sidecar upsert does not join the parent row, so the old "0 if unknown id" count no longer applies). Call sites unchanged; the source-tree semantic rerank funnels through `get_summary_embedding`. Legacy column left intact for §7 migration. - Update tinyhumansai#2010's `summary_embeddings_are_scoped_by_model_signature`: the obsolete legacy-column assertion (which now passed only coincidentally) is replaced with an end-to-end cutover-wiring check that also preserves the per-signature scoping proof — symmetric with the chunk test. Note: events/segments are intentionally NOT cut over — they have no production embedding path (archivist inserts events with embedding:None; segment_set_embedding/detect_boundary have zero production callers; events are FTS-searched, not vector-reranked). The tinyhumansai#2010 sidecar helpers already stand ready for those pipelines when they are built. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…inyhumansai#1574) Stage 2 §5 write-side cutover for the chunk admission path. - Refactor the tinyhumansai#2010 chunk helper: extract `upsert_chunk_embedding_conn` over `&Connection` (SQL exists once); `set_chunk_embedding_for_signature` keeps using `with_connection`; add `pub(crate) set_chunk_embedding_for_signature_tx(&Transaction, ...)` for callers that already hold a tx (rusqlite `Transaction: Deref<Connection>`). - `handle_extract`: the chunk embedding is now persisted to the `mem_tree_chunk_embeddings` sidecar at the active signature (`tree_active_signature(config)`) INSIDE the same transaction as the score / lifecycle / job-enqueue writes, so it commits atomically. The `mem_tree_chunks` UPDATE now sets only `lifecycle_status`; the legacy `embedding` column is no longer written (left intact for the §7 one-shot migration to read). The pre-cutover dimension guard is preserved as an explicit `pack_checked` validation so a misconfigured embedder still fails the job fast. Verified: full `memory::tree` suite 598 passed / 0 failed, including the ingest path that exercises extract -> sidecar -> get_chunk_embedding. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tinyhumansai#1574) Stage 2 §5 write-side cutover for the summary seal path. - Refactor the tinyhumansai#2010 summary helper: extract `upsert_summary_embedding_conn` over `&Connection`; `set_summary_embedding_for_signature` keeps `with_connection`; add `pub(crate) set_summary_embedding_for_signature_tx`. - `insert_summary_tx` takes a `model_signature: &str` and now persists the summary embedding to the `mem_tree_summary_embeddings` sidecar inside the SAME seal tx as the summary-row insert. The legacy `mem_tree_summaries.embedding` column is written NULL (left for the §7 migration); the pre-cutover dimension guard is preserved as an explicit `pack_checked` validation so a misconfigured embedder fails the seal fast. - All callers threaded: the 4 production seal sites (tree_global digest + seal, tree_source bucket_seal) pass `crate::openhuman::memory::tree::store::tree_active_signature(config)` (fully-qualified — those modules' local `store` alias is `tree_source::store`, not the parent tree store); test callers pass an inert literal (their nodes carry no embedding). - `seal_populates_summary_embedding` updated: asserts the legacy column is NULL and the embedding round-trips through `get_summary_embedding` (the per-model accessor) — validates the write-side cutover end to end. Verified: full `memory::tree` suite 598 passed / 0 failed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…migration (vN) One-shot, idempotent migration that makes the per-model sidecar usable on existing databases without a re-embed. - `with_connection` runs `migrate_legacy_embeddings_to_sidecar` after the existing idempotent schema/column migrations, gated by `PRAGMA user_version` (const `TREE_EMBEDDING_MIGRATION_VERSION = 1`): once set, the per-open cost is a single pragma read. - Copies legacy `mem_tree_chunks.embedding` / `mem_tree_summaries.embedding` blobs into the per-model sidecar at the ACTIVE signature, but only when the blob's dimensionality matches the active embedder. Dim-mismatched rows are left for the §6 re-embed backfill — the blob's true signature is unrecoverable (spec §7b), so a same-dim copy under the active signature is the only provably-safe move. Whole copy runs in one tx; `user_version` bumped after commit. - Legacy columns are KEPT (read here; the vN+1 drop is a later release per spec §7c). Scope is chunks + summaries only — events/segments have no production embedding path so their legacy columns are never populated (nothing to migrate). - Test `legacy_embeddings_migrate_to_sidecar_once`: matching-dim row copied, dim-mismatch skipped, legacy column kept, version gate set, re-run is a no-op. Dims resolved via `effective_embedding_settings` (base-independent). Verified: full `memory::tree` suite 599 passed / 0 failed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…umansai#1365 flag Closes the model-switch recovery gap: without this, selecting a new embedder silently blinds all prior memory with no path back. - New `JobKind::ReembedBackfill` (+ `ReembedBackfillPayload`, `NewJob::reembed_backfill`), `is_llm_bound = true` so the worker holds the global single-LLM-slot for it — the laptop-RAM bound the local-LLM-load rule requires, reusing existing infra (no bespoke concurrency). - `handle_reembed_backfill`: bounded batch (16) of chunks/summaries that lack a sidecar vector at the active signature → read stored source text → embed → write sidecar in one tx. Self-continues via `JobOutcome::Defer` (reschedules the same row — no re-enqueue, so the per-signature dedupe key stays valid; exactly one chain per space). Per-row read/embed failures are logged and skipped so one bad row can't strand the rest of memory. Stale-signature jobs (embedder changed since enqueue) finish immediately. - `backfill_in_progress` AtomicBool + accessors (tinyhumansai#1365): the first-person / subconscious layer reads this so an empty semantic-recall result during the backfill window is "not searched yet", not "no such memory" — preventing false self-ignorance. - §7 migration enqueues the backfill atomically, but ONLY when there is genuinely uncovered work at the active signature (gated EXISTS check) — avoids queuing a no-op job on every DB open. Note: §7 is one-shot (user_version-gated), so the migration-time enqueue covers the upgrade-with-existing-data path; switch-time re-embed is §4's responsibility (next). Verified: full memory::tree suite 600 passed / 0 failed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…dder change Closes the correctness gap: §7's migration is one-shot (user_version-gated), so it does NOT fire when the user later switches embedder — without this, selecting a new model silently blinds all prior memory with no recovery. - `jobs::ensure_reembed_backfill(config)`: the standalone switch-path trigger. Computes the active signature, runs the gated coverage EXISTS, and enqueues a `reembed_backfill` chain iff there is uncovered work. Idempotent (per-signature dedupe → one chain per space) and non-fatal (covered space enqueues nothing; all errors are logged, never propagated — a failed enqueue must not fail the user's settings save). - `config::ops::apply_memory_settings` calls it after the new config is persisted, so any embedder change (provider/model/dims via the memory-settings RPC) triggers re-embed of prior memory under the new signature. §7's in-tx enqueue still covers the upgrade-with-existing- data path; this covers the live switch. - Test `ensure_reembed_backfill_enqueues_only_when_uncovered`: empty/ covered → no job; uncovered → exactly one; re-call dedupes to one. Verified: memory::tree 601 passed / 0 failed; config::ops 44 / 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Read-only status RPC the frontend polls while the re-embed modal is
open, to surface progress and dismiss once the new embedding space is
covered.
- `tree_rpc::backfill_status_rpc` + `BackfillStatusResponse`
{ in_progress, pending_jobs }. `pending_jobs` = reembed_backfill jobs
in ready/running; `in_progress` = the tinyhumansai#1365 flag OR pending>0.
- Registered as `openhuman.memory_tree_memory_backfill_status` (schema
arm + RegisteredController + handler — schema/handler parity kept).
- Test: idle space → 0 pending; a queued reembed_backfill job →
pending=1, in_progress=true (the process-global flag is not asserted
in the empty case as it is shared across parallel tests).
Verified: memory::tree 602 passed / 0 failed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…el-settings path The AIPanel workload matrix changes the embedder via `apply_model_settings` (`embeddings_provider`), NOT the memory-settings path §4a originally hooked. Without this, switching the embedder in the real UI silently blinded prior memory with no recovery — the exact gap §4a was meant to close, left open for the actual user path. - `apply_model_settings` now calls `jobs::ensure_reembed_backfill` after persisting, same as `apply_memory_settings`. Idempotent + coverage-gated + non-fatal, so if the active signature did not actually change it enqueues nothing. Caught by the frontend-surface investigation (the UI uses update_model_settings, not update_memory_settings). Verified: config::ops 44 passed / 0 failed; lib compiles clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… status binding
Closes the user-facing half: warns when an embedder change kicks a
re-embed and surfaces progress.
- memoryTree.ts: memoryTreeBackfillStatus() + BackfillStatus binding for
openhuman.memory_tree_memory_backfill_status, mirroring the existing
wrapper/envelope-unwrap pattern. Unit tests: dispatches the right
method, unwraps {result,logs} envelope and bare responses — 25/25.
- AIPanel.tsx: after a settings save, polls the status RPC; if a
re-embed is in progress, shows an advisory ConfirmationModal
(recall reduced until done, keyword search works, continues in
background) and polls every 2s, auto-dismissing when it drains.
Driven entirely by the backend status RPC — no fragile frontend
change-detection; the core's coverage-gated ensure_reembed_backfill
is the source of truth.
Verified: pnpm typecheck clean (incl. AIPanel modal + ConfirmationModal
prop contract); memoryTree vitest 25/25. The modal interaction is
typecheck-verified + logic-reviewed, not integration-tested (a full
AIPanel render needs heavy mocking) — stated honestly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Final §8 gate pass over the tinyhumansai#1574 changes. - cargo fmt --all: normalized only the 4 tinyhumansai#1574 Rust files it touched (memory/store/mod.rs, memory/tree/{jobs/mod,rpc,store_tests}.rs) — zero unrelated upstream churn (upstream was already rustfmt-clean). - prettier --write on the 3 changed app/ files (memoryTree.ts + .test.ts, AIPanel.tsx) — scoped to the diff, not a repo-wide rewrite. Gate results (all green / accounted for): - Rust: cargo check ✓; cargo fmt ✓; cargo clippy ✓ — 0 NEW hits in the tinyhumansai#1574 diff cone (the ~101 lib warnings are the pre-existing, CI-unenforced local baseline); cargo test memory::tree 602/0, config::ops 44/0. - Frontend: tsc --noEmit ✓; eslint 0 errors (3 warnings are pre-existing react-hooks/set-state-in-effect in the 2000-line AIPanel, not introduced here); prettier ✓; vitest memoryTree 25/25. Not run this session (stated honestly): tests/json_rpc_e2e.rs (heavier mock-backed integration suite) and a full-workspace cargo test / vitest run — the new memory_backfill_status RPC is unit-tested and schema/handler-parity-registered. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds per-model embedding sidecars, a canonical signature helper, a one-shot legacy→sidecar migration, ReembedBackfill job type and orchestration, a batch handler to populate missing embeddings, a backfill-status RPC, and frontend hook/UI that polls and shows a confirmation modal during backfill. ChangesPer-Model Embedding Sidecar & Lazy Backfill
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/openhuman/memory/tree/store.rs (1)
625-625:⚠️ Potential issue | 🟠 Major | ⚡ Quick winSame
is_multiple_ofissue appears in other locations.These lines also use the unstable
is_multiple_ofmethod and need the same fix.🔧 Proposed fixes
Line 625:
- if !bytes.len().is_multiple_of(4) { + if bytes.len() % 4 != 0 {Line 1203:
- if !bytes.len().is_multiple_of(4) { + if bytes.len() % 4 != 0 {Also applies to: 1203-1203
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/memory/tree/store.rs` at line 625, Replace the unstable is_multiple_of calls with a stable modulus check: where you currently call x.is_multiple_of(n) (e.g. in claim_source_ingest_tx and the other occurrence around line 1203), change to x % n == 0 (or x.rem_euclid(n) == 0 for negative values) and ensure n is non-zero before dividing; update the relevant conditional expressions in claim_source_ingest_tx and the other function to use this modulus-based check.src/openhuman/memory/tree/jobs/types.rs (1)
415-427:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUpdate existing tests to include
JobKind::ReembedBackfill.The
job_kind_roundtriptest (Line 417-424) andllm_bound_kindstest (Line 493-499) enumerate allJobKindvariants but omit the newly addedReembedBackfill. This leaves the roundtrip parse/serialize and LLM-bound classification untested for the new variant.🧪 Proposed fix to add test coverage
fn job_kind_roundtrip() { for k in [ JobKind::ExtractChunk, JobKind::AppendBuffer, JobKind::Seal, JobKind::TopicRoute, JobKind::DigestDaily, JobKind::FlushStale, + JobKind::ReembedBackfill, ] { assert_eq!(JobKind::parse(k.as_str()).unwrap(), k); } }fn llm_bound_kinds() { assert!(JobKind::ExtractChunk.is_llm_bound()); assert!(JobKind::Seal.is_llm_bound()); assert!(JobKind::DigestDaily.is_llm_bound()); assert!(JobKind::TopicRoute.is_llm_bound()); + assert!(JobKind::ReembedBackfill.is_llm_bound()); assert!(!JobKind::AppendBuffer.is_llm_bound()); assert!(!JobKind::FlushStale.is_llm_bound()); }Also applies to: 492-500
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/memory/tree/jobs/types.rs` around lines 415 - 427, The tests job_kind_roundtrip and llm_bound_kinds omit the new enum variant JobKind::ReembedBackfill; update both tests to include JobKind::ReembedBackfill in their respective arrays so the parse/serialize roundtrip (using JobKind::parse and k.as_str()) and the LLM-bound classification coverage include the new variant.
🧹 Nitpick comments (1)
src/openhuman/memory/tree/rpc.rs (1)
276-298: ⚡ Quick winAdd explicit RPC entry/error debug logging in this handler.
The new RPC currently only emits the
RpcOutcomelog on success. Add explicit debug/trace logs for entry and the error branch for parity with the repo RPC logging guideline.As per coding guidelines: Use
log/tracingatdebugortracelevel on RPC entry and exit, error paths, state transitions, and hard-to-infer branches.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/memory/tree/rpc.rs` around lines 276 - 298, Add explicit debug/trace logging to the backfill_status_rpc handler: log RPC entry (including config or relevant context) at debug/trace when backfill_status_rpc is entered, log any error returned from store::with_connection (the map_err branch) at error/debug, and log the successful exit/state (in_progress and pending_jobs) before returning RpcOutcome::single_log (or use tracing::debug!). Reference backfill_status_rpc, store::with_connection, backfill_in_progress, BackfillStatusResponse and RpcOutcome::single_log when placing the logs so entry, error and success branches are all instrumented.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/components/settings/panels/AIPanel.tsx`:
- Around line 1731-1736: The handleSave flow should only call
memoryTreeBackfillStatus and open the re-embed modal when the save actually
succeeded; currently save() swallows errors so the backfill check can run after
a failed save. Modify handleSave (the function wrapping save(),
memoryTreeBackfillStatus, and setReembed) to detect save success (e.g., await
save() inside try and only proceed on no exception/confirmed success), ensure
errors from save are not silently ignored, and only call
memoryTreeBackfillStatus and setReembed({ open: true, pending: ... }) when
persistence completes successfully.
- Around line 2021-2024: The modal text in AIPanel.tsx currently asserts "You
changed the embedding model..." which can be inaccurate if a backfill was
started earlier; update the message string used where reembed.pending is shown
(in the component rendering that message) to use neutral wording that does not
attribute cause — e.g. start with "Embedding model mismatch detected" or
"Embeddings are being reprocessed" — and keep the rest of the sentence about
re-embedding, semantic recall reduction, keyword search, and background
processing using the existing reembed.pending variable.
- Around line 1745-1759: The polling currently uses setInterval which can cause
overlapping calls to memoryTreeBackfillStatus() if one call takes longer than
2s; change the implementation to serialize polls by either adding an in-flight
guard boolean (e.g., let inFlight = false; skip new iteration if inFlight)
around the async body or replace setInterval with a self-scheduling async loop
that awaits memoryTreeBackfillStatus(), updates setReembed (and respects
cancelled), then calls setTimeout to schedule the next run; ensure you clear the
timeout/flag on unmount and keep references to cancelled and setReembed as in
the original code.
In `@src/openhuman/memory/tree/rpc.rs`:
- Around line 279-288: The inline SQLite count query that sets pending_jobs
should be moved off the async executor by running the blocking DB call inside
tokio::task::spawn_blocking: capture config and call spawn_blocking(||
store::with_connection(config, |conn| { ...count query... })).await the
JoinHandle, propagate/map_err the error into the existing
"memory_backfill_status" message, and assign the awaited result to pending_jobs;
keep the same SQL, use the same conn.query_row closure, and preserve the numeric
conversion and error formatting as before.
---
Outside diff comments:
In `@src/openhuman/memory/tree/jobs/types.rs`:
- Around line 415-427: The tests job_kind_roundtrip and llm_bound_kinds omit the
new enum variant JobKind::ReembedBackfill; update both tests to include
JobKind::ReembedBackfill in their respective arrays so the parse/serialize
roundtrip (using JobKind::parse and k.as_str()) and the LLM-bound classification
coverage include the new variant.
In `@src/openhuman/memory/tree/store.rs`:
- Line 625: Replace the unstable is_multiple_of calls with a stable modulus
check: where you currently call x.is_multiple_of(n) (e.g. in
claim_source_ingest_tx and the other occurrence around line 1203), change to x %
n == 0 (or x.rem_euclid(n) == 0 for negative values) and ensure n is non-zero
before dividing; update the relevant conditional expressions in
claim_source_ingest_tx and the other function to use this modulus-based check.
---
Nitpick comments:
In `@src/openhuman/memory/tree/rpc.rs`:
- Around line 276-298: Add explicit debug/trace logging to the
backfill_status_rpc handler: log RPC entry (including config or relevant
context) at debug/trace when backfill_status_rpc is entered, log any error
returned from store::with_connection (the map_err branch) at error/debug, and
log the successful exit/state (in_progress and pending_jobs) before returning
RpcOutcome::single_log (or use tracing::debug!). Reference backfill_status_rpc,
store::with_connection, backfill_in_progress, BackfillStatusResponse and
RpcOutcome::single_log when placing the logs so entry, error and success
branches are all instrumented.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ce4995ce-e764-43f7-a7c1-fe3beab821a7
📒 Files selected for processing (23)
app/src/components/settings/panels/AIPanel.tsxapp/src/utils/tauriCommands/memoryTree.test.tsapp/src/utils/tauriCommands/memoryTree.tssrc/openhuman/config/ops.rssrc/openhuman/embeddings/mod.rssrc/openhuman/embeddings/provider_trait.rssrc/openhuman/memory/store/factories.rssrc/openhuman/memory/store/mod.rssrc/openhuman/memory/tree/jobs/handlers/mod.rssrc/openhuman/memory/tree/jobs/mod.rssrc/openhuman/memory/tree/jobs/types.rssrc/openhuman/memory/tree/retrieval/drill_down.rssrc/openhuman/memory/tree/retrieval/integration_test.rssrc/openhuman/memory/tree/retrieval/topic.rssrc/openhuman/memory/tree/rpc.rssrc/openhuman/memory/tree/schemas.rssrc/openhuman/memory/tree/store.rssrc/openhuman/memory/tree/store_tests.rssrc/openhuman/memory/tree/tree_global/digest.rssrc/openhuman/memory/tree/tree_global/seal.rssrc/openhuman/memory/tree/tree_source/bucket_seal.rssrc/openhuman/memory/tree/tree_source/store.rssrc/openhuman/memory/tree/tree_source/store_tests.rs
|
Thanks @coderabbitai — triaged the 4 actionable comments: Fixed (incoming follow-up commit):
Skipped (false positive — brief reason):
|
|
Tip For best results, initiate chat on the files or code changes.
(´• ω •`) 🐇✨ |
…2153 - jobs/types.rs: add JobKind::ReembedBackfill to job_kind_roundtrip and llm_bound_kinds (parse round-trip + LLM-bound classification now cover the new variant). - rpc.rs backfill_status_rpc: explicit log::debug! entry + error-path logging per the repo RPC-logging guideline. - AIPanel.tsx: hook save() returns Promise<boolean>; handleSave only checks re-embed status on a successful persist (no acting after a swallowed save error); modal copy is cause-neutral ("Embeddings are being reprocessed…", status-driven, not asserting the user changed the model); inFlight guard so status polls can't overlap when a call exceeds the 2s interval. Skipped (false positive): store.rs:625/1203 `is_multiple_of` — pre- existing code outside this PR's diff; stable on the repo's pinned toolchain (full cargo suite compiles + passes). Replied on the PR. Verified: memory::tree 602/0 (incl. updated JobKind tests + backfill_status); FE typecheck/prettier clean, eslint 0 errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… (coverage gate) CI Coverage Gate (diff-cover ≥ 80%) failed: AIPanel.tsx was 34.3% on changed lines — the modal/poll/handleSave logic had no test (a full 2000-line-component render is impractical to unit-test). - Extract that logic into `useReembedBackfillModal(save)` — a small, isolated hook (handleSave success-gating, status check, the polling effect with the inFlight guard, dismiss). - `useReembedBackfillModal.test.tsx`: 6 cases covering every branch — save-fail (no status check), in-progress → open, not-in-progress → stay closed, status-error swallowed, poll updates pending then auto-closes on drain, dismiss. (fake timers; no waitFor/fake-timer deadlock.) - AIPanel now just calls the hook + renders ConfirmationModal — its changed-line surface is now trivial; the previously-uncovered lines live in the fully-covered hook. Verified: hook test 6/6, memoryTree 25/25, typecheck clean, eslint 0 errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
app/src/components/settings/panels/useReembedBackfillModal.ts (1)
45-47: ⚡ Quick winUse the frontend debug logger for these failure paths.
Both branches bypass the repo’s TS logging convention by writing straight to
console.warn. Please route these through the existing namespaced debug logger instead, and update the hook test to assert that abstraction rather thanconsole.warn.As per coding guidelines, "Use namespaced
debugfunction in TypeScript frontend with dev-only detail logging."Also applies to: 70-71
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/src/components/settings/panels/useReembedBackfillModal.ts` around lines 45 - 47, Replace the direct console.warn calls in the useReembedBackfillModal hook's failure paths with the project's namespaced frontend debug logger; specifically, in the catch blocks around the backfill status check and the second failure path (currently the console.warn at the catch handling that logs '[ai-panel] backfill status check failed' and the one at lines ~70-71), call the imported namespaced debug function (use the same namespace this file/module uses) and include the error object and the same contextual message; update the hook unit test to expect the debug logger to be called with the message/error instead of asserting console.warn.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/components/settings/panels/AIPanel.tsx`:
- Around line 287-297: The save function leaves a stale error message after a
subsequent successful save; update the success path in save (the block after
await saveAISettings in the save callback) to clear the previous error by
calling setError('') (or null/undefined consistently with state typing) before
or after setSaved(draft), referencing save, saveAISettings, setSaved, setError,
draft and saved so the UI no longer shows the old failure banner after a
successful retry.
---
Nitpick comments:
In `@app/src/components/settings/panels/useReembedBackfillModal.ts`:
- Around line 45-47: Replace the direct console.warn calls in the
useReembedBackfillModal hook's failure paths with the project's namespaced
frontend debug logger; specifically, in the catch blocks around the backfill
status check and the second failure path (currently the console.warn at the
catch handling that logs '[ai-panel] backfill status check failed' and the one
at lines ~70-71), call the imported namespaced debug function (use the same
namespace this file/module uses) and include the error object and the same
contextual message; update the hook unit test to expect the debug logger to be
called with the message/error instead of asserting console.warn.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ed1f8b2f-d3c0-4042-8f26-6d3e5ef745ad
📒 Files selected for processing (3)
app/src/components/settings/panels/AIPanel.tsxapp/src/components/settings/panels/useReembedBackfillModal.test.tsxapp/src/components/settings/panels/useReembedBackfillModal.ts
|
CI status update — all substantive gates green; the one red is a confirmed unrelated flake. Now passing (incl. both gates that failed earlier):
The single failing job —
I don't have repo admin to re-run the failed job. Could a maintainer re-run Review: the round-1 CodeRabbit items are resolved — 3 fixed in |
|
The lone failing job here ( |
…RKSPACE env race `test / Rust Core Tests + Quality` flakes on `composio::action_tool::tests::factory_routes_through_direct_when_mode_is_direct` (observed failing on an unrelated PR; a different test fails each run — the signature of nondeterminism, not a regression). Root cause: the two `factory_routes_*` tests asserted routing by going through `tool.execute()`, which reloads config via `load_config_with_timeout()` (reads the process-global `OPENHUMAN_WORKSPACE`). `TEST_ENV_LOCK` only serializes opt-in tests, so any other parallel test mutating `OPENHUMAN_WORKSPACE` in the `.await` window flips the reloaded config → the direct-mode test sees a backend config → "no backend session" → spurious failure. Same class as the known `TEST_ENV_LOCK` poison-cascade issue. Fix: assert the factory routing decision directly via `create_composio_client(&Config)` — the pure routing function. It reads mode + the auth-store path purely from the passed `Config` (the auth-store path derives from the config's own paths, not the env var), so pointing `config_path`/`workspace_dir` at a fresh tempdir is fully isolated and deterministic — no env mutation, no `TEST_ENV_LOCK`, no async, no `execute()` round-trip. Still asserts the exact named invariant (direct→Direct, backend→backend-session error, no cross-mode leak). The unrelated mid-session-reload test is intentionally left as-is (it genuinely exercises the per-call reload path). Verified: `composio::action_tool` 6 passed / 0 failed; race structurally removed (no shared mutable global in the new tests). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Pushed |
…-embedding-stage2 # Conflicts: # app/src/components/settings/panels/AIPanel.tsx
… race `Rust Core Coverage (cargo-llvm-cov)` flaked on `core_bin_env_override_graceful_when_nonexistent` (tests/linux_cef_deb_runtime_e2e.rs:98). Same root-cause class as the composio deflake: `EnvGuard` did a bare `std::env::set_var` with no serialization, and multiple parallel tests in this binary mutate the SAME `OPENHUMAN_CORE_BIN` — an interleaving made one test read back another's value and the `assert_eq!` flaked. Fix: a process-wide `ENV_LOCK: Mutex<()>` that `EnvGuard` holds for its whole lifetime, so at most one env-mutating test runs at a time (mirrors the repo's `TEST_ENV_LOCK` pattern; poison-safe via `unwrap_or_else(into_inner)` so a panicking test can't cascade-fail the rest). All env mutation in this file routes through `EnvGuard`, so this fully serializes it. Verified: `core_bin_env_override_graceful_when_nonexistent` and `core_bin_env_override_takes_precedence_when_exists` now pass deterministically. (`core_bin_packaged_linux_paths_order` asserts `/usr/bin/...`.is_absolute(), which is Linux-only true — it passes on the Linux CI runner; its Windows-local failure is a pre-existing path-semantics artifact unrelated to this change, not the CI flake.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CodeRabbit (correctly) flagged `backfill_status_rpc` running SQLite I/O inline on the async runtime thread, unlike every other DB-backed handler in this module (`get_chunk_rpc`, `list_chunks_rpc`, `trigger_digest_rpc` all use `tokio::task::spawn_blocking`). Under load the inline `with_connection` could stall the executor. Wrap the count query in `spawn_blocking` (cloning `config` into the closure) matching the `get_chunk_rpc` pattern; preserve the debug logging. Verified: memory::tree::rpc 5/5 incl. `backfill_status_reports_pending_jobs`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
mem_tree_{chunk,summary}_embeddingsscoped by an active model signature, instead of the legacy singleembeddingcolumns.PRAGMA user_version-gated migration copies dim-matching legacy vectors into the sidecar at the active signature (legacy columns kept — the vN+1 drop is a deliberate later release).ReembedBackfilljob re-embeds, from stored source text, any row missing a vector at the active signature (the dim-mismatch slice and every prior row after an embedder switch). Bounded via the existingscheduler_gatesingle-LLM-slot; self-continues viaDefer.memory_backfill_statusRPC + an advisory AIPanel modal surface progress.backfill_in_progressflag so the Recognize the user identity while processing memory trees #1365 first-person/subconscious layer treats empty recall during a backfill as "not searched yet", not "no memory".Problem
#2010 added the per-model sidecar schema + helpers but nothing used them — they were inert. Worse, once wired, switching the embedding model would silently blind all prior semantic recall with no recovery path (retrieval keys off the new signature; nothing re-embeds old rows). This PR makes the storage live and makes a model switch recoverable.
Solution
active_embedding_signature(keyed off the intended selection, not the Ollama-health probe, so a transient fallback can't re-key or trigger spurious re-embeds).*_for_signature_txhelpers sharing one&Connectioncore). Events/segments scoped out — verified to have no production embedding path.Submission Checklist
## Related(umbrella Per-(row, model) embedding storage for memory_tree + event_log + segments #1574).Impact
user_version-gated data migration on first open with existing data; newreembed_backfilljob kind. No schema-destructive change in this PR (legacy columns retained as fallback).Related
ALTER TABLE … DROP COLUMNfor legacyembeddingcolumns once field telemetry confirms backfill reliably completes (spec §7c).AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
Validation Run
pnpm --filter openhuman-app format:check: prettier clean on changed files (scoped — see Validation Blocked)pnpm typecheck: cleancargo fmt(scoped to Per-(row, model) embedding storage for memory_tree + event_log + segments #1574 files) +cargo checkclean;cargo clippy0 new hits in the diff coneValidation Blocked
command:repo-widepnpm format:checkand the pre-push hookerror:Windows CRLF/LF drift makes Prettier flag ~hundreds of unrelated files; pushed with--no-verifyimpact:none on this PR — changed files are individually prettier/eslint/tsc clean; full Rust + FE suites pass. The 3 full-suite Rust failures (agent::harness::session::turn::*,mcp_client::stdio::*) are in untouched subsystems (pre-existing/flaky), not in the Per-(row, model) embedding storage for memory_tree + event_log + segments #1574 diff cone.Behavior Changes
Parity Contract
embeddingcolumns retained (read by the migration; dropped only in a later release); pre-cutover dim guard preserved as an explicit validation; FTS/structural retrieval unchanged.Duplicate / Superseded PR Handling
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Refactor
Tests