feat(agent): overhaul cloud agent reliability, streaming, and error handling - #10256
Merged
Conversation
…exploration Declare a 'researcher' subagent via the Claude Agent SDK agents option, restricted to the three omi-tools data tools (execute_sql, semantic_search, get_daily_recap). Broad/exploratory data questions now run in a disposable child context that returns distilled findings, keeping raw query rows out of the main session's context window — the main driver of long-session degradation. Task added to allowedTools (required for subagent invocation); option assembly extracted to query-config.mjs so the definitions are unit testable (first tests in agent-cloud). Verification: - npm test (agent-cloud): 3/3 pass (researcher tool allowlist, schema in prompt, Task in allowedTools). - Live one-shot CLI run against a seeded local omi.db copy (120 screenshots, 3 action items): model output began "I'll delegate this to the researcher agent", returned correct distilled findings (60/30/30 app split, rotation pattern, open tasks) with no raw rows. SDK 0.2.41 accepted the agents option end-to-end. Note: CLI-mode tool_activity logging does not surface subagent tool calls; delegation evidenced by model output and cost profile.
cancelRun only cancelled its target: delegated (run-and-wait) children kept running as orphans after the user stopped the parent. The delegations parent_run_id link existed but nothing traversed it on cancel — only the mid-batch spawn-failure path (control-tools sibling cleanup) reaped children. cancelRun now cascades best-effort to every delegated child after the parent's own cancellation is persisted and dispatched. Recursion covers grandchildren; terminal children hit the existing accepted:false early-return (idempotent). Delegation rows stay untouched — executeDelegationAsync settles them to "cancelled" when the child's execution resolves, so the parent's pending await resolves normally with no double-settle. Background agents spawned via spawn_background_agent have no delegation row and are deliberately NOT cascaded: they are user-visible floating-bar work designed to outlive the parent turn (pinned by regression test). Intentional edges: cancelling an already-terminal parent does not reap still-running spawn-mode children (early-return precedes cascade); the multi-spawn failure cleanup now cascades transitively through cancelled delegated siblings. Failure-Class: none Verification: - npx vitest run tests/cancellation.test.ts: 5/5 pass (3 new: in-flight delegated child cascades with single delegation settle; terminal child untouched; background agent not cancelled). - Confirmed the new cascade test fails on the unfixed kernel-runs.ts (adapter.cancelled length 1 vs expected 2) and passes with the fix. - npx tsc --noEmit: clean.
Add optional toolPolicy.allowedToolNames to SpawnBackgroundAgentInput and DelegateAgentInput. The policy travels in run metadata JSON (already persisted by both spawn paths and already parsed by the capability broker — no schema change) and is enforced at capability registration: the broker intersects it with the role/adapter-computed tool set before freezing allowedToolNames, so every authorize() call is covered. Empty or malformed policy fails closed to no tools. Restriction-only semantics: a policy can narrow, never widen, the child's tool set. Exposed on the spawn_agent and spawn_background_agent control-tool schemas and manifests so the coordinator model can restrict children (Claude Code principle: specialized subagents with reduced toolsets). run_agent_and_wait deliberately skipped until needed. Generated tool surfaces regenerated via agent/scripts/generate-tool-surfaces.mjs (GeneratedToolCapabilities.swift, GeneratedToolExecutors.swift, tool-manifest fixture). Verification: - npm test (desktop/macos/agent): 49 files, 587/587 pass, including new cases: broker intersection honored (get_memories allowed, search_memories tool_not_allowed), empty/malformed policy fails closed, policy persisted through both spawn paths, absent policy leaves behavior unchanged, both spawn schemas accept valid toolPolicy and reject malformed shapes. - npx tsc --noEmit: clean. - Note: runtime-stdio-contract tests require a built dist/ (fresh-worktree timeout is environmental, passes after npm run build; verified unrelated to this change by stash-testing).
Evidence (2026-07-21, 26 instrumented runs on the mini against a real 27MB omi.db copy — 5,110 screenshots, 305 open tasks, 1,059 memories): - haiku researcher: fastest (44s vs 64s inherited-Opus forced runs) and ~3x cheaper ($0.13 vs $0.44 on content recall) with equal-or-better accuracy (pinned the correct last-seen date where baseline was off by one). - Delegation tax (~20s) only pays off on row-heavy tasks; natural delegation correctly fired on patterns/triage/recall and stayed home on quick lookups. - Worst observed pathology: researcher called get_daily_recap once per day (12-turn run). Prompt-level turn-discipline instructions were tested and are model-dependent (Opus obeys, haiku gets worse) — rejected; fixing the tool description is model-independent. Changes: pin researcher model to haiku; route delegation by row-volume in both the agent description and the main-prompt guideline; state in both the tool description and researcher prompt that one get_daily_recap call covers any date range. Verification: npm test 3/3 (model + range-wording pinned); full 8-turn session sim on real data ran the researcher config 23% faster and 22% cheaper than baseline with parity accuracy (results in PR discussion).
… progress, interrupted partials — with hermetic e2e Root cause found by instrumented session sims on real data: agent.mjs never set includePartialMessages, so its entire stream_event handling was dead code — no token ever streamed, text arrived only as whole assistant messages, and every turn contained 17-46s of total client silence (measured; the mechanism behind 'Response stopped' churn). Enabling partials cut measured max silent gaps to ~13s before the mini link dropped mid-validation; the durable guard is the new hermetic e2e below. Changes: - includePartialMessages: true in both query option blocks; the existing delta/dedup handling was already written for it. - stream-routing.mjs: subagent-origin messages (parent_tool_use_id) surface tool starts as 'subagent:<name>' tool_activity and never leak internal text into the answer stream. - result messages carry interrupted: true when a stop/preemption cut the turn short — clients can finally distinguish partials from complete answers. - Fix (found by the e2e): the prewarm-completion branch reset turnActive, clobbering a query that arrived before prewarm finished and turning that turn's stop into a silent no-op. Prewarm never sets turnActive; it no longer clears it. - Controllable SDK seam (OMI_AGENT_SDK_MODULE) + OMI_AGENT_DISABLE_UPDATES gate so the server runs hermetically under test. - tests/agent-ws-e2e.test.mjs: spawns the real server with a scripted SDK fixture, drives a real WebSocket, asserts: text streams as multiple deltas, subagent progress surfaces, the SECRET-INTERNAL leak canary never reaches the client, and stopped turns return interrupted: true with partial text. - agent-cloud-tests registered in .github/checks-manifest.yaml (local + ci lanes) — also puts the existing unit tests under CI for the first time. Verification: bash desktop/macos/agent-cloud/scripts/run-tests.sh from clean node_modules — 3 files, 8/8 pass (unit + e2e). python3 .github/scripts/test_run_checks.py — manifest contract OK. Live partial validation on the mini (2 turns before network drop): max_gap 20-46s → ~13s, subagent tool events now visible to the client.
…ntact Reconnect amnesia, reproduced live on real data before this fix: stop a turn, reconnect (what the phone does after backgrounding), ask "which of those task themes you listed earlier had the most items?" — the agent replied "I don't have any earlier conversation context here". The session was owned by the WS connection and torn down on close. The VM is single-user, so the persistent session now lives at server scope with a swappable client sink: disconnect detaches the sink (in-flight turns keep running; their result text survives for the reattached client), reconnect reattaches, and a session_state hello tells the client whether a live session with context exists. detachSink only clears its own socket's sink — a stale close event firing after a newer socket attached must not silence the fresh connection (race caught by the e2e while writing it). Follow-up (proxy-side, separate PR): agent-proxy injects last-10-message history on each connection's first query; with surviving sessions that becomes redundant on reconnect — the session_state hello is the signal it should key on. Failure-Class: none Verification: npm test — 9/9 including the new e2e case: turn counter proves the same SDK session consumes queries across a disconnect/reconnect (a fresh session would restart the count); hello reports active:true.
…l-battery evidence Evidence loop (2026-07-22, 18 live runs + 2 session sims on a real 27MB DB): round 1 exposed four waste patterns; round 2 after these changes measured wall −62%, tool calls 53→20, cost −33% across the affected tasks with correctness verified against ground truth (dupes 297/8 exact, Jul-14 4x activity ratio, honest no-data answer, Warp 37% share). Tool reshapes (new data-tools.mjs, unit-tested, injection-safe dates): - get_daily_recap accepts start_date/end_date — it could only anchor to "now" (days_ago), so date-specific recaps fell back to 13+ hand-rolled SQL turns (82s→29s, 17→2 tools). Short ranges add an hourly timeline and top window titles. - Empty ranges return an authoritative zero-activity statement — an empty-ish reply previously triggered 7 verification queries (39s→21s). - New get_app_usage: day-by-app matrix in one call, replacing per-day GROUP BY spelunking (compare task 73s→44s, 16→3 tools). Researcher gets it. - Task near-duplicate guidance (COUNT(DISTINCT description)) — the store carries ~63% near-duplicate task extractions; the dupes task dropped 221s→28s while matching exact ground truth. - Documented negative result: the ~2 ToolSearch turns per conversation are SDK schema deferral; allowedTools naming does not preload (tested). WS contract (session sims on fixed streaming code): - Turn heartbeat (OMI_TURN_HEARTBEAT_MS, default 10s): light turns now gap 1-3s but delegated turns still went 40-64s silent while subagent text is (deliberately) swallowed — a periodic status caps the client-visible gap regardless of source. - Real-SDK interrupts terminalize with a non-success subtype; a user stop previously surfaced as an error. Non-success + interruptRequested now maps to an interrupted:true partial result. Fixture models the real subtype. Also validated this cycle: reconnect memory probe answers from context in 5.5s with zero tool calls (was: total amnesia). Verification: npm test — 4 files, 15/15 (data-tools unit tests; e2e asserts heartbeats during held turns and interrupted partials). Battery and sim logs: scratchpad eval/ (round1 vs round2 TSVs).
Collaborator
|
Thanks for the careful work here — the direction looks useful and the focused regression coverage is strong. I reviewed this as an agent-control/agent-behavior change. The important maintainer consequence is that this PR changes how Omi agents are instructed and constrained:
That all looks conceptually safe from this pass: the tool policy intersects with the existing role/adapter allowlist rather than widening it, malformed persisted policy fails closed, and the cancellation behavior is covered by regression tests. I also ran the targeted tests locally:
I’m leaving this as a positive signal rather than formal approval because it still warrants human maintainer sign-off before merge: it changes agent behavior/control-plane semantics, touches a lockfile/dependency surface, and GitHub still shows blocked/pending/cancelled checks. Once the queued/cancelled CI state is resolved, this looks like a good candidate for maintainer review. by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with |
Git-on-my-level
added
positive-signal
Rule table over the observed 30-day chat_agent_error corpus: billing exhaustion, auth expiry, OAuth timeout, connection, oversized payload, runtime crash, tool-schema rejection, rate limit/overload, local data errors. Each class carries honest user copy and a retryable verdict — the opaque bucket's top cause (exhausted Anthropic credits, 117/401 events) was being told 'please try again', producing measured retry storms (32 events from 2 users on Jul 6 alone). Verification: AgentErrorClassifierTests (9 behavioral cases, committed with the test file later in this series); full build green.
userFacingAgentErrorMessage delegates to AgentErrorClassifier — copy and retry advice now come from one bounded table instead of ad-hoc substring checks that collapsed most causes into 'Something went wrong. Please try again.' Verification: build green; behavior pinned by AgentErrorClassifierTests.
…etry chat_agent_error carried only the coarse error_class enum; the daemon's typed failure envelope (failureCode/source/adapterId/provider/retryable) was parsed into AgentRuntimeFailure and then dropped, which is why the churn dashboard needed raw-string archaeology. The failed event now threads an optional ChatQueryErrorDetail: error_code (classifier code or daemon failureCode), retryable, failure_code, failure_source, adapter_id, provider — all closed vocabularies, never raw exception text. Note: enum case gains a slot, so this commit compiles only with the two follow-up call-site commits (per-file commit series per maintainer request; series tip is build- and test-green). Verification: ChatQueryTelemetryTests 25/25 incl. new payload tests asserting bounded properties and no raw-text leakage.
Pattern-match gains the detail slot; no behavior change. Verification: build green at series tip.
The disposition-classify site has the full Error in hand — derive ChatQueryErrorDetail.from(error) so agentError strings and runtime failure envelopes reach telemetry as bounded dimensions. Verification: build green; payload pinned by ChatQueryTelemetryTests.
Nine cases mirroring the observed error corpus: billing exhaustion must not say 'try again', auth expiry routes to reconnect, connection and runtime crashes are retryable, local data errors are classified for bug tracking, tool-schema rejections don't blame the user, unknown preserves readable messages.
Asserts error_code/retryable reach the payload, daemon taxonomy fields survive from AgentRuntimeFailure, and raw exception text never enters analytics properties.
Collapse the detail-threading call to one line (fits the 120-col limit) instead of raising the 6214-line ratchet baseline.
AgentBridge.swift shrank when the ad-hoc error heuristics moved to the classifier; the down-only ratchet records the new ceiling.
Error copy for chat failures is exercised by the same hermetic chat flow that covers AgentBridge and ChatQueryTelemetry.
Pre-existing drift on main (blocks every branch's swift-format-lint preflight); formatted with the pinned wrapper, no behavior change.
…tchet The pinned swift-format output for the drifted file is 4544 lines; the 4543 baseline froze the unformatted shape, wedging swift-format-lint against the ratchet for anyone touching the file. Formatter shape, not feature growth.
Second drifted file from the same upstream refactor (d9947b5); formatted with the pinned wrapper, no behavior change. Full lint-scope is clean after this.
Delegated turns were the longest client-visible silences (40-64s of generic heartbeats while the subagent generated text we swallowed — experiment report gap #3). stream-routing now exposes createSubagentRouter(): subagent text deltas surface as throttled (2.5s) ephemeral status events carrying a rolling 90-char snippet, so the wait shows real progress. Contract sharpened, not loosened: subagent text still never enters the answer stream (text_delta / result); the e2e leak canary now pins exactly that split and requires at least one Researching… progress event during delegation. Verification: - npm test (agent-cloud): 17/17 pass, including updated hermetic WS e2e asserting canary absent from answer stream + progress status present. - node --check agent.mjs clean.
…cale bench (E6) E6 scale experiment (deterministic 600K-row inflation, local 2026-07-22): every data-tool query holds at churned-user scale — full 7-day recap 175ms p50, worst single call (30-day app matrix) 126ms, FTS recall sub-ms. SQL is not the churn driver; the covering idx_screenshots_timestamp serves every range-comparison shape (EXPLAIN: SEARCH). date()-wrapped WHERE full-scans (31ms vs ~1.5ms) — so the idx_screenshots_date_local creation, which also failed with 'non-deterministic date()' on every open, is removed as dead. scripts/inflate-db.mjs (seeded, reproducible) + scripts/bench-data-tools.mjs let any machine (Mac mini pending power-on) rerun the exact experiment. Verification: bench run at 600000 rows locally; npm test 17/17; node --check clean.
execute_sql now accepts a queries[] array run in one call with per-query error isolation and labeled results (data-tools.runSqlBatch). Independent queries (comparisons, multi-table, multi-range) collapse to ONE tool call — the tool-shape lever, since prompt-level batching was a measured negative result. E7 local run: a 3-range task comparison issued 1 call x 3 statements (twice) instead of 6 sequential calls, correct answer, 44s. experiments/user-profile.json records the real average omi user from PostHog (project 302298, 30d, via Composio): median chat user sends 3 msgs/30d with 0 tool calls (churn is the p99 tail at 43 tools/msg), voice is ~half of chat usage, and the median user is mobile with ~37 tasks / 7 memories and no screenshots. Grounds experiments in real distributions instead of the synthetic uniform 600K profile. Verification: npm test 19/19; live E7 run against seeded db; node --check clean.
… unknown The 413 branch only matched "413 " (trailing space) or "413" alongside "request body", so a bare 'HTTP 413' at end of string fell through to unknown (retryable=true), inviting a pointless retry of an oversized payload. Match \b413\b (word-boundary, so '1413' does not false-positive) plus the 'payload too large' reason phrase. Also drop the redundant == "response stopped." already covered by the following hasPrefix.
The decision guard matched only .critical, so .extreme (>= 3 GB, the worst level) returned shouldReportCritical=false / shouldRemediate=false and fell to a low-severity path with no remediation. Include .extreme in the guard. New hermetic tests cover the extreme-reports and warning-does-not paths.
The vector search loaded every embedded screenshot in the time window into JS (each a 3072-float blob) before ranking and slicing to top-15, so a heavy history could materialize unbounded rows/memory. Cap the scan to the most recent 5000 embedded rows (LIMIT after ORDER BY timestamp DESC); ranking still runs in JS over the capped set. ponytail: most-recent-N scan, raise if old-screen recall matters more than the memory ceiling.
…ements # Conflicts: # .github/scripts/product_file_line_count_ratchet_baseline/desktop-swift-floatingcontrolbar.json
Git-on-my-level
previously requested changes
Jul 24, 2026
Git-on-my-level
left a comment
Git-on-my-level
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for continuing to harden this. I’m changing this from a positive signal to changes requested because the current head has concrete merge blockers that need to be resolved before maintainer review can be effective.
Blocking items:
- Hygiene is failing on the product-invariant gate: the PR touches
web/app/src/components/memories/MemoriesPage.tsxbut the PR body does not includeINV-MEM-1underProduct invariants affected. The CI log gives the exact block to paste, includingINV-AGENT-*,INV-AUTH-1,INV-CHAT-1, andINV-MEM-1. - Desktop Swift Test Suites compiled successfully but was cancelled at the 60-minute job limit. Please either make the selected suite complete inside the CI budget or narrow/split the manifest-selected Swift test route so the changed desktop coverage is actually green.
- Desktop Swift Build & Tests is failing because it depends on that cancelled Swift test result.
Agent-behavior / maintainer-impact note: this still changes agent-control guidance and behavior. The PR adds/updates cloud-agent research findings and instructions, researcher-subagent routing, batch SQL/data tools, cloud-agent streaming/error handling, and spawn/delegation tool-policy constraints. The important maintainer consequence is that AI coding/review/runtime agents may explore user data differently, delegate row-heavy DB work to a constrained subagent, and rely on the new tool-policy/cancellation contracts. The direction still looks conceptually safe from this pass because the policy narrows rather than widens access and the agent-cloud tests are passing, but this is enough behavior/control-plane surface that human maintainer sign-off remains required after CI is green.
I’m not asking to split the PR just because it is large; the work is mostly cohesive around agent reliability/control-plane behavior. The request is to clear the specific invariant/CI blockers and keep the existing human/dependency review path before merge.
by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.
Git-on-my-level
added
needs-tests
…ements # Conflicts: # .github/scripts/product_file_line_count_ratchet_baseline/desktop-swift-chat.json # .github/scripts/product_file_line_count_ratchet_baseline/desktop-swift-floatingcontrolbar.json # .github/scripts/product_file_line_count_ratchet_baseline/desktop-swift-providers.json # backend/requirements.txt # desktop/macos/e2e/flows/chat-hermetic.yaml
The .failed telemetry fix touched AnalyticsManager.swift and ChatQueryTelemetry.swift, which desktop-e2e-flow-coverage reported as uncovered. chat-hermetic already exercises them end to end: it sends a chat query, ChatProvider emits ChatQueryTelemetry events, and AnalyticsManager converts them to the analytics payload. Declare that coverage explicitly.
Repo Checks fires on both code events (push/synchronize) and metadata events (edited/labeled/unlabeled), and the two run disjoint job sets: metadata-preflight for the latter, changes/hygiene/formatting for the former. They shared one cancel-in-progress concurrency group, so editing a PR title or body cancelled an already-running code lane — and GitHub reports those cancelled jobs as failing checks. That is the flaky red seen on #10256: Formatting and Hygiene showed 'fail' at 0s while the sibling run of the same workflow passed in 3m13s. Key the concurrency group by lane so newer-cancels-older still applies within each lane but never across them. Verified: actionlint v1.7.12 (the pinned CI version) passes on the workflow, and the lane expression resolves to code for push/synchronize/opened/reopened and metadata for edited/labeled/unlabeled.
…udget The pre-session-id prewarm case spawns its own agent server, but carried the 20s timeout used by the cases that reuse the shared server from beforeAll. waitForServer alone can burn 15s on a loaded CI runner, leaving under 5s for the WS connect and prewarm round trip — a flake waiting to happen on slow runners. Match beforeAll's 30s budget, which covers the same spawn plus wait. Verified: desktop/macos/agent-cloud/scripts/run-tests.sh (the CI entrypoint) passes 61/61.
`MemoryPressureEpisodeTracker.evaluate` was widened so `.extreme` also returned shouldReportCritical/shouldRemediate. The only reader of those flags is `ResourceMonitor.sampleResources`, and it gates them behind `decision.level == .critical` — so the widening changed no production behavior while breaking the pre-existing guard test `testExtremeEntryDoesNotDuplicateCriticalReportingOrRemediation`. `.extreme` is not silent: the call site handles it before the critical block with a fatal Sentry capture and an auto-restart, then returns. Duplicating a critical report and remediation on top of that is what the existing test deliberately pins against. Reverts the guard to main's shape and removes ResourceMonitorTests.swift, whose whole premise was the reverted change (its second case is already covered by MemoryPressureEpisodeTrackerTests). Verification: `xcrun swift test --package-path Desktop --filter MemoryPressureEpisodeTrackerTests/` — 6/6 pass (was 6 tests / 2 failures in CI run 30139413246).
…ease-built
`DesktopDiagnosticsManager.resetForTests()` was declared inside `#if DEBUG`, so
`swift test -c release` failed to compile the test bundle:
error: value of type 'DesktopDiagnosticsManager' has no member 'resetForTests'
(MemoryExportStatusTests.swift:10, SBOnboardingCloudContextStateTests.swift:15)
Both call sites landed on main in b61792d and this PR does not touch them. The
release step (`run-swift-ci.sh --release-notification-regression`) is path-gated on
the four DESKTOP_NOTIFICATION_REGRESSION_INPUTS in scripts/pre_push_ci_prediction.py,
none of which b61792d changed — so the break landed latent on main. This PR
touches one of them (Providers/ChatToolExecutor.swift), which arms the step.
Fixes it at the single owner instead of at each caller: the signature now links in
every configuration while the body stays DEBUG-only, so release still carries no
test seam into diagnostics state. The 11 suites that wrap themselves in `#if DEBUG`
are unaffected — each covers 2-74 other test seams, so none becomes redundant.
Kept net-zero on line count: DesktopDiagnosticsManager.swift sits at 1499 lines,
one under the product-file ratchet's 1500 pin, so the rationale is a trailing
comment rather than a doc block that would have forced a baseline raise. Leaving
both test files untouched also keeps their 22 grandfathered SwiftLint baseline
entries on their exact (line, character) keys.
Verification:
- `xcrun swift test -c release --package-path Desktop --filter UserNotificationCallbackBridgeTests/` — build complete, 3/3 (was a compile error)
- `xcrun swift test --package-path Desktop --filter MemoryExportStatusTests/` — 19/19
- `xcrun swift test --package-path Desktop --filter SBOnboardingCloudContextStateTests/` — 6/6
- `xcrun swift test --package-path Desktop --filter DesktopDiagnosticsManagerTests/` — 31/31
- `swift-format lint --strict` clean; file still 1499 lines
Git-on-my-level
added
security-review
Git-on-my-level
dismissed
their stale review
July 26, 2026 09:39
Superseded on head 32e1452: the CI/Hygiene blockers from this review are resolved; current automation feedback is comment-only and preserves human/security/dependency review.
Git-on-my-level
left a comment
Git-on-my-level
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for the follow-up work here. The previous automation blockers I called out on 851e7c20 look resolved on this head: Hygiene is green, the PR body now cites the memory invariant, and the desktop/backend/agent-runtime checks are passing.
I’m not formally approving because this is still a large agent-control/user-data change and the PR remains in the human/dependency review lane. From this pass, the implementation direction looks promising and much better covered: the cloud-agent streaming/session tests are green locally (npm test in desktop/macos/agent-cloud: 61/61), CI is green, and the new tool-policy path narrows child-agent tools rather than widening them.
Maintainer-impact / agent-behavior note: this PR changes how Omi’s AI agents are instructed and constrained. It adds cloud-agent research guidance, researcher-subagent routing for row-heavy local DB work, batch SQL/date-range data tools, streaming/error/session behavior, and spawn-time toolPolicy.allowedToolNames enforcement. That can change how AI coding/review/runtime agents explore user data and delegate work in this repo; the guidance looks directionally safe from this pass because the enforceable parts are outside compactable model context and are fail-closed/narrow-only, but it still warrants human maintainer sign-off.
One privacy/security item for the human review checklist: the new integration registry maps calendar, gmail/email, and contacts to one google_calendar OAuth provider and derives a single scope bundle from all provider capabilities. That means a one-click connect flow for one Google capability may request Calendar + Contacts + Gmail read scopes together. That may be intentional product direction for a unified Google connection, but it is a user-data consent expansion that should be explicitly accepted before merge.
I’m removing the stale needs-tests label and adding security-review for that OAuth/user-data surface while keeping needs-maintainer-review and dependency-review.
by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.
…ements Seven conflicts, all semantic — #10599/#10600/#10601 landed on main while this branch was in review, and they touch the same files. - agent-proxy/main.py: keeps this branch's restart circuit-breaker (the restart_failed/restart_succeeded fields) AND main's usable-IP guards. Both are needed: the breaker stops hammering GCE, the guard stops persisting a placeholder address as ready. - routers/agent_tools.py: takes main's executor offload and bounded error logging, and keeps this branch's record_fallback on the degraded-toolset path — AGENTS.md requires it on a fail-open branch and main has no equivalent. - routers/integrations.py: the registry supersedes the inline OAUTH_CONFIGS / AUTH_PROVIDERS tables (neither is referenced elsewhere in the file), but GMAIL_READONLY_SCOPE, GOOGLE_INTEGRATION_KEY and google_integration_has_scope each still have a live caller, so their import is kept. - agent-cloud/agent.mjs: main fixed the /sync row-schema data loss (#10602) by grouping rows by column set; this branch added a SQL-identifier guard on the interpolated column names. Kept both — the guard now runs against the union of every row's columns, which is a superset of the old rows[0] check. - utils/llm/app_generator.py: took main's deletion of _generate_app_icon_via_openai. It has no callers and main removed it deliberately in "enforce gateway-only LLM routing" (bbbdf3b). - chat-hermetic.yaml: union of both covers lists. - desktop-swift-providers.json: line-count baseline, regenerated below.
Collaborator
Author
|
Conflicts resolved against latest main ( On the OAuth item: the registry derives exactly the scope set main already ships — |
…nition Merging main left two definitions of the Google OAuth scope bundle: the integrations registry derives what the consent screen asks for, while google_utils carried a hand-maintained tuple that google_integration_has_scope checks stored grants against. They happen to be equal today, so nothing fails — but nothing keeps them equal either, and drifting in either direction is a user-visible bug: a scope requested but absent from the gate is never enforced, and one required by the gate but never requested reads as ungranted on every check, looping the user through reconnect forever. Derive GOOGLE_OAUTH_SCOPES from the registry and alias GMAIL_READONLY_SCOPE to the registry's constant, so there is one definition and the duplicate cannot drift. The dependency points from the runtime client to the import-light config module, not the reverse; the registry keeps its typing-only imports. After the merge, google_utils' tuple had no production caller left — the router now builds the authorization query through the registry — so this deletes dead state rather than adding indirection. Verified: backend/.venv pytest tests/unit/test_gmail_scope_gating.py tests/unit/test_gmail_scope_contract.py → 12 passed.
…surface test_connecting_google_requests_the_gmail_scope asserted against routers.integrations.AUTH_PROVIDERS, the inline OAuth table main used to build the consent request. The merge resolved that surface in favour of the integration registry, so the symbol no longer exists and the test failed with AttributeError — a merge casualty, not a behavior regression. Assert against oauth_authorization_query instead, which is what now builds the URL the user actually sees. The test keeps its original intent: a scope the runtime gates on must be one the user was asked to grant. Verified: backend/.venv pytest tests/unit/test_gmail_scope_gating.py tests/unit/test_gmail_scope_contract.py → 12 passed (was 1 failed / 11 passed).
Collaborator
|
#10686 moves the cloud-agent runtime to the Python Claude Agent SDK while preserving the VM protocol, mixed-schema sync, database context, and Playwright MCP. Please resolve the deleted agent-cloud files before merge. |
Git-on-my-level
requested changes
Jul 27, 2026
Git-on-my-level
left a comment
Git-on-my-level
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for keeping this branch updated. I'm requesting changes for the current head because GitHub now reports the PR as conflicting/dirty, and the new maintainer note points to the concrete conflict source: #10686 moves the cloud-agent runtime toward the Python Claude Agent SDK and deletes/replaces parts of the desktop/macos/agent-cloud surface that this PR still changes. Until those file-level conflicts are resolved against the current runtime direction, this is not reviewable/mergeable.
Maintainer-impact / agent-behavior note: this PR still changes AI-agent behavior and instructions across the cloud-agent runtime, researcher-subagent routing, tool-policy narrowing, streaming/session handling, and the added desktop/macos/agent-cloud/experiments/RESEARCH-FINDINGS.md guidance. Those changes may alter how Omi agents explore user data, delegate work, stream progress, and constrain child-agent tools. The direction may still be useful, but after the #10686 runtime migration the agent-behavior guidance and implementation need to be reconciled with the new Python runtime before a human maintainer can sign off.
Please rebase/merge current main after the runtime migration decision is settled and resolve the deleted/replaced agent-cloud files so the remaining diff reflects the intended production runtime. I'm keeping the existing human/dependency/security review labels; no formal approval is possible for this large agent-control/user-data PR.
by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.
Four conflicts, all resolved as unions after checking the merged tree rather than picking a side: - product_file_line_count_ratchet_baseline/desktop-swift-chat.json — neither side's count was right for the merged file; set AgentBridge.swift to its actual 2204 lines. Verified with check_product_file_line_count_ratchet.py. - product_file_line_count_ratchet_baseline/desktop-swift-floatingcontrolbar.json — kept this branch's justification, which is a superset: the merged RealtimeHubSession.swift does carry the URLSession teardown it describes (finishTasksAndInvalidate, main.py:312). - e2e/flows/chat-hermetic.yaml — both sides appended to `covers:`; kept both. check-e2e-flow-coverage.py --strict passes with main's ChatOmiMark/ TypingIndicator and this branch's entries all COVERED. - agent-proxy/main.py — each side added a different firestore import (Increment here, DELETE_FIELD on main; both are used) and each added different error handling to the VM restart path. Kept main's VmNotFoundError branch that clears a reaped record, ahead of this branch's generic handler that sets the restart_failed circuit-breaker flag, so the specific case is caught first and neither behavior is lost. main's test_agent_vm_reaped_record_recovery loads agent-proxy/main.py by file path with only BACKEND_DIR importable. That file now imports this branch's new sibling module (`from resilience import ...`), so the test errored on collection after the merge. Put AGENT_PROXY_DIR on sys.path too — folded in here rather than left to a follow-up so this commit is not red on its own. Verified: backend agent-proxy suites 31 passed (was 25 passed / 4 errors); check-e2e-flow-coverage.py --strict exit 0; ratchet checker OK on both files.
undivisible
added a commit
that referenced
this pull request
Jul 27, 2026
Bring #10686 onto current main and account for features that landed after the branch diverged: - Keep the Python Cloud Run / agent_vm cutover; retire Node agent-cloud and the agent-cloud-tests preflight entry that #10256 reintroduced. - Preserve SCA-194 DESKTOP_* production secret bindings, Pinecone removal, image-lineage verification, and Firestore probe signer mounts from main. - Point desktop-backend-release-boundary triggers at backend/Dockerfile.desktop_backend and desktop_chat.py while retaining verify_desktop_backend_image_lineage.py. - Align the production secret-bridge ordered step with the Python image build step name so release-policy checks stay green after the cutover. - Drop the stale agent-cloud path comment from agent-proxy resilience helpers. Failure-Class: none
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent reliability & performance overhaul
One PR, four measured iterations on the agent stack (cloud VM lane + desktop kernel + desktop error UX). Every change traces to an instrumented experiment, a telemetry corpus, or a live reproduction — the evidence loop ran ~90 live agent runs and session simulations against a real 27MB database copy, plus a 30-day PostHog
chat_agent_errordeep dive.Headline results (same 8-turn conversation, real data, before → after)
interrupted: truepartialWhat's in here, by area
Subagent architecture (desktop kernel + cloud)
researchersubagent (SDKagentsoption): row-heavy data exploration runs in a disposable child context and returns distilled prose findings — Haiku-pinned (measured fastest, ~3× cheaper, accuracy parity) and routed by row-volume. A typed handoff remains parked because this SDK stream exposes no production interception point.toolPolicy.allowedToolNamesenforced at the capability broker (narrow-only, fail-closed, no schema changes).Streaming & session reliability (cloud lane)
includePartialMessageswas never enabled — all streaming code was dead; every turn had 17–46s of total client silence (the stall→Stop churn mechanism). Now: real deltas, subagent progress assubagent:*tool_activity, bounded ephemeral researcher snippets in status events (never the answer/transcript), and a 10s turn heartbeat capping any remaining gap.session_statehello); reconnect amnesia reproduced live before the fix ("I don't have any earlier conversation context") and re-verified fixed.interrupted: truepartial results instead of errors; pending tools terminalize asinterruptedrather than falsecompleted; a prewarm race that made Stop a silent no-op on fast first queries is fixed.Tool system (both sides of the VM↔backend boundary)
get_daily_recapgains explicit date ranges (it could only anchor to "now" — root cause of 13+ SQL-turn fallbacks), authoritative empty-range statements, hourly timeline/top windows; newget_app_usageday×app matrix; batchexecute_sql; task near-duplicate guidance (the store carries ~63% duplicate extractions).stmt.readonlyreplaces a prefix guard that rejected CTE reads and false-positived on user data containing words like "create"; row cap moved to iteration.resp.okchecked; PydanticOptionalaccepts explicitnull; results bounded at 10K chars;agent_tools.pygets per-tool schema isolation +record_fallbackon degraded toolsets.Two-plane error handling & resilience
exc_info=True, breaker reads off the event loop.maxTurns: 10per user turn + tool-result truncation (SDK compaction is API-beta only in 0.2.39 — tracked for 0.2.40+).Desktop error UX (from the PostHog deep dive)
raw_error; the three "unexplained" error spikes were retry storms caused by the copy itself (users with exhausted Anthropic credits obeying "please try again").AgentErrorClassifier: 12 bounded codes with honest copy and retryable verdicts — "try again" only when a retry can help;chat_agent_errortelemetry gainserror_code/retryable/failure_code/failure_source/adapter_id/provider(closed vocabularies, never raw exception text — test-pinned).Product invariants
desktop/macos/agent/src/runtime/**. Lifecycle affected by the cascade; guard tests updated (cancellation.test.ts+3 behavioral cases,run-tool-capability.test.ts+3,control-tools.test.tsschema cases, newspawn-tool-policy.test.ts). toolPolicy strengthens leaf-worker restrictions (narrow-only, kernel-internal checks).ChatProvider.swifttelemetry call site, regeneratedSources/Generated/*); no chat write-path, projection, or timeline behavior changed.web/app/src/components/memories/MemoriesPage.tsx(anduseMemories.ts), which this PR touches as Prettier reformatting with zero logic change. No tier vocabulary, canonical collection, or default Short-term+Long-term access policy is altered; Archive remains excluded from default access.Failure-Class: none
Deliberate non-changes (on record)
bypassPermissionsstays — the per-user VM is a single-tenant sandbox; SDK-level tool gating is intentionally off (maintainer decision). No Sentry (not in the backend stack); no chat-save outbox (the write target is the failure domain); no cross-language error library (7-string contract, ~15 lines per side); prompt-level turn-discipline and batching rejected by experiment (model-dependent).Integration order
Merge #10600, #10601, and #10599 first, then merge current
maininto this branch before clearing the final review. Those focused PRs repair the VM-IP recovery boundary, the backend event-loop/error plane, and end-to-end tool outcome transport that this larger branch also touches.Verification
agent-cloud-testsin the checks manifest (first CI coverage for the package).AgentErrorClassifierTests+ChatQueryTelemetryTests34/34; backend units 12/12 (breaker transitions, classification, per-tool isolation).