chore(core): slim the core — remove dead modules, move desktop features to Tauri, cut goals/todos to tinyagents - #5125
Merged
Conversation
…_companion + dead webview modules
Phase 1 of the module-slimming effort. Removes six modules from the Rust
core and their full UI surface to make the app lighter.
Deleted outright (core + UI + docs):
- codegraph — agent code-nav tools (codegraph_search/index); rewired
the code_executor agent prompt to grep/glob navigation.
- council_registry — saved-council RPC surface.
- model_council — multi-model deliberation RPC + graph; removed its
json_rpc_e2e tests and the orphaned council-juror
Agent builder (from_config_for_read_only_council_juror).
- desktop_companion — Clicky-style interaction loop; removed its controller,
the socketio companion:state_changed bridge, the
DomainEvent Companion* variants, and about_app catalog.
Dead/orphaned webview modules (Tauri already owns the real implementations):
- webview_apis — Gmail webview bridge, retired; dead in core.
- webview_accounts — orphaned; Tauri shell owns the live version.
- webview_notifications — empty stubs; the title-prefix contract already lives
shell-side.
Follow-on cleanups:
- meet_agent::wav is now gated under `meet` — desktop_companion was its only
always-on consumer, so the load-bearing carve-out is gone. Updated AGENTS.md.
- Removed dead scripts/mock-webview-bridge.mjs (+ test) and inventory entries.
- UI: deleted council/model-council API clients + ModelCouncilTab, the
companion Redux slice/panel/socket listener/nav, and all modelCouncil.* +
desktopCompanion.* i18n keys across 14 locales (parity preserved).
Builds green: `cargo check --all-targets` (default) and
`cargo check --lib --no-default-features` (slim); `pnpm typecheck`, i18n checks.
Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
…at_scanner)
Phase 1 over-deleted core `webview_accounts`. The mapping missed that the
Tauri shell's `wechat_scanner` imports `openhuman_core::openhuman::
webview_accounts::{validate_scan, list_ingest_envelope, WechatScanPayload,
…}` — and Phase 1 never ran the shell `cargo check`, so the break slipped
through. Restore only the `wechat_ingest` half (the WeChat scan → memory-doc
ingest normalization the scanner needs); the genuinely-dead cookie-login
heuristic (`ops::detect_webview_logins`, zero callers) stays removed.
Both `cargo check --lib` (core) and `cargo check --manifest-path
app/src-tauri/Cargo.toml` (shell) now pass.
Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
Phase 2a. Completes the companion migration: Phase 1 removed it from core; this adds it to the Tauri shell (where the hotkey/mic/screen/overlay already live), so companion is a shell-owned desktop feature that calls back into core only for the services core owns. Shell (`app/src-tauri/src/companion/`, new): idle→listening→thinking→speaking →pointing state machine, one-turn pipeline, `[POINT:x,y:label:screenN]` parsing, and native cpal mic capture (mono i16 → 16kHz + shell-side WAV RIFF packer, so no dependency on the now-gated meet_agent::wav). Five Tauri commands (start/stop/status/config get/set) replace the old core companion.* controllers; the shell emits a `companion://state_changed` event. - STT/TTS over core RPC (`voice_transcribe_bytes`, `voice_tts`); the LLM turn runs shell-side against the backend, fetching creds via `auth_get_session _token` + `config_resolve_api_url` (no credential is embedded in the shell). - `core_rpc.rs`: factored `post_json_rpc` out of `relay_http_rpc` + added `call_core_rpc`/`unwrap_rpc_outcome`. - No dependency on core accessibility/text_input/autocomplete (being removed). UI: restored the companion surface Phase 1 had removed (companionSlice, CompanionPanel, nav, settings routes) and rewired it from core socket.io to the Tauri event via new `companionEvents.ts`; OverlayApp listens to the same event directly. Restored the 22 companion i18n keys across all 14 locales. Validated: shell + core `cargo check`, `pnpm typecheck`, i18n checks, and 67 companion frontend tests — all green. Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
…t_input, autocomplete, computer-use tools) Phase 1c. Strips desktop-control surfaces from the core now that companion lives in the shell and native desktop control is being retired. accessibility (Option A — slimmed, NOT deleted): screen_intelligence + voice still depend on its permissions / screen-capture / foreground-context / globe-hotkey / focus / terminal backbone, so that is KEPT. Removed only the desktop-control half: automate, ax_interact, uia_interact, overlay, vision_click, element_match, paste, keys, app_fastpaths (+ their re-exports). Removed wholesale: - text_input/ (OS focused-field read/insert) + its controller + CLI arm. - autocomplete/ (inline typeahead engine) + controller + CLI adapter + the runtime.autocomplete app-state payload + config migration + about_app entry. - computer-use agent tools: ax_interact, automate, launch_app, mouse, keyboard, human_path, main_thread (tools/impl/computer + system/launch_app). - computer_control config (ComputerControlConfig + the ax-auto-approve block). - Excised voice/always_on.rs's automate/launch_platform intents (Play/OpenApp now defer to the agent fallback) so voice compiles without the AX backend. - Dropped the `uiautomation` dep + the Win32_System_Com windows-sys feature. - Shell: removed the MainThreadInputOp native handler that served the synthetic-input tools. UI: deleted the autocomplete + desktop-agent surfaces (panels, setup modal, tauriCommands, routes/nav/store, ghost-completion composer path) and their i18n groups (107 keys/locale across 15 files, parity preserved); kept the screen_intelligence permission UI. Validated green (all independently re-run): core `cargo check --lib` + `--all-targets` + slim `--no-default-features`, shell `cargo check` (app/src-tauri), `pnpm typecheck`, i18n checks, frontend tests. Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
… native bridge) Phase 2b. The WhatsApp SQLite store + ingest + query logic moves to the shell (where the scanner already lives); the agent keeps its 3 query tools via the in-process event-bus native request/response — no WebSocket loopback, since the core runs in-process inside the shell. Core (`src/openhuman/whatsapp_data/`) — slimmed to the shared contract: - Kept: types.rs (DTOs, single source of truth for both crates), the 3 agent tools, and a `methods` module of native-request key constants. - Tools dispatch `request_native_global(whatsapp_data.list_chats|list_messages| search_messages, req)`; when no shell handler is registered (headless CLI/ docker/slim) they degrade gracefully to an empty result with a "WhatsApp data unavailable (desktop only)" note — never an error. - Deleted store.rs/sqlite_retry.rs/global.rs/ops.rs/rpc.rs/schemas.rs(+tests). Removed the RPC controllers + internal ingest controller from all.rs, the whatsapp_data boot-plan field/init in runtime/context.rs, and the `[whatsapp_data] ingest failed` error classifiers in observability.rs. Shell (`app/src-tauri/src/whatsapp_data/`, new): ported SQLite store + ops (90-day prune, upsert dedup), `register_native_handlers()` wired in lib.rs setup (next to companion::setup), and Tauri commands whatsapp_data_list_chats/ _list_messages/_search_messages. Promoted rusqlite to an all-platform dep. Scanner: post_whatsapp_data_ingest dispatches the native whatsapp_data.ingest in-process instead of POSTing core RPC. Frontend memory.ts uses invoke(...) of the new Tauri commands. Kept the whatsapp_data agent-write denylist entry (DB still under the core workspace_dir). Reconstructed in-worktree from the validated implementation (an earlier agent run had validated it green but written to the wrong checkout); module files copied verbatim, wiring re-applied against this branch's post-Phase-1c files. Re-validation to follow. Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
Reconciles 17 upstream commits with the slimming work. Key divergence: upstream tinyhumansai#5049/tinyhumansai#4019 feature-GATED the desktop-automation cluster (accessibility, screen_intelligence, autocomplete, desktop_companion, computer tools) behind a default-ON `desktop-automation` feature; this branch had DELETED that cluster (companion migrated to the Tauri shell). Resolution keeps the deletions and removes upstream's `desktop-automation` feature entirely: - Kept deleted: autocomplete, text_input, the computer-use tools, and the accessibility automation half. Restored this branch's clean ungated accessibility + screen_intelligence (always-on Platform, not gated); dropped upstream's stub.rs files and the re-added `uiautomation` dep. - Removed `DomainGroup::DesktopAutomation` + the `desktop_automation` DomainSet axis + the `desktop-automation` Cargo feature and all its `#[cfg]` gates and paired tests (all_tests/ops_tests/jsonrpc_tests/legacy_aliases). - Kept upstream's unrelated improvements, incl. tinyhumansai#5050's shared `Arc<Config>` in the session builder (adapted: dropped the ax-interact `resolve_tool_config` toggle since computer_control is gone), the redirect_links removal (tinyhumansai#5052), flows/api/keyring fixes, and v0.63.0. - Frontend: dropped upstream's inline-completion additions (autocomplete UI was removed in Phase 1c); cleaned orphaned Conversations.tsx remnants. Green: core `cargo check --lib`, `pnpm typecheck`. all-targets/shell/slim/i18n verification to follow. Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
The upstream merge removed the core `desktop-automation` feature, but app/src-tauri/Cargo.toml still forwarded it to `openhuman_core`, breaking shell dependency resolution (`openhuman does not have that feature`). Shell `cargo check` now resolves and compiles green. Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
…store
The vendored tinyagents `graph::goals` KV store is now AUTHORITATIVE for thread
goals; the legacy `{workspace}/thread_goals/` file-JSON store is retired. Every
consumer (RPC ops, the harness turn loop, the heartbeat continuation runtime,
the agent tools, post-turn accounting) is untouched — `thread_goals::store`
keeps its exact `store::*(workspace_dir, …)` signatures and now forwards each
operation to the crate store, converting the crate ThreadGoal/error back to the
local types.
- store.rs: file-JSON persistence deleted; thin crate-backed adapter (delegates
to tinyagents::graph::goals::store, converts via crate_adapter). The one op
without a crate equivalent, the unconditional set_continuation_suppressed, is
replicated via the adapter's raw mirror read/write.
- crate_adapter.rs: dropped the shadow/dual-write scaffold (shadow_mirror_*,
the OPENHUMAN_THREAD_GOALS_CRATE_SHADOW flag) now that the crate is primary;
kept the conversion + mirror helpers; the migration now reads the legacy
file-JSON tree directly (a self-contained reader) rather than via the
now-crate-backed store.
- services.rs: wired migrate_legacy_goals_into_crate_store into
start_boot_once_jobs behind a Once so existing legacy goals are copied into
the crate store once at boot (idempotent, fast on an empty/absent dir).
- Rewrote the crate_adapter migration tests to seed the legacy file-JSON tree
directly; dropped the shadow-flag test. Docs updated (mod.rs, crate_adapter).
Core `cargo check --lib` green. Test-compile + gates to follow.
Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
…ate store
The vendored tinyagents `graph::todos` KV store is now AUTHORITATIVE for the
per-thread task board; the legacy `{workspace}/agent_task_boards/*.json`
file-JSON store is retired. Unlike thread_goals (already async), the task-board
store + `todos::ops` were synchronous while the crate store is async, so this
does the full sync→async conversion.
- agent/task_board.rs: `TaskBoardStore::{get,put,delete}` + `board_for_thread`
made async, re-backed onto the crate via the new adapter — get reads raw
(preserves None-vs-empty), put normalises locally (keeps `task-<uuid>` ids)
then delegates to crate `replace` (enforces single-InProgress), delete does
an existence pre-check + raw key delete so a later get is `None`.
- todos/graph_shadow.rs → renamed crate_adapter.rs: repointed the store handle
from the shadow dir to the shared `tinyagents_store/kv` tree (ns graph.todos),
kept the conversion helpers + added from_crate_card/board round-trips, dropped
the shadow mirror/claim scaffold, and added the boot migration.
- todos/ops.rs + ~25 caller sites across task_dispatcher, task_sources, triage,
agent/tools, threads, runs, skills: cascaded to async/.await (no block_on).
The scratch + claim locks moved parking_lot → tokio async mutex. Scratch
(`BoardLocation::Scratch`) stays on the local in-memory path — only Thread
boards route to the crate. `emit_progress`/`AgentProgress::TaskBoardUpdated`,
markdown rendering, and `runs.rs` (its own *.runs.json ledger) stay local.
- services.rs: `spawn_task_boards_migration` wired behind a Once beside the
thread-goals one, copying legacy boards (skipping *.runs.json) into graph.todos.
- Rewrote the sync tests async (incl. concurrent_claims_only_one_wins).
Validated: core `cargo check --lib` clean, `cargo test --lib` todos+task_board
74 passed, slim `--no-default-features` build OK.
Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
Working scratch used during the slimming effort; not product code. Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
… + apply hook auto-fixes
`cargo check --all-targets` (integration tests, which `cargo check --lib` skips)
caught 2 raw-coverage e2e tests calling the now-async `TaskBoardStore::{get,put,
delete}`/`board_for_thread` without `.await` (Phase 3b made them async). Added
`.await`; converted the sync `#[test]` inference-agent task-board test to
`#[tokio::test] async`. Also includes the pre-push hook's cargo-fmt/prettier/
eslint auto-fixes across the branch.
Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
…age e2e tests agent_triage_dispatch_round23: made the add_card/card_status helpers async + awaited their callers; awaited ops::update_status. (Completes the async cascade into the integration tests that cargo check --lib didn't compile.) Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
…ings) The shell-side companion never wired is_active into its session logic (only a test used it), so `cargo clippy --manifest-path app/src-tauri/Cargo.toml -- -D warnings` flagged it dead. Removed the method + its test. Claude-Session: https://claude.ai/code/session_01Km3VTggbYDgByXmmExYW5i
Contributor
|
Important Review skippedToo many files! This PR contains 229 files, which is 79 over the limit of 150. To get a review, narrow the scope: Upgrade to Pro+ to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (352)
You can disable this status message by setting the Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dddcb46eba
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c9c31f497c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f3b2aff898
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 48dd7a6a4b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 78c2178f8d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 597034f4d0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+279
to
+283
| let result = crate::core_rpc::call_core_rpc("openhuman.voice_tts_dispatch", params).await?; | ||
| debug!( | ||
| "{LOG_PREFIX} tts audio_mime={:?}", | ||
| result.get("audio_mime").and_then(|mime| mime.as_str()) | ||
| ); |
There was a problem hiding this comment.
When a companion turn produces a non-empty reply, this call returns the voice_tts_dispatch audio payload, but the new Tauri companion path does not consume that audio: companion/audio.rs only captures microphone input, and this function only logs the MIME type before returning success. The session then reports tts_synthesized=true and transitions through Speaking/Idle, so users who start the desktop companion get no spoken response even though synthesis succeeded; decode/play or forward the returned audio before treating TTS as complete.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b67ca0bf3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -161,10 +155,6 @@ pub mod webhooks; | |||
| #[cfg(feature = "channels")] | |||
| pub mod webview_accounts; | |||
| #[cfg(feature = "channels")] | |||
There was a problem hiding this comment.
With the core webview_apis module/controllers removed here, the desktop shell still runs app/src-tauri/src/lib.rs:3219-3242 to start the old WebSocket bridge and aborts setup when that loopback listener fails to start. Since a repo-wide search leaves only the shell-side bridge and no remaining core/app caller for webview_apis RPCs, a transient bridge bind/start failure now prevents the shipped desktop app from launching for a removed capability; remove this startup dependency or make it non-fatal.
AGENTS.md reference: AGENTS.md:L24-L25
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Slims the Rust core and relocates desktop-coupled features to the Tauri shell (~34.6k net lines removed):
codegraph,council_registry,model_council, and the orphaned webview modules (webview_apis,webview_accountscookie-login half,webview_notifications).desktop_companion→ the Tauri shell — nativecpalmic capture, STT/TTS over core RPC, LLM shell-side; UI rewired to acompanion://state_changedTauri event.whatsapp_datastore → the Tauri shell — SQLite store + ingest move shell-side; the agent's 3 query tools keep working via the in-process event-bus native request/response bridge (the core runs in-process in the shell).accessibilityautomation half (kept the capture/permissions/globe backbone thatvoice+screen_intelligenceneed),text_input,autocomplete, and the computer-use agent tools (ax_interact/automate/mouse/keyboard/launch_app) +computer_controlconfig.thread_goalsandtodos/task-board persistence over to the vendoredtinyagentscrate (graph::goals/graph::todos), which is now authoritative; the legacy file-JSON stores are retired with idempotent per-workspace boot migrations.upstream/main(through v0.63.0).Problem
The core had accumulated dead modules, desktop-only features that belong in the Tauri layer, and two goal/task-board stores kept in a half-finished "shadow" migration to the vendored
tinyagentscrate. This bloats the core binary/compile surface and keeps two sources of truth for goals/todos.Solution
src/openhuman/mod.rs, its controller registration insrc/core/all.rs, agent-tool registrations, and event-bus variants; verified no dangling references.register_native_global/request_native_global(zero-serialization, in-process) — no WebSocket loopback. Both degrade gracefully headless.voice/always_on's AX-automation path was excised so voice still compiles.thread_goalswas already async;todos/task_boardrequired a sync→async conversion cascade across ~25 call sites (dispatcher/triage/tools/threads/task_sources), with the in-memory Scratch board staying local. On each core boot, idempotent migrations copy legacy file-JSON goals/boards for the active workspace into the crate store while preserving existing crate values.#5049feature-gated the same desktop-automation cluster this branch deleted; the merge keeps the deletions and fully excises upstream'sdesktop-automationfeature +DomainGroup::DesktopAutomation, while retaining upstream's unrelated improvements (the#5050Arc<Config>refactor,redirect_linksremoval, flows/api/keyring fixes).Submission Checklist
concurrent_claims_only_one_wins); UI tests updated for the relocated companion/whatsapp surfaces.## Related— N/A: no tracking issue.uiautomationdropped); the tinyagents crate is already vendored.Impact
thread_goals/andagent_task_boards/file-JSON into the crate KV store; no data loss.task-<uuid>.uiautomationdep dropped.Related
docs/TEST-COVERAGE-MATRIX.md+RELEASE-MANUAL-SMOKE.mdreconciliation if CI flags; optional dedup of the now-duplicate markdown/normalise helpers betweentodosand the crate.AI Authored PR Metadata
Linear Issue
Commit & Branch
chore/slim-core-modulesValidation Run
pnpm typecheck— passedpnpm i18n:check/i18n:english:check— passedcargo check --lib(default +--no-default-features) green;cargo test --libfor the migrated stores green (thread_goals 26, todos/task_board 74)cargo check --manifest-path app/src-tauri/Cargo.tomlgreenBehavior Changes
Parity Contract
Duplicate / Superseded PR Handling