Skip to content

feat: multi-agent orchestration, interactive UX, and memory infra - #15

Merged
cuttlefisch merged 31 commits into
mainfrom
feature/gemini-provider
Apr 22, 2026
Merged

feat: multi-agent orchestration, interactive UX, and memory infra#15
cuttlefisch merged 31 commits into
mainfrom
feature/gemini-provider

Conversation

@cuttlefisch

Copy link
Copy Markdown
Owner

Summary

Implemented a comprehensive multi-agent suite and interactive pair programming infrastructure. This transforms MAE's AI peer into a proactive 'Senior Developer' capable of structured reasoning, specialized delegation, and safe change management.

Multi-Agent Orchestration

  • delegate Tool: Spawn specialized sub-agents (Explorer, Planner, Reviewer) with isolated contexts.
  • Async Proxy Loop: Background sub-sessions run concurrently with dedicated conversation buffers (e.g., *AI-Explorer-123*).
  • Targeted Routing: AI output can now be directed to specific buffers via target_buffer metadata in AiEvent.

Interactive UX & Guardrails

  • ask_user Tool: Structured interviews and clarifying questions with natural chat-based replies.
  • propose_changes Tool: Mandatory approval for potentially destructive edits via a dedicated *AI-Diff* buffer.
  • AI Operating Modes: standard (manual approval), plan (drafting only), and auto-accept (hands-free execution for small tasks).
  • User Controls: Added :ai-accept, :ai-reject, and fuzzy pickers for modes/profiles (SPC a m, SPC a P).

Memory & Planning Infrastructure

  • Hidden State Storage: Persist architectural plans and facts in .mae/memory/ and .mae/plans/.
  • Context Injection: Automatic loading of active plans and long-term memory into the system prompt for session continuity.
  • Planning Tools: New create_plan and update_plan tools for documented multi-step execution.

Infrastructure & Accuracy

  • Structural Prompting: Shifted to XML-based prompts with a mandatory <reasoning> block for all tool calls.
  • Cache-Aware Pricing: Updated budget tracker to discount cached tokens for DeepSeek/Anthropic (fixes cost overestimation).
  • Prompt Library: Prioritized dynamic loading (Project > User > Bundled) with live re-evaluation (no editor restart needed).
  • Terminal Integration: AI can now drive visible interactive PTY sessions via terminal_spawn/send/read.

Test plan

  • make ci passes (1,100+ tests)
  • Verify mode switching via SPC a m (Normal -> Plan -> Auto)
  • Verify profile switching via SPC a P (Explorer/Planner/Reviewer)
  • Verify change proposal flow: AI opens *AI-Diff*, user approves with :ai-accept
  • Verify delegation: 'delegate to an explorer' spawns a background sub-agent
  • Verify memory: AI findings are saved to .mae/memory/ and appear in future prompts
  • Verify cost tracking: DeepSeek cache hits correctly reflected in status line USD

cuttlefisch and others added 18 commits April 20, 2026 12:37
- crates/ai/src/gemini.rs: implement GeminiProvider for Google's Generative AI API
- crates/ai/src/pricing.rs: add Gemini 3.1, 3.0, and 2.5 model pricing
- crates/mae/src/bootstrap.rs: register GeminiProvider in setup_ai
- crates/mae/src/config.rs: add gemini provider support to config resolution and wizard
- crates/mae/src/agents.rs: add gemini-cli agent with auto-approval bootstrap strategy
- scheme/init.scm: add gemini-cli to ai-editor example
- crates/gui/src/shell_render.rs: fix clippy::unnecessary_map_or lint

Co-Authored-By: Gemini 2.0 Flash <noreply@google.com>
- crates/shell/src/path.rs: implement logic to pull PATH from interactive login shell
- crates/mae/src/main.rs: call sync_path_from_shell at startup
- crates/mae/src/bootstrap.rs: call sync_path_from_shell in setup_ai
- crates/core/src/commands.rs: register debug-path command
- crates/core/src/editor/dispatch.rs: handle debug-path to show current PATH
- crates/mae/src/main.rs: fix needless borrow clippy lint

This ensures that when MAE is launched from a desktop environment, it can
find binaries (like 'gemini', 'claude', or 'rust-analyzer') that are
defined in the user's shell profile (~/.bashrc, ~/.zshrc, etc.).

Co-Authored-By: Gemini 2.0 Flash <noreply@google.com>
Three changes shipped together on feature/gemini-provider:

1. MessageContent::TextWithToolCalls — preserves assistant reasoning text
   between tool-call rounds, fixing "exceeded max tool call rounds" for
   DeepSeek/reasoning models that return text alongside tool calls.

2. AI buffer focus preservation — AI tools (open_file, switch_buffer,
   create_file) now route opened files to a non-conversation window via
   switch_to_buffer_non_conversation() and open_file_non_conversation(),
   auto-splitting if needed. The *AI* buffer stays visible during tool
   calls instead of losing focus.

3. Word wrap duplication fix — the GUI conversation renderer now handles
   all InputPrompt screen lines as a group when in cursor mode, preventing
   duplicated text when long prompts wrap across multiple rows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- crates/mcp/src/shim.rs: rewrite proxy loop using tokio::io::copy for robustness
- crates/mae/src/bootstrap.rs & main.rs: pass rootUri to LSP servers on init
- crates/ai/src/openai.rs: check full response body for context overflow errors
- crates/ai/src/session.rs: dynamically halve context window on overflow (self-healing)
- crates/ai/src/context_limits.rs: lower DeepSeek default limits to 64k
- crates/mae/src/main.rs: automatic per-project AI session restoration/persistence
- crates/mae/src/system_prompt.md: elevate AI identity to 'Senior Peer Engineer'
- crates/mae/src/self_test_prompt.md: update self-test instructions for peer actor protocol
- crates/ai/src/context_limits.rs: add ModelLimits with max_rounds per model
- crates/ai/src/session.rs: implement collapse_transaction() for ephemeral tool history
- crates/ai/src/session.rs: add dynamic context-aware tool loop termination
- crates/ai/src/types.rs: add RoundUpdate event to AiEvent
- crates/core/src/editor/mod.rs: track AI round and transaction state in Editor
- crates/mae/src/ai_event_handler.rs: update Editor from RoundUpdate events
- crates/ai/src/tool_impls/introspect.rs: expose AI metrics in introspection
- crates/ai/src/executor.rs: add tool_callstack tests to self-test suite
- crates/ai/src/session.rs: update unit tests for new event sequence
- crates/scheme/src/runtime.rs: add recent-files-add! and recent-projects-add! scheme primitives
- crates/mae/src/bootstrap.rs: implement history.scm generation and evaluation
- crates/mae/src/main.rs: integrate history load on startup and save on shutdown
- crates/core/src/editor/mod.rs: add ai_target_buffer_idx for AI tool context tracking
- crates/ai/src/tool_impls: prioritize ai_target_buffer_idx in buffer and LSP tools
- crates/ai/src/tools.rs: move introspection tools to Core tier and lower dap_start to Shell tier
- crates/core/src/kb_seed.rs: expand LESSON_AI and CONCEPT_AI_AS_PEER documentation
- crates/core/src/kb_seed.rs: fix keymap table in help index
…spection

- crates/core/src/editor/file_ops.rs: add open_file_hidden() to load buffers without switching focus
- crates/core/src/editor/mod.rs: reimplement non-conversation buffer switching to ensure visibility without stealing human focus
- crates/ai/src/tool_impls/git.rs: implement structured AI tools for git status, diff, log, commit, stage, push, pull
- crates/ai/src/tools.rs: register git tools, move introspection tools to Core tier, and lower dap_start to Shell tier
- crates/ai/src/executor.rs: register git tool executions and update tests for new buffer focus model
- crates/ai/src/tool_impls: update resolve_buffer_idx and cursor_info to prioritize ai_target_buffer_idx
- crates/core/src/buffer.rs: add sync_conversation_rope() to make AI buffer searchable and selectable
- crates/core/src/editor/mod.rs: add focused_window_viewport_height() for accurate scroll management
- crates/mae/src/main.rs: use accurate viewport height in render loops
- crates/gui/src/buffer_render.rs & cursor.rs: implement word wrap rendering and cursor positioning
- crates/ai/src/session.rs: harden aggressive_prune to preserve OpenAI message sequence rules
- crates/scheme/src/runtime.rs: register default values for state-injected variables to fix startup errors
- crates/mae/src/ai_event_handler.rs & key_handling.rs: trigger rope sync on AI activity
- crates/core/src/hooks.rs: add app-start, app-exit, focus-in, focus-out hooks
- crates/core/src/editor/mod.rs: implement set_mode() and sync_conversation_buffer_rope()
- crates/core/src/editor/dispatch.rs: replace mode assignments with set_mode() and fire focus hooks
- crates/mae/src/bootstrap.rs: implement debug_dump() for log/chat tombstoning
- crates/mae/src/main.rs: integrate lifecycle hooks and fix GUI shutdown persistence gaps
- crates/ai/src/tools.rs: register trigger_hook tool for AI-driven hook testing
- crates/ai/src/tool_impls: implement execute_trigger_hook
- crates/mae/src/ai_event_handler.rs & key_handling.rs: trigger rope sync on AI activity
- crates/core/src/git_status.rs: new data model for structured git status
- crates/core/src/editor/git_ops.rs: implement porcelain v2 status and staging actions
- crates/core/src/editor/syntax_ops.rs: implement org-cycle folding and todo cycling
- crates/core/src/editor/keymaps.rs: add default git-status and org keymaps
- crates/core/src/syntax.rs: split org emphasis markers for visibility control
- crates/gui/src/canvas.rs: implement three-stage font fallback (Primary -> Icons -> System)
- crates/gui/src/buffer_render.rs: implement GUI word wrap and folded line skipping
- crates/renderer: add git-status TUI display and folding support
- crates/core/src/kb_seed.rs: add extensive documentation for new Git and Org features
- assets/sample-config.toml: document new font and org options
- crates/ai/src/tools.rs: add org_cycle, org_todo_cycle, and org_open_link tools
- crates/ai/src/executor.rs: register Org tool implementations and import execute_trigger_hook
- crates/core/src/editor: add extensive info! and error! logging to Git and Org operations
- crates/ai/src/tool_impls/introspect.rs: expose folded_ranges and git_status availability in buffer introspection
- crates/ai/src/tool_impls/editor_tools.rs: implement Org-specific tool handlers
- crates/mae/src/ai_event_handler.rs: fix syntax and borrow errors in event loop
- crates/core: Add BufferKind::Visual and VisualBuffer scene-graph
- crates/gui: Implement Skia-based visual buffer rendering and italic support (skew matrix)
- crates/gui: Implement variable-height Org headings (1.5x scale) in buffer_render
- crates/renderer: Implement summary TUI view for visual buffers
- crates/ai: Add AI tools for creating visual rectangles, lines, circles, and text
- crates/scheme: Expose visual buffer drawing primitives to Scheme runtime
- crates/core/src/editor: finalize org_hide_emphasis_markers wiring
@cuttlefisch

Copy link
Copy Markdown
Owner Author

Bug Fixes & Audit

  • Fixed Context Overflow Recovery: Corrected collapse_transaction to only preserve Assistant text responses. This prevents orphaned Tool messages from being pushed to the end of the history, resolving the 400 Bad Request error from OpenAI/DeepSeek.
  • Fixed Loop Termination: Updated the context-full check to return early, preventing the misleading 'max tool call rounds' error from being emitted simultaneously.
  • Codebase Audit: Cleaned up AI reasoning artifacts and 'placeholder' comments from session.rs and ai_event_handler.rs.
  • New Test Case: Added test_trim_preserves_tool_call_pairs to ensure the API schema is always respected during message pruning.

@cuttlefisch

Copy link
Copy Markdown
Owner Author

Workflow & Core Fixes

  • Infinite Tool Loop Protection: Added a circuit breaker in AgentSession that aborts the loop if 3 identical tool calls are requested consecutively.
  • GitHub CLI Integration: Added github_pr_status (includes CI checks) and github_pr_create tools. Updated git_status description to clarify it doesn't provide PR info.
  • Fixed AI Cancellation: Corrected SPC a c (ai-cancel) to actually send AiCommand::Cancel to the background AI task.
  • Fixed Startup Font Size: Corrected the initialization sequence so init.scm font settings are no longer clobbered by the default TOML config on startup.

@cuttlefisch

Copy link
Copy Markdown
Owner Author

CI & Safety

  • Enhanced Pre-commit Hook: Updated .githooks/pre-commit to include cargo clippy (alongside fmt). This will catch lint failures locally before they reach CI.
  • Alpha Disclaimer: Added a [!CAUTION] block to README.md and CLAUDE.md warning users that MAE is in Alpha and AI cost guardrails are experimental. Always monitor your provider dashboards!

cuttlefisch added a commit that referenced this pull request Jul 23, 2026
Adds kb/query.{capabilities,get,search,graph} to the OAuth HTTPS listener
(daemon/src/oauth.rs) so a thin client with no local KB replica can search/
read a hub-hosted KB it has Viewer+ access to, without the daemon ever
weakening the E2E-encryption confidentiality boundary (ADR-037).

Two corrections to ADR-053's literal text, found during implementation
planning and recorded as an ADR addendum (principle #15): the reuse target
is DocStore/kb_content.rs (the collaborative/hub document model), not
handler.rs's KbQueryLayer (which serves a structurally different,
locally-federated data model -- Phase D's own rewrite target); and
daemon_mode=shared cannot be a daemon-side gate at all (it has zero presence
in daemon/src -- a pure editor-side attach-policy concept). The real gate is
a new oauth.kb_query_enabled TOML boolean (default false, independently
toggleable from oauth.enabled), plus collab.enabled (a DocStore must exist
to serve from).

Encryption-aware by construction, checked before any node content is
touched: kb/query.search structurally refuses for an Encryption::E2e KB
(never silently returns empty); kb/query.get returns real content for
unencrypted KBs but only raw op-set ciphertext for E2E ones (the daemon
cannot decrypt it -- not a policy choice); kb/query.graph returns real edges
for unencrypted KBs but node-existence only for E2E ones, since links live
inside the same encrypted node-doc schema as title/body. A new, narrow
collab_handler::check_kb_read_access wrapper (Read-only, no path to
Edit/Manage) is the only new public surface into the access engine.

The client-side lazy-fetch cache ADR-053 describes (only a genuine KB member
holds the decryption key, so caching decrypted content is inherently
client-side) reuses shared/kb/src/cache.rs::NodeCache unmodified, with
principal-prefixed keys so a multi-tenant client process can't cross-serve
one principal's decrypted content to another. mae_sync::op_set::materialize
promoted from a test-only helper to a reusable pub primitive for this.

New adversarial tests (daemon/src/tests/kb_query_tests.rs, in-process, real
DocStore/crypto/principal strings, default CI): the required
hostile-hub-operator test (real sealed op-set ciphertext, structural search
refusal, and a byte-level scan of the serialized wire response proving the
plaintext secret never appears in it -- the first daemon-process-level proof
of this property, extending op_set.rs's existing pure-crypto unit tests);
search-cap enforcement (the scan itself is capped, not just the result
count); the access gate firing before the encryption branch on both KB
types; and the disabled-surface case returning a distinct error from
"method not found". Scoped deliberately below the full HTTP/TLS/JWT
transport layer, which Phase F's own oauth.rs tests already cover -- see
the new test file's module doc for the reasoning.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cuttlefisch added a commit that referenced this pull request Jul 25, 2026
Closes the real gap found while confirming `mae --headless` is a
legitimate standalone "engine" for external editors' AI agents (ADR-055,
already shipped): tool tiering (`mcp_tools_tiered_by_default`) only
restricts what's *advertised* in tools/list, never what's *dispatchable*
-- a connected MCP client that already knows (or discovers via
request_tools/search_tools) a tool name can call any of MAE's ~700+
tools regardless of tiering. A headless instance meant to expose only
KB+guidance operations had no way to actually enforce that intent.

- crates/ai/src/tools/categories.rs: `PermissionPolicy` gains
  `allowed_categories: Option<HashSet<ToolCategory>>` +
  `is_category_allowed()`. Fail-closed for uncategorized tools
  (`execute_command`, `shell_exec`) -- an uncategorized tool is exactly
  the case the taxonomy hasn't judged yet, and this is a trust boundary.
  `request_tools`/`search_tools` stay reachable under any restriction
  (pure discovery, invoke nothing).
- crates/ai/src/executor/tool_dispatch.rs: new step 2b in
  execute_tool_dispatch_body, right after the existing tier check --
  orthogonal axis (tier = how mutating, category = which subsystem),
  both must pass.
- crates/mae/src/ai_event_handler.rs: second, independent category
  check added to the Scheme-command bridge -- found only by writing
  this ADR's own required adversarial test, since that bridge matches
  and dispatches BEFORE ever reaching execute_tool_dispatch_body, so
  the chokepoint check alone did not cover it (recorded as a correction
  in the ADR itself, principle #15). effective_permission_policy now
  also computes the category-set intersection (global ∩ per-session
  declaration), mirroring the existing tier `.min()` composition.
- shared/mcp/{session,lib,shim}.rs: `declared_tool_categories` threaded
  end-to-end -- ClientSession field, `toolCategoryAllowlist` initialize
  param, RequesterContext field, MAE_MCP_TOOL_CATEGORY_ALLOWLIST env var
  in mae-mcp-shim -- identical trust shape and wiring to the existing
  `permissionCeiling`/MAE_MCP_PERMISSION_CEILING (ADR-051): self-declared,
  can only narrow, never widen.
- crates/core/src/options.rs + editor/{mod,option_ops}.rs: new
  `mcp_tool_category_allowlist` option (config-driven, principle #7) for
  an instance-wide restriction, seeded into the global PermissionPolicy
  at boot in crates/mae/src/main.rs.
- docs/adr/056-tool-category-session-scoping.md (new, Accepted) and
  docs/adr/055-headless-service-mode.md (Proposed -> Accepted, a stale
  status left behind after the design was already shipped/dogfooded).

6 new adversarial tests (2 chokepoints x denial, allow, global/session
intersection, Scheme-bridge composition), each independently verified to
genuinely fail when its corresponding check is temporarily neutered.
cargo fmt/clippy --workspace -D warnings/test --workspace clean (zero
failures across the entire editor workspace).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cuttlefisch added a commit that referenced this pull request Jul 25, 2026
The extension now lives at github.com/cuttlefisch/mae-vscode, with an
independent release cadence (compatible-MAE-version noted in its own
README, not a lockstep version number). Extracted via `git filter-repo`
(preserved the 7 real feature commits + this repo's own pre-extraction
hardening commits, not a clean/history-discarding copy) and pushed with
its own CI (3-OS lint/unit/integration matrix -- this extension carries
real Windows-specific spawn/PATH logic that was never exercised on
Windows CI before -- a pinned-floor-version leg, package-validate, and a
scheduled/dispatchable real-binaries job against the latest published
mae release; publishing to the Marketplace + Open VSX is a separate,
human-gated workflow).

This closes the "carrying Node.js/npm tooling in the main Rust monorepo"
cost -- editors/vscode/ was the sole npm footprint in this repo; mae's
own CI (`vscode-extension` job + setup-node) no longer has one.

- Makefile: build-vscode/package-vscode/test-vscode removed;
  install-vscode becomes a pointer-and-exit target (fails loudly with
  the new repo's URL, rather than a raw "no such directory" error --
  verified: `make install-vscode` exits 1 with the message).
- .github/workflows/ci.yml: vscode-extension job removed.
- README.md, docs/EXTERNAL_EDITOR_MCP_PAIRING.md: VS Code pairing
  sections point at the new repo instead of describing it in-place.
- docs/adr/050-external-editor-mcp-pairing.md: new "Implementation
  note" addendum recording the extraction (principle #15 -- don't
  silently rewrite history, record the drift). docs/adr/055's own
  verification-note citation of the (now-moved) real-binary e2e job
  updated to point at the new repo.

Verified: cargo fmt/clippy -D warnings clean; zero remaining
editors/vscode references outside historical ADR context (grepped);
the new repo's own CI is green on the pushed extraction (7 original
commits + this repo's pre-extraction hardening + standalone-repo
adaptations, all verified locally: npm compile/lint/test:unit 25/25,
a real `vsce package --no-dependencies` producing a valid .vsix).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cuttlefisch added a commit that referenced this pull request Jul 26, 2026
…ite needed

Before writing the directory + per-tenant-lock structure the ADR's Decision
text originally described, re-checked the current architecture first (the
same principle-#15 discipline ADR-054's own Implementation Note applied to
its own originally-proposed mechanism). Found ADR-054 had already
generalized "snapshot-then-drop" to every read/hygiene arm in
daemon/src/handler.rs and to scheduler.rs's background maintenance ticks,
and that main.rs's accept_loop already spawns one independent task per
connection with the daemon's shared lock never held across a blocking read.
A new directory + per-tenant lock would have duplicated synchronization
that already exists (principle #8).

What was actually missing was the tests -- ADR-060's own Verification
section explicitly deferred them out of Phase A since they'd be meaningless
before Phase B's mechanism existed. All three of this ADR's named
adversarial cases are now written against the real architecture and pass:

- handler::tests::concurrent_slow_tenant_a_query_does_not_measurably_degrade_b_or_c_reads
  -- a >=3-tenant fixture (principle #14) where tenant A's real slow bulk
  query (5x full-text scan over 500 real, varied-content nodes, no
  artificial sleep) runs concurrently with tenant B's and C's single-node
  reads. Measured: B/C's concurrent latency was statistically identical to
  their solo baseline (e.g. 3.4ms vs 3.6ms in one representative run), not
  the ~150-300ms a genuinely serialized implementation would show.
- daemon/src/tests/kb_socket_malformed_and_disconnect_tests.rs (new file,
  reusing the existing spawn_kb_socket real-socket harness ADR-054 built):
  malformed JSON on one connection, and a client that sends a partial
  message then disconnects mid-request (the named Emacs bug#11639/bug#23499
  reproduction) -- both verified to have zero effect on a separate,
  concurrently-issued, unrelated tenant's request, including with a third,
  currently-stalled peer connection also present.

docs/adr/060-daemon-multi-tenancy.md gained a full "Implementation note"
section (mirroring ADR-054's own, same header convention) documenting this
finding in detail, plus a Status line update. assets/mae-adr.cozo
regenerated (make adr-kb) to keep the ADR KB staleness gate green. Issue
#410 closed with the same explanation.

Verified: daemon workspace's full test suite (cd daemon && cargo test
--all-targets) -- 315 total tests across all binaries/integration files, 0
failed; cargo clippy -- -D warnings and cargo fmt --check both clean. Ran
the 3 new/newly-relevant tests 5x each locally to rule out timing flakiness
before relying on their thresholds. Editor workspace (cargo check
--workspace --all-targets) unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cuttlefisch added a commit that referenced this pull request Jul 26, 2026
…ied not rewritten

Tackled ahead of Phase C by explicit decision (see this session's stocktaking): Phase D
was already well-specified and named as this ADR's own single highest-priority
adversarial test (real Gitea CVE-2026-27771/CVE-2026-58444 + Vaultwarden CVE-2026-27898
precedent), while Phase C's stated premise needed correcting first. Same principle-#15
discipline as Phase B: write the adversarial tests against current code before assuming
new enforcement needs building.

Three named cases, three different outcomes:

- IDOR case (this phase's PRIMARY test): a genuine structural property of Phase A's own
  addressing, not a bolted-on check -- snapshot_query_layer/snapshot_store resolve the
  instance address to one specific Arc<CozoKbStore> before any inner ID is looked at, so
  every lookup is backed by only that store's relations. Proved with a real cross-instance
  ID collision (handler::tests::
  idor_a_valid_instance_address_never_resolves_a_different_tenants_id): a node inserted
  directly into tenant B's store, requested via a validly-addressed tenant A request,
  resolves Null across kb/get/kb/links_from/kb/links_to, with a third uninvolved tenant C
  per principle #14's N-way requirement.
- Role composition: roles are derived per-collection in collab_handler/mod.rs's
  kb_access -- an entirely separate daemon listener (mTLS collab) from the KB Unix-socket
  path Phase A/B touched (which has no principal/role concept at all, by design -- see
  daemon/src/config.rs's own comments). No existing test proved a principal holding
  DIFFERENT roles on two different KBs doesn't leak the stronger one across the boundary,
  so daemon/src/collab_handler/tests/collab_handler_cross_kb_role_isolation_tests.rs (new
  file) closes that gap: bob, real Owner of his own KB, denied an Owner-only action on a
  second KB where he's only Viewer; the reverse also verified.
- Forged/rotated-key signature: already covered, pre-existing, unrelated to this ADR --
  shared/sync/src/membership.rs's tampering_any_field_breaks_the_signature (+ several
  collab_handler forged-signature tests) confirmed by direct reading, not duplicated.

docs/adr/060-daemon-multi-tenancy.md gained a Phase D Implementation Note (same
convention as Phase B/C's) plus a Status line update. assets/mae-adr.cozo regenerated.
Issue #412 closed, including an honest caveat: the issue's own "regardless of quota
headroom" DoD phrasing can't be fully tested yet since Phase C's quota mechanism doesn't
exist -- the role-isolation property itself is verified; the quota-interaction half needs
re-checking once Phase C ships.

Verified: daemon workspace full test suite (cd daemon && cargo test --all-targets) --
104 bin + 156 lib tests, 0 failed; clippy -D warnings and fmt --check both clean. New
tests run 3x locally to rule out flakiness before relying on them (both are deterministic
logic tests, not timing-based, so risk was low but checked anyway). Editor workspace
(cargo check --workspace --all-targets) unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cuttlefisch added a commit that referenced this pull request Jul 26, 2026
…uota keys not one

Design-only pass (no code), per the same principle-#15 discipline that resolved Phase B
and Phase D: re-checked Phase C's own premise against the real codebase before treating
it as a starting point.

Two real corrections found:

1. Phase C's Decision text claimed quotas would "extend ADR-054's already-existing
   per-principal/per-IP soft throttle mechanism." That mechanism doesn't exist anywhere
   in this daemon -- only daemon/src/conn_limit.rs's ConnLimiter (a global, identity-blind
   connection-count cap) exists, confirmed by daemon/src/config.rs's and main.rs's own
   doc comments and a Cargo.lock grep turning up zero rate-limiting crates. Phase C is
   "build the first one," not "extend."

2. Quotas can't be keyed on principal identity uniformly across all three listeners --
   the KB Unix socket (where Phase A's instance addressing lives, and which carries the
   bulk of routine local traffic) has zero principal concept at all, by deliberate,
   documented design ("no principal or IP on a Unix domain socket"). Resolution: two
   quota keys, not one -- Phase A's instance address on the KB socket, real authenticated
   principal on collab/OAuth. Not a workaround; the KB socket's local-trust model is
   permanent, not a gap to route around with a new auth handshake.

Corrected design, grounded in real codebase investigation (two Explore agents) and real
external precedent (WebSearch): a [[tenant]] daemon.toml schema + dashmap-backed
TenantRegistry sibling to DaemonState (not inside it, per Phase B's own lock-contention
finding -- dashmap v6.2.1 is already fully resolved transitively via yrs, zero new
crates); a cost-weighted single points budget per 60s window (GitHub's own production
secondary-rate-limit shape: reads cost 1, writes cost 5, not four flat counters);
eviction reusing existing precedent exactly (DocStore::evict_idle,
mcp_session_windows' coarse-evict-then-self-heal, run_watcher_tick's existing
retain/recreate idiom) rather than inventing a new one. Kubernetes ResourceQuota/
LimitRange's "start generous, tighten from real usage" guidance grounds the default
values. Verification section's Phase C bullets rewritten to match (6 concrete
adversarial tests, up from 2 vague ones).

Issue #411 body rewritten to match the corrected DoD, cross-linked to this ADR section.
assets/mae-adr.cozo regenerated (make adr-kb) to keep the ADR KB staleness gate green.

No code changes in this pass -- actual implementation (TenantRegistry, the daemon.toml
schema, the cost-weighted check wired into dispatch, the eviction sweep, the 6 tests) is
separately-scoped future work, the same way Phases A/B/D were each their own
implementation pass after design was settled.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cuttlefisch added a commit that referenced this pull request Jul 26, 2026
…ion (#411)

Implements the corrected two-key TenantRegistry design (daemon/src/tenant.rs):
a cost-weighted points budget per fixed 60s window (reads=1, scans=3,
mutations=5, GitHub-style secondary-rate-limit shape) plus a tenant-scoped
concurrent-request cap, keyed on Phase A's instance address for the
identity-blind KB Unix socket. Wired end-to-end into handler::dispatch via
snapshot_query_layer/snapshot_store, the chokepoint all 15 KB-query/hygiene
arms already funnel through. Idle-tenant eviction sweep in
run_maintenance_tick plus a manual daemon/evict_tenant RPC, reusing the
existing evict_idle/mcp_session_windows coarse-evict-then-self-heal idiom.

One design simplification made with evidence during implementation: checks
the tenant budget inside DaemonState's lock (already proven cheap by Phase
B's own real concurrent-load test) rather than before it, avoiding a second
parameter threaded through 15 production + 44 test dispatch() call sites for
no measurable benefit. TenantRegistry itself still lives outside the lock
(an Arc field, cloned cheaply), so Phase B's "no per-request-contended state
inside this lock" finding is unaffected.

14 new adversarial/integration tests (tenant.rs unit tests + handler.rs
end-to-end dispatch() tests + config.rs schema/validation tests, including a
round-trip of the documented [[tenant]] TOML shape) plus manual
--check-config verification against real valid/conflicting configs.

Collab/OAuth-side principal-keyed wiring is implemented and tested in
isolation but not yet plugged into a listener -- explicitly deferred to
issue #456 rather than left as a silent gap (principle #15), since that
surface's RPC methods need their own reasoned cost table, not a reuse of the
KB-socket one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cuttlefisch added a commit that referenced this pull request Jul 29, 2026
…n --config (#525)

* fix(kb): #357 - stopword filtering, merged strict/relaxed retrieval, light stemming

kb_search_context still missed 5/6 of #357's originally-reported queries
against a realistically-shaped hub/atom KB despite the earlier "fix"
(12e6885), which only handled the trivial toy-fixture case. Three
compounding gaps in search_ranked's retrieval stage:

- Conversational filler words (how/should/i/the/...) never literally
  substring-match a node, so multi-filler-word queries never engaged the
  existing soft-AND fallback correctly. Added a shared stopword filter
  (mae_kb::filter_stopwords) used by both search_ranked and score_node.
- The soft-AND relaxed pass only ran when the strict pass returned
  literally zero results across the whole KB — so a hub/meta node
  satisfying strict AND in full silently kept a more specific target
  (which just missed strict AND by one real term) out of the candidate
  pool entirely, before kind_role_prior's hub down-weight ever got a
  chance to compare them. Now both passes always run and merge by id
  (strict score wins; relaxed-only ids get the existing FALLBACK_PENALTY).
- Plain literal-substring field matching missed simple morphological
  variants ("targets"/"target", "self-documented"/"self-documentation").
  Added a small suffix-stripping stem() helper, tried alongside the
  literal term in search_ranked_pass.

Verified via a new test built from a synthetic KB mirroring the original
reporter's dev-practices-kb shape (3 hubs, 1 meta, 5 target notes) running
the issue's exact 6 canonical queries — all 6 now land their target in the
top 3, up from 1/6 before this fix. Full mae-kb + mae-ai suites and the
kb_search_grading regression-floor dipstick still pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(kb): #366 - extend AI-residency seed exemption to links_from/links_to/shortest_path

"Bucket B" follow-up to #358/#361. Unlike the Bucket-A tools (kb_related,
kb_graph, kb_neighborhood, kb_health, kb_graph_view_state), neither KB
backend's links-relation query ever touched a linked node's full Node, so
kb_links_from/kb_links_to were denied outright (or, for links_from,
under-filtered) whenever ANY registered KB was LocalModelsOnly-restricted,
with no way to let MAE's own seed content through.

- kb_links_from/kb_links_to: reclassified SingleTarget/UnscopedFederatedContent
  -> SingleTargetFilterable. Each linked id is now enriched via the same
  GraphNeighbors::describe() backend kb_graph/kb_related already use (new
  LinksBackend helper in crates/ai/src/tool_impls/kb.rs), then post-filtered
  with filter_residency_exempt_by. Output shape is now consistently
  `[{"dst"/"src", "rel_type"?}]` for both the query-layer and in-memory
  paths (previously the in-memory path returned bare id strings -- an
  inconsistency this removes as a side effect).
- kb_shortest_path: reclassified SingleTarget -> SingleTargetFilterable.
  CozoKbStore::shortest_path is currently a reachability check, not real
  path reconstruction (only ever returns [from, to]), so today from/to are
  already covered by the anchor gate -- filtering here anyway future-proofs
  this tool instead of silently regressing if that reachability check is
  ever upgraded to return real intermediate hops. Drops the WHOLE path
  (not just the offending hop) when any returned id isn't seed-exempt,
  since a partial path isn't a meaningful result.

kb_list's CozoDB-backed path is explicitly NOT done here (CLAUDE.md
principle #15's "bounded down payment" -- extending it needs a
KbQueryLayer::list_ids trait signature change across all eight
implementors, a much larger blast radius than the other three tools for a
precision improvement rather than a security fix). kb_list stays
UnscopedFederatedContent (safe today, just coarser than necessary);
tracked as a follow-up issue cross-linked from #366.

8 new adversarial tests prove both directions: a restricted federated
instance's non-seed content is dropped while its seed content survives,
and a local-provider requester bypasses filtering entirely. Full mae-ai +
mae ai_residency:: suites (including the every_kb_tool_and_help_open_is_
explicitly_classified exhaustiveness check) pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(daemon): #461 - honor --config in the scheduler, not the default path

main() called DaemonScheduler::new(DaemonConfig::load(), ...) -- the
hardcoded-default-path config -- instead of the already CLI-resolved
`config` local, so a `--config <path>` override silently didn't apply to
any scheduler-driven behavior (watcher/maintenance/health tick intervals,
enrichment settings) even though every other part of main() correctly
used the resolved config.

Added Clone to DaemonConfig (every nested substruct already derived it)
so the resolved config can be passed to the scheduler without consuming
the local main() still needs afterward.

New regression test proves the actual property the bug broke: a scheduler
constructed with a fast custom watcher_interval_ms produces meaningfully
more drain cycles within a bounded window than the 500ms default could --
if the scheduler were still silently running on a default config instead
of the one it was constructed with, this test would see ~0 cycles.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
cuttlefisch added a commit that referenced this pull request Aug 5, 2026
…647)

`spawn_collab_server` put `doc_store`, `broadcaster` and `owner` into
`DaemonState` inside the `"key" =>` arm of its auth match. Under `psk` or
`none` — and `none` is the DEFAULT — the collab server ran normally while all
three stayed `None`, with no error anywhere.

What that cost, none of it visible as a failure:

  * `daemon/status` reported `kb_collections: []` and `primary_exists: false`
    for a daemon that genuinely hosted the primary KB. An editor reads exactly
    those two fields (`probe_daemon_hosts_primary` -> `should_attach_daemon_reads`)
    to decide whether to route KB reads through the daemon, so it permanently
    declined to — contradicting that function's own doc comment.
  * `kb/node_crdt` returned `NotReady`, so the thin-client node hydration path
    (ADR-029 Phase D3b) was dead on every psk/none daemon.
  * `connections.collab.sessions` was absent from `daemon/status`.

The comment on the assignment cites ADR-025 §"Driving surfaces" and ends "Key
mode only: a P2P share needs the owner-signing identity" — which justifies
`owner` and only `owner`. That ADR section is about surface parity across
CLI/command/Scheme/MCP; it says nothing about scoping state by auth mode.
`git blame` puts all three lines in one commit (fdec8d8, P2P Phase 2a): they
were introduced together to serve `p2p/share_kb`, in the arm where the identity
happened to be in scope. Every later reader started reading a field that was
never scoped for it.

This is the fourth time the same defect has been worked around rather than
fixed. ADR-053 hit it and routed around it — `doc_store_for_query` is built in
`main()` precisely so the OAuth query surface works "independent of whether the
TCP listener's own auth setup succeeds" — and the connection counters added
alongside this deliberately avoided the broadcaster for the same reason.
Principle #15: consolidate, don't add a fifth.

`doc_store` and `broadcaster` now go into `DaemonState` in `main()`, beside
`doc_store_for_query`, for every auth mode. `owner` stays in the `"key"` arm,
correctly: it is a signing identity that only exists there, and `config.rs`
already rejects `p2p.enabled` unless `auth.mode == "key"`.

Moves `p2p/share_kb`'s actionable diagnostic from the `doc_store` check to the
`owner` check. It was on `doc_store` because that was the field that was `None`
in psk/none; now that doc_store is always populated, leaving it there would have
traded a data-visibility bug for "daemon owner identity unavailable" — a support
ticket instead of an answer.

Gates: `daemon/tests/daemon_state_auth_modes_e2e.rs` spawns a real daemon on the
DEFAULT config (auth.mode=none) and asserts `kb/node_crdt` serves rather than
erroring, and that `sessions` is reported. Both fail against the old wiring
(verified by reinstating it).

Each has a paired non-vacuity control that runs with collab genuinely disabled
and asserts the OPPOSITE — `NotReady`, and `collab` absent from the report.
Without those, "does not return NotReady" would pass whether or not the fix is
present, because it would never have been shown that NotReady is reachable at
all. The controls still pass against the old code, which is the point: they
measure something different.

`hub_observability_e2e.rs`'s `sessions`-is-absent assertion becomes an equality
assertion against `active`, exactly as its own comment instructed: "If this now
reports a number, the wiring changed — extend this test to assert it equals
`active` instead of removing it."

Also reworks the shared harness's readiness probe: it opened a TCP connection to
the collab port, which is a real client that lands in `connections.collab.active`
and races any test asserting a zero baseline (that flaked once already), and
which cannot work at all for a collab-disabled daemon. Readiness is now a
`daemon/status` query on the KB socket — non-perturbing, and valid for either
configuration.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant