Skip to content

fix(collab): pre-dogfood hull patch — batch-collapse + scale defaults + config honesty (#188) - #264

Merged
cuttlefisch merged 1 commit into
mainfrom
fix/pre-dogfood-hull-patch
Jul 3, 2026
Merged

fix(collab): pre-dogfood hull patch — batch-collapse + scale defaults + config honesty (#188)#264
cuttlefisch merged 1 commit into
mainfrom
fix/pre-dogfood-hull-patch

Conversation

@cuttlefisch

Copy link
Copy Markdown
Owner

Pre-dogfood "hull patch" — verified fixes from the system review (#188)

A systematic 6-subsystem trace (KB ingestion/scale, sharing/derivation, E2E, identity, mesh, config) ran ahead of the enterprise-KB dogfood (~2,800 nodes / ~12k links / 472 MB over the hub — mesh deferred by decision). Every finding was verified against the code before acting. This PR ships the verified, tested, safe set. Architecture/scale items are tracked in #263; the three daemon-security bugs below are handed off with precise diagnoses for careful implementation.

Fixed here (verified + tested)

  1. Batch-collapse (silent data loss). Several KB lifecycle actions in one Scheme/AI eval (bulk member onboarding) collapsed to the lastqueue_kb_collab_action overwrote a single pending_intent slot. Now fans overflow through the existing one-per-tick reconnect_intents queue. Test added.
  2. Scale defaults. max_documents 1000→4096 (one yrs doc per node; 1000 thrashes a 2,800-node KB — pure LRU cap), max_update_size_bytes 1→4 MiB (a large node's full-state push was silently rejected). DAEMON_ADMIN.md documents the doc-per-node model + tuning.
  3. Config honesty. Marked RESERVED (verified zero readers) the dead options collab_batch_update_ms, collab_max_pending_updates, kb_backup_interval, kb_backup_retention (backup module has no callers) and dead daemon fields sync_interval_secs/decay_interval_secs/heartbeat_interval_secs; removed the misleading DAEMON_ADMIN line. (Marked not deleted — removing a registered option errors on config-load.)

Handed off — daemon-security bugs (verified; precise diagnoses; deserve fresh, careful implementation + tests, not a rushed end-of-session change)

  • Owner rotation → self-lockout on a hub (un-anchored) KB. daemon/src/collab_handler.rs:2870 mirrors the rotation successor into member_roles only on the member self-service branch; the owner (Manage=Allow) leaves rebind_pairs empty, so after the documented "rotate then reconnect" the owner gets role_of(new_fp)=NoneDeny on their own KB. Fix must extract the owner's Rebind (successor,predecessor) from the update and mirror it — but the owner's rotation update also carries E2E content-key re-wraps, so it can't reuse verify_member_self_service_update (rejects anything beyond a bare Rebind); needs a dedicated, separately-tested extractor that cannot become a roster-injection hole.
  • Keyless add on an E2E KB. crates/mae/src/collab_bridge.rs:3893 gates the wrap on !add, so kb-add-member by fingerprint on an encrypted KB creates a member who can never decrypt, with success UX. Only join→approve wraps. Fix: warn/refuse and steer to request-join+approve (or reactive re-wrap once they publish a wrap key).
  • Import silently becomes an unpersisted in-memory KB. crates/core/src/editor/kb_ops.rs:246 falls back to import_org_dir (in-memory, handle discarded) on any import_org_dir_to_store error, and federation.rs:550 aborts on the first bad node → partial persistent state + an ephemeral copy that vanishes on restart. Fix: per-node error tolerance + a hard, surfaced error instead of a silent ephemeral swap.

Also deferred (see #263): links:by_dst index (backlinks/neighborhood/related full-scan 12k links), ≥3-member convergence + recovery-on-anchored + leaked-recovery-supersede + offline-across-rotation tests, op-log pruning, streaming mesh join.

Verification: mae-core + daemon (142) lib green; workspace clippy --all-targets clean (both workspaces).

…ults + config honesty (#188)

From the pre-dogfood system review (6-subsystem trace, #188). This is the
verified, tested set; the deeper daemon-security items are handed off with
precise diagnoses (see the PR body) and the architecture/scale items are
tracked in #263. Dogfood is hub-scoped (maintainer decision), so mesh-only
findings are deferred.

Fixes:
- Batch-collapse (data loss, verified): the Scheme/AI surface can lower
  several KB lifecycle actions in ONE apply cycle (bulk member onboarding),
  but queue_kb_collab_action wrote a single pending_intent slot, so all but
  the LAST were silently dropped with no error. Now the overflow fans out
  through the existing one-per-tick reconnect_intents queue (FIFO, order
  preserved). Test batched_kb_collab_actions_do_not_collapse_to_the_last.

- Scale defaults for a 472 MB / ~2,800-node KB: max_documents 1000 -> 4096
  (one yrs doc PER node -> 1000 thrashes a 2,800-node KB; pure LRU cap, so
  raising it only costs memory when exceeded). max_update_size_bytes 1 MiB ->
  4 MiB (a large node's full-state push on reseal/share was silently REJECTED;
  4 MiB keeps a DoS bound with headroom). DEFAULT_MAX_UPDATE_SIZE kept in
  sync. DAEMON_ADMIN.md documents the doc-per-node model + the tuning knobs.

- Config honesty (false control surfaces, verified zero readers): marked
  RESERVED (see #263) the dead options collab_batch_update_ms,
  collab_max_pending_updates, kb_backup_interval, kb_backup_retention
  (backup module has no callers) and the dead daemon fields sync_interval_secs,
  decay_interval_secs, heartbeat_interval_secs (the editor's
  collab_heartbeat_interval is the real client keepalive). Removed the
  misleading sync_interval_secs line from DAEMON_ADMIN.md. (Marked, not
  deleted: removing a registered option errors on config-load.)

Refs #188, #263.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cuttlefisch cuttlefisch added the release:none Skip version bump on merge label Jul 3, 2026
@cuttlefisch
cuttlefisch merged commit 0e3980c into main Jul 3, 2026
27 of 28 checks passed
@cuttlefisch
cuttlefisch deleted the fix/pre-dogfood-hull-patch branch July 3, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:none Skip version bump on merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant