fix(collab): pre-dogfood hull patch — batch-collapse + scale defaults + config honesty (#188) - #264
Merged
Merged
Conversation
…ults + config honesty (#188) From the pre-dogfood system review (6-subsystem trace, #188). This is the verified, tested set; the deeper daemon-security items are handed off with precise diagnoses (see the PR body) and the architecture/scale items are tracked in #263. Dogfood is hub-scoped (maintainer decision), so mesh-only findings are deferred. Fixes: - Batch-collapse (data loss, verified): the Scheme/AI surface can lower several KB lifecycle actions in ONE apply cycle (bulk member onboarding), but queue_kb_collab_action wrote a single pending_intent slot, so all but the LAST were silently dropped with no error. Now the overflow fans out through the existing one-per-tick reconnect_intents queue (FIFO, order preserved). Test batched_kb_collab_actions_do_not_collapse_to_the_last. - Scale defaults for a 472 MB / ~2,800-node KB: max_documents 1000 -> 4096 (one yrs doc PER node -> 1000 thrashes a 2,800-node KB; pure LRU cap, so raising it only costs memory when exceeded). max_update_size_bytes 1 MiB -> 4 MiB (a large node's full-state push on reseal/share was silently REJECTED; 4 MiB keeps a DoS bound with headroom). DEFAULT_MAX_UPDATE_SIZE kept in sync. DAEMON_ADMIN.md documents the doc-per-node model + the tuning knobs. - Config honesty (false control surfaces, verified zero readers): marked RESERVED (see #263) the dead options collab_batch_update_ms, collab_max_pending_updates, kb_backup_interval, kb_backup_retention (backup module has no callers) and the dead daemon fields sync_interval_secs, decay_interval_secs, heartbeat_interval_secs (the editor's collab_heartbeat_interval is the real client keepalive). Removed the misleading sync_interval_secs line from DAEMON_ADMIN.md. (Marked, not deleted: removing a registered option errors on config-load.) Refs #188, #263. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-dogfood "hull patch" — verified fixes from the system review (#188)
A systematic 6-subsystem trace (KB ingestion/scale, sharing/derivation, E2E, identity, mesh, config) ran ahead of the enterprise-KB dogfood (~2,800 nodes / ~12k links / 472 MB over the hub — mesh deferred by decision). Every finding was verified against the code before acting. This PR ships the verified, tested, safe set. Architecture/scale items are tracked in #263; the three daemon-security bugs below are handed off with precise diagnoses for careful implementation.
Fixed here (verified + tested)
queue_kb_collab_actionoverwrote a singlepending_intentslot. Now fans overflow through the existing one-per-tickreconnect_intentsqueue. Test added.max_documents1000→4096 (one yrs doc per node; 1000 thrashes a 2,800-node KB — pure LRU cap),max_update_size_bytes1→4 MiB (a large node's full-state push was silently rejected).DAEMON_ADMIN.mddocuments the doc-per-node model + tuning.RESERVED(verified zero readers) the dead optionscollab_batch_update_ms,collab_max_pending_updates,kb_backup_interval,kb_backup_retention(backup module has no callers) and dead daemon fieldssync_interval_secs/decay_interval_secs/heartbeat_interval_secs; removed the misleading DAEMON_ADMIN line. (Marked not deleted — removing a registered option errors on config-load.)Handed off — daemon-security bugs (verified; precise diagnoses; deserve fresh, careful implementation + tests, not a rushed end-of-session change)
daemon/src/collab_handler.rs:2870mirrors the rotation successor intomember_rolesonly on the member self-service branch; the owner (Manage=Allow) leavesrebind_pairsempty, so after the documented "rotate then reconnect" the owner getsrole_of(new_fp)=None→Denyon their own KB. Fix must extract the owner's Rebind(successor,predecessor)from the update and mirror it — but the owner's rotation update also carries E2E content-key re-wraps, so it can't reuseverify_member_self_service_update(rejects anything beyond a bare Rebind); needs a dedicated, separately-tested extractor that cannot become a roster-injection hole.crates/mae/src/collab_bridge.rs:3893gates the wrap on!add, sokb-add-memberby fingerprint on an encrypted KB creates a member who can never decrypt, with success UX. Only join→approve wraps. Fix: warn/refuse and steer to request-join+approve (or reactive re-wrap once they publish a wrap key).crates/core/src/editor/kb_ops.rs:246falls back toimport_org_dir(in-memory, handle discarded) on anyimport_org_dir_to_storeerror, andfederation.rs:550aborts on the first bad node → partial persistent state + an ephemeral copy that vanishes on restart. Fix: per-node error tolerance + a hard, surfaced error instead of a silent ephemeral swap.Also deferred (see #263):
links:by_dstindex (backlinks/neighborhood/related full-scan 12k links), ≥3-member convergence + recovery-on-anchored + leaked-recovery-supersede + offline-across-rotation tests, op-log pruning, streaming mesh join.Verification: mae-core + daemon (142) lib green; workspace clippy
--all-targetsclean (both workspaces).