feat(hub): turn hubs into everything — named roles, index engine, hub-to-hub subscription (0383) - #607
Merged
crs48 merged 15 commits intoJul 20, 2026
Conversation
…cross-hub agreement Extracts fuseFederatedResults as a pure exported function and swaps the order: reciprocal rank fusion now sees every per-source copy of a result, and deduplication keeps one representative per cid afterwards (preferring the local copy). Previously deduplicateByCid ran first, collapsing each document to a single source, so fusion degenerated into a rank transform of one hub's ordering — a document three hubs agreed on earned the same credit as a single-source hit (explorations 0367/0383 W0). Signed-off-by: xNet Test <chris.smothers@gmail.com>
Signed-off-by: xNet Test <chris.smothers@gmail.com>
Adds HUB_ROLES (personal/demo/community/index/registry) as config presets spread into resolveConfig between DEFAULT_CONFIG and explicit options, so precedence is preset < config < flags and a role can never override an operator's explicit choice. --role <name> and HUB_ROLE select a role; --demo and HUB_MODE=demo remain as aliases for --role demo. Demo mode is now a preset plus per-cap resolvers (resolveMaxBlobBytes, resolveDiskWatchdogBytes, resolveResetIntervalMs, resolveResetOnCorruption, resolveHandshakeDemoLimits) following #603's one-resolver rule — server.ts carries zero open-coded demo ternaries. Federation, shards and crawl are reachable from the CLI for the first time via the community and registry presets; the index preset pins the legacy search stack off (0367). The startup banner and /health/badge label are role-aware, and railway.toml moves the live demo hub to --role demo (byte-identical config, proven by test). Records the R6 decision: the per-user cap stays universal, not managed-only. Signed-off-by: xNet Test <chris.smothers@gmail.com>
Grows the mount-only HubFeature contract with four optional hooks — services, loops, ws, storage — with the registry owning loop start/stop (reverse-order shutdown, failures isolated) and storage running before mount with table-prefix ownership enforced at the DDL seam (fed_/crawl_/idx_/sub_/pi_). Existing features remain valid unchanged. Federation, shards and crawl now assemble through the registry like the integration features: mounts and lifecycle moved out of server.ts's imperative wiring into closed-over feature definitions. Adds the public-interactions feature (0378's read surface, born as a module): GET /public/interactions/:nodeId resolves the author's PublicInteractionPolicy — found O(1) at the new deterministic publicInteractionPolicyId in @xnetjs/data — falling back to schema defaults, 404ing non-public nodes exactly like the public read surface. On in the community and index role presets. Signed-off-by: xNet Test <chris.smothers@gmail.com>
Adds the atprotoIndex feature module: enumerate the adopted site.standard.* collections (0372), fetch records via an injected IndexSource (fixtures in tests; listReposByCollection + listRecords over fetch by default), quarantine malformed records (0367 E22), and serve a DETERMINISTIC snapshot — sorted by URI, no wall-clock — so two rebuilds from identical inputs are byte-identical. That determinism test is 0374's rebuild-and-diff gate running in the ordinary CI lanes, and scripts/index/rebuild-and-diff.mjs is the same property against the live network — the mirror-not-master receipt as a runnable command. The role is derived-only by construction: assertDerivedOnlyDataDir refuses a data dir holding tenant state (hub.db without the idx_role claim) before storage opens, artifacts are idx_-prefixed files (restart-from-source is the DR — Bobbin's model, 0381), and the legacy search/shard stack stays off in the index preset. Signed-off-by: xNet Test <chris.smothers@gmail.com>
…ate (0383 W4) The literal hub-of-hubs, by composition rather than protocol: a gateway hub embeds the client-side MultiHubSyncManager (one per peer, rooms multiplexed over one socket) and speaks the hub's existing wire protocol as an ordinary client — subscribe for the live tail, node-sync-request from a persisted high-water mark for backfill, paged via hasMore. The mirror is derived state in sub_-prefixed files, folded per node in lamport order, served ONLY under /sub/* — never re-exported into rooms or the public surface, which is what makes subscription cycles harmless (no amplification path). Direct self-subscription is rejected at boot. Proven end-to-end: hub B mirrors hub A through A's restart. The hub gains a persistent system identity (0371's blocker): did:key minted at first boot, kept in the data dir, stable across restarts, surfaced on /health, and consumed by relay envelope signing (previously an ephemeral per-boot identity). The DID signs TRANSPORT only — never node authorship (0371's rule, recorded in code). The 0258 trust tiers are now enforced where bytes flow: MultiHubSyncManager.publishScoped withholds plaintext from zero-knowledge destinations and reports published/withheld; mayReceivePayload in replication-scope is the single definition. Both 'not yet enforced' flags are gone. Adds the gateway role preset, ADR-26 (cross-hub grants: plane split before propagation — delegation chains, never grant-table replication) and ADR-27 (@xnetjs/server: scope out, stays a separate product). Signed-off-by: xNet Test <chris.smothers@gmail.com>
…3 W5) deploy/hub-with-pds/ is the blessed one-command pair: the hub (any role) plus the OFFICIAL bluesky-social/pds container behind one Caddy, with the wildcard-DNS requirement documented. The PDS stays a neighbour, never a hub role (0365; 0382's MinIO-gateway lesson). ProvisionSpec gains a sidecars slot for managed placement; the Cloud Run adapter refuses loudly rather than silently dropping a sidecar it cannot host yet. Signed-off-by: xNet Test <chris.smothers@gmail.com>
Records the comment-storm measurement in 0383 (idle sync RTT p50 0.4ms / p95 1.8ms; unchanged under a 300-comment storm), asserts the hub DID never appears as a stored change author, validates the compose template, and gives the six-preset boot loop a 60s budget so full-suite CPU contention cannot flake it. Signed-off-by: xNet Test <chris.smothers@gmail.com>
…se artifacts The new role/subscriber/index tests bound ports in the 14480-14499 range, colliding with node-change-batch.test.ts (14481) in full-suite runs. Move them to 14580-14595 (including the cycle-guard peer URL, which must match its own hub port for the guard to trip). Adds the changeset for the @xnetjs/data and @xnetjs/runtime surface and the changelog fragment. Signed-off-by: xNet Test <chris.smothers@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…plementation plan Signed-off-by: xNet Test <chris.smothers@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Preview removed for PR #607. |
…port The hub image Dockerfile enumerates hub's workspace closure explicitly, and W4's @xnetjs/runtime dependency pulled six new packages into that closure (runtime, data-bridge, history, plugins, trust, licenses) — added to all four COPY blocks; the list now diffs clean against `pnpm --filter '@xnetjs/hub...'`. Prettier over the seven new files CI flagged, and the regenerated data API report for the intended publicInteractionPolicy exports (matches the minor changeset). Signed-off-by: xNet Test <chris.smothers@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Since #603 armed quotaBytes on every hub, each append re-ran the SUM(LENGTH(payload_json)) over the author's entire log — O(rows²) across a bulk ingest, the 0357 regression class, and the cause of main's red test (3/3) lane (10k batch ingest 36-37s vs the 30s budget). Usage is now cached per author and bumped as appends land; a write is never rejected on a cached number (the gate re-reads storage at the quota boundary), and a 30s TTL bounds drift from out-of-band writers (pack import, eviction). Local: 10k ingest 17.6s -> 4.1s; managed-quota suite unchanged and green. Signed-off-by: xNet Test <chris.smothers@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The earlier regeneration ran against a stale local dist and picked up a polluted dts chunk name (types-B3LD0ueI); a forced clean rebuild restores the canonical types-gws1tSf- chunk CI computes. The intended publicInteractionPolicy surface additions are unchanged. Signed-off-by: xNet Test <chris.smothers@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A drift that only reproduces on CI (round-tripping a platform-sensitive dts chunk name) is undebuggable from a diffstat alone. Signed-off-by: xNet Test <chris.smothers@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ubs-into-everything-the-role-implementa
crs48
deleted the
claude/0383-turning-hubs-into-everything-the-role-implementa
branch
July 20, 2026 23:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements exploration 0383 (#605) in full — all six workstreams, 21/21 implementation items and 10/10 validation checks.
What landed
fuseFederatedResultssees every per-source copy before dedupe. PR-tax docs in the hub README.--role personal|demo|community|index|registry|gateway(HUB_ROLEenv;--demokept as deprecated alias). Presets are one spread inresolveConfig— precedence preset < explicit config < CLI. Five new named resolvers replace every open-codeddemo ?ternary in server.ts (the fix(hub): enforce the change-log storage quota on managed hubs (0381) #603 one-resolver rule).services/loops/ws/storagehooks; the registry owns loop lifecycle (reverse-order stop, error isolation) and enforces table-prefix discipline (fed_/crawl_/idx_/sub_/pi_) viaassertOwnTable. Federation, shards, crawl, and public interactions all run as features.idx_role.jsonsentinel refuses tenanthub.db), deterministic snapshot (sorted by URI, no wall-clock) so two rebuilds diff to zero — restart-from-source IS the DR. Never touches the legacy search/shard stack.scripts/index/rebuild-and-diff.mjsis the stranger's live recipe.node-sync-requestbackfill + publish-wrapped live tail), mirrors under/sub/*only (no transitive re-export by construction), reconnects with backoff, and refuses self-subscription. Persistentdid:keyhub identity (hub_identity.json) replaces the ephemeral relay key and unblocks 0371;/healthexposeshubDid+role.publishScopedgains the 0258 trust gate (mayReceivePayload): plaintext withheld from zero-knowledge destinations,{published, withheld}return.ProvisionSpec.sidecarsslot (cloud-run adapter refuses),deploy/hub-with-pds/compose bundle (hub + official PDS + Caddy), ADR-26 (cross-hub grants: plane split, delegation chains rooted at hub DID) + ADR-27 (@xnetjs/serverscope-out).Deviations from the doc (recorded in the exploration)
docker compose configlevel (no Docker daemon in the environment).Evidence
turbo run typecheck --forcegreen across 97 tasks.@xnetjs/dataminor (publicInteractionPolicyId),@xnetjs/runtimeminor (publishScoped trust gate). Changelog fragment included.🤖 Generated with Claude Code