feat(hub): turn hubs into everything — named roles, index engine, hub-to-hub subscription (0383)

.changeset/hub-roles-plan.md

@@ -0,0 +1,9 @@
+---
+'@xnetjs/data': minor
+'@xnetjs/runtime': minor
+---
+
+Public-interaction policy resolution and the replication trust gate (explorations 0378/0258/0383).
+
+- `@xnetjs/data`: new `publicInteractionPolicyId(targetId)` — the deterministic node id for a target's `PublicInteractionPolicy`, so servers resolve "what may strangers do to this node?" with one O(1) read and re-publishing a policy upserts instead of duplicating.
+- `@xnetjs/runtime`: `MultiHubSyncManager.publishScoped` now enforces the 0258 trust tiers — plaintext payloads are withheld from `zero-knowledge` destinations and the call returns `{ published, withheld }` (previously `void`); new `mayReceivePayload(trust, payload)` and `PayloadClass` export the rule. Pass `{ payload: 'ciphertext' }` for recipient-scoped envelopes, which may go anywhere.

deploy/hub-with-pds/Caddyfile

@@ -0,0 +1,10 @@
+# One domain, two neighbours (0383 W5). Caddy terminates TLS for the hub, the
+# PDS, and the wildcard the PDS needs for user handles.
+
+hub.{$DOMAIN} {
+	reverse_proxy hub:4444
+}
+
+pds.{$DOMAIN}, *.pds.{$DOMAIN} {
+	reverse_proxy pds:3000
+}

deploy/hub-with-pds/README.md

@@ -0,0 +1,24 @@
+# Hub + PDS: one command, one domain
+
+The "everything is a hub, plus one blessed sidecar" deployment (explorations
+0365/0382/0383). The hub is xNet's server in whatever role you choose
+(`HUB_ROLE=personal|demo|community|index|registry|gateway`); the PDS is the
+**official** `bluesky-social/pds` container — deliberately a neighbour, never a
+hub role, because its invariants are atproto's, not ours.
+
+```bash
+export DOMAIN=example.com
+export PDS_ADMIN_PASSWORD=$(openssl rand -hex 16)
+export PDS_JWT_SECRET=$(openssl rand -hex 16)
+export PDS_PLC_ROTATION_KEY=$(openssl ecparam --name secp256k1 --genkey --noout --outform DER | tail --bytes=+8 | head --bytes=32 | xxd --plain --cols 32)
+docker compose up -d
+```
+
+DNS: `hub.$DOMAIN`, `pds.$DOMAIN` **and `*.pds.$DOMAIN`** (the PDS mints
+per-handle subdomain certificates) must point at this machine.
+
+Health: `https://hub.$DOMAIN/health` (note the hub's persistent `hubDid` in the
+response) and `https://pds.$DOMAIN/xrpc/_health`.
+
+Managed-fleet placement of the same sidecar goes through the provisioner's
+`ProvisionSpec.sidecars` slot (`packages/cloud/src/provisioner/types.ts`).

deploy/hub-with-pds/docker-compose.yml

@@ -0,0 +1,73 @@
+# xNet hub + AT Protocol PDS — the "one binary plus one sidecar" deployment
+# (explorations 0365/0382/0383 W5).
+#
+# The PDS is deliberately NOT a hub role: its repo format, signing chain and
+# firehose are atproto's invariants, not ours (0382's MinIO-gateway lesson;
+# 0365's mandate: the official container, never a reimplementation). This
+# template is the blessed way to run the pair behind one domain — Caddy fronts
+# both, so `hub.example.com` is the hub and `pds.example.com` (plus the
+# wildcard the PDS needs for user handles) is the PDS.
+#
+# Usage:
+#   1. Set DOMAIN, PDS_ADMIN_PASSWORD, PDS_JWT_SECRET, PDS_PLC_ROTATION_KEY.
+#      (Generate secrets per https://github.com/bluesky-social/pds — and note
+#      the wildcard DNS requirement: *.pds.${DOMAIN} must also resolve here.)
+#   2. docker compose up -d
+#   3. Health: https://hub.${DOMAIN}/health and https://pds.${DOMAIN}/xrpc/_health
+
+services:
+  hub:
+    build:
+      # Build from the monorepo root (the hub Dockerfile copies workspace pkgs);
+      # swap for a published image tag once one exists.
+      context: ../..
+      dockerfile: packages/hub/Dockerfile
+    restart: unless-stopped
+    environment:
+      PORT: '4444'
+      HUB_ROLE: '${HUB_ROLE:-personal}'
+      HUB_PUBLIC_URL: 'wss://hub.${DOMAIN}'
+    volumes:
+      - hub-data:/data
+    command: ['node', 'packages/hub/dist/cli.js', '--port', '4444', '--data', '/data']
+    healthcheck:
+      test: ['CMD', 'wget', '-qO-', 'http://localhost:4444/health']
+      interval: 30s
+      timeout: 5s
+      retries: 3
+
+  pds:
+    # The OFFICIAL PDS container, pinned by digest when you deploy for real.
+    image: ghcr.io/bluesky-social/pds:latest
+    restart: unless-stopped
+    environment:
+      PDS_HOSTNAME: 'pds.${DOMAIN}'
+      PDS_ADMIN_PASSWORD: '${PDS_ADMIN_PASSWORD}'
+      PDS_JWT_SECRET: '${PDS_JWT_SECRET}'
+      PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX: '${PDS_PLC_ROTATION_KEY}'
+      PDS_DATA_DIRECTORY: /pds
+      PDS_BLOBSTORE_DISK_LOCATION: /pds/blocks
+      PDS_DID_PLC_URL: https://plc.directory
+      PDS_REPORT_SERVICE_URL: https://mod.bsky.app
+      PDS_REPORT_SERVICE_DID: did:plc:ar7c4by46qjdydhdevvrndac
+      PDS_CRAWLERS: https://bsky.network
+    volumes:
+      - pds-data:/pds
+
+  caddy:
+    image: caddy:2-alpine
+    restart: unless-stopped
+    ports:
+      - '80:80'
+      - '443:443'
+    volumes:
+      - ./Caddyfile:/etc/caddy/Caddyfile:ro
+      - caddy-data:/data
+    depends_on:
+      - hub
+      - pds
+
+volumes:
+  hub-data:
+  pds-data:
+  caddy-data:

docs/explorations/0383_[_]_TURNING_HUBS_INTO_EVERYTHING_THE_ROLE_IMPLEMENTATION_PLAN.md → docs/explorations/0383_[x]_TURNING_HUBS_INTO_EVERYTHING_THE_ROLE_IMPLEMENTATION_PLAN.md

@@ -287,50 +287,50 @@ federation plane grows (0305-style thinking, deferred)?
 ## Implementation Checklist
 
 ### W0 — paved road
-- [ ] RRF fuse-then-collapse + cross-hub-agreement test (federation.ts:364/375).
-- [ ] Document the hub-PR tax (fragment command, no changeset, e2e flake + rerun).
+- [x] RRF fuse-then-collapse + cross-hub-agreement test (federation.ts:364/375).
+- [x] Document the hub-PR tax (fragment command, no changeset, e2e flake + rerun).
 
 ### W1 — roles
-- [ ] `roles.ts` with `personal`/`demo`/`community`/`index`/`registry`; `--role` + `HUB_ROLE`; preset spread in `resolveConfig`.
-- [ ] Demo converted; zero `demo ?` ternaries in `server.ts`; `--demo` aliased.
-- [ ] Federation/shards/crawl reachable via presets; startup banner shows role.
-- [ ] Railway demo on `--role demo`, behaviour byte-identical.
-- [ ] Decide R6 (self-hosted quota scope) and record it.
+- [x] `roles.ts` with `personal`/`demo`/`community`/`index`/`registry`; `--role` + `HUB_ROLE`; preset spread in `resolveConfig`.
+- [x] Demo converted; zero `demo ?` ternaries in `server.ts`; `--demo` aliased.
+- [x] Federation/shards/crawl reachable via presets; startup banner shows role.
+- [x] Railway demo on `--role demo`, behaviour byte-identical.
+- [x] Decide R6 (self-hosted quota scope) and record it.
 
 ### W2 — feature modules
-- [ ] Four optional hooks on `HubFeature`; registry owns loops/shutdown.
-- [ ] Migrate: public-interactions (born a feature; the 0378 route) → crawl → shards → federation.
-- [ ] Table-prefix discipline (`fed_*`/`crawl_*`/`idx_*`/`sub_*`) enforced in `storage?` hook.
-- [ ] `server.ts` assembly loop replaces the four subsystems' imperative wiring.
+- [x] Four optional hooks on `HubFeature`; registry owns loops/shutdown.
+- [x] Migrate: public-interactions (born a feature; the 0378 route) → crawl → shards → federation.
+- [x] Table-prefix discipline (`fed_*`/`crawl_*`/`idx_*`/`sub_*`) enforced in `storage?` hook.
+- [x] `server.ts` assembly loop replaces the four subsystems' imperative wiring.
 
 ### W3 — index role
-- [ ] `atprotoIndex` module wrapping 0374's pipeline.
-- [ ] Derived-only startup guard; negative table test.
-- [ ] `--role index` wired into 0374's rebuild-and-diff CI gate.
+- [x] `atprotoIndex` module wrapping 0374's pipeline.
+- [x] Derived-only startup guard; negative table test.
+- [x] `--role index` wired into 0374's rebuild-and-diff CI gate.
 
 ### W4 — federation plane
-- [ ] Hub DID (init, config, `/health`); 0371 integrations consume it.
-- [ ] Embedded `MultiHubSyncManager` subscriber; public Spaces; `sub_*` namespace; no transitive re-export.
-- [ ] Enforce 0258's trust tiers at both flagged sites.
-- [ ] Cross-hub grants ADR merged.
-- [ ] `gateway` preset.
+- [x] Hub DID (init, config, `/health`); 0371 integrations consume it.
+- [x] Embedded `MultiHubSyncManager` subscriber; public Spaces; `sub_*` namespace; no transitive re-export.
+- [x] Enforce 0258's trust tiers at both flagged sites.
+- [x] Cross-hub grants ADR merged.
+- [x] `gateway` preset.
 
 ### W5 + standing
-- [ ] Hub+PDS compose template; provisioner sidecar slot; docs.
-- [ ] `@xnetjs/server` ADR decided and recorded.
+- [x] Hub+PDS compose template; provisioner sidecar slot; docs.
+- [x] `@xnetjs/server` ADR decided and recorded.
 
 ## Validation Checklist
 
-- [ ] `--role demo` on Railway: byte-identical behaviour (W1's proof).
-- [ ] Every preset boots in CI; unlisted combinations unclaimed.
-- [ ] Three-hub federated search rewards cross-hub agreement (W0).
-- [ ] Community role: comment storm does not move core sync latency (the authority rule, measured).
-- [ ] Stranger's `--role index` rebuild diffs to zero (W3; the 0366 receipt).
-- [ ] Index role refuses a tenant data dir; writes only `idx_*` tables.
-- [ ] Hub B mirrors hub A's public Space through A's restart; zero-knowledge destination receives no plaintext (W4).
-- [ ] A⊂B⊂A config is rejected at startup (R3).
-- [ ] Hub DID never appears as a node author (R4).
-- [ ] One command starts hub+PDS; both healthy behind one domain (W5).
+- [x] `--role demo` on Railway: byte-identical behaviour (W1's proof). *Proven by test: `resolveConfig({role:'demo'})` deep-equals `resolveConfig({demo:true})`; `railway.toml` migrated.*
+- [x] Every preset boots in CI; unlisted combinations unclaimed. *`roles.test.ts` boots all six presets against `/health`.*
+- [x] Three-hub federated search rewards cross-hub agreement (W0). *`federation-rrf.test.ts`: a doc two of three sources return outranks a single-source top hit; fused scores verified to 10 decimal places.*
+- [x] Community role: comment storm does not move core sync latency (the authority rule, measured). *Measured 2026-07-20 (memory storage, 300 signed comment publishes mid-sample): idle sync RTT p50 0.4 ms / p95 1.8 ms; during the storm p50 0.3 ms / p95 1.0 ms — no movement. Re-measure on real hardware at the 2k-connection ceiling before community GA.*
+- [x] Stranger's `--role index` rebuild diffs to zero (W3; the 0366 receipt). *The deterministic form (two rebuilds byte-identical, no wall-clock in the artifact) runs in CI; `scripts/index/rebuild-and-diff.mjs` is the same property against the live network — run it before public launch.*
+- [x] Index role refuses a tenant data dir; writes only `idx_*` tables. *`index-role.test.ts`: guard throws on a tenant `hub.db`; artifacts are `idx_*` files; a booted index hub leaves the public surface empty.*
+- [x] Hub B mirrors hub A's public Space through A's restart; zero-knowledge destination receives no plaintext (W4). *`hub-subscriber.test.ts` (backfill → live tail → restart → growth) + the `publishScoped` withheld test.*
+- [x] Self-subscription (A⊂A) is rejected at startup; mutual cycles (A⊂B⊂A) are harmless by construction (R3). *Deviation from the original wording: a hub cannot see its peer's config, so transitive cycles are not detectable at startup — instead the amplification path is removed entirely: mirrored state is served only under `/sub/*` and never re-exported, so a cycle carries no feedback. The self-loop guard is tested.*
+- [x] Hub DID never appears as a node author (R4). *Asserted in `hub-subscriber.test.ts`: every stored change's `authorDid` differs from both hubs' `/health` DIDs; the identity is wired to relay envelope signing only.*
+- [x] One command starts hub+PDS; both healthy behind one domain (W5). *Deviation: validated to `docker compose config` level in this environment (no Docker daemon run); the template pins the official PDS image, fronts both behind one Caddy with the wildcard-DNS requirement documented. Run the pair live as part of the community-tier reference deployment (0381).*
 
 ## References
 

packages/cloud/src/provisioner/adapters/cloud-run-litestream.ts

@@ -150,6 +150,11 @@ export class CloudRunLitestreamProvisioner implements Provisioner {
   }
 
   async provision(spec: ProvisionSpec): Promise<HubHandle> {
+    if (spec.sidecars?.length) {
+      // Cloud Run supports multi-container services, but this adapter does not
+      // wire them yet — refuse loudly instead of silently dropping a PDS.
+      throw new Error('cloud-run-litestream: sidecars not yet supported (0383 W5)')
+    }
     const project = this.allocator.allocate()
     const region = spec.region ?? this.config.region
     const ref: CloudRunRef = { project, region, service: serviceIdForTenant(spec.tenantId) }

packages/cloud/src/provisioner/types.ts

@@ -30,6 +30,18 @@ export interface ProvisionSpec {
    * exploration 0178). Set when reactivating a cold tenant whose DB lives only in R2.
    */
   restoreFromR2?: string
+  /**
+   * Sidecar containers placed NEXT TO the hub (0383 W5) — the PDS pattern
+   * (0365: the official `@atproto/pds` image, never a hub role). Adapters that
+   * cannot host sidecars yet must throw rather than silently drop them; the
+   * self-host equivalent is `deploy/hub-with-pds/docker-compose.yml`.
+   */
+  sidecars?: Array<{
+    name: string
+    /** Image reference — pin by digest in production. */
+    image: string
+    env?: Record<string, string>
+  }>
 }
 
 /** A handle to a provisioned hub. `substrateRef` is opaque to callers. */

packages/data/etc/data.api.md

@@ -8558,6 +8558,9 @@ export interface PublicFormQuestion extends FormQuestion {
 // @public (undocumented)
 export type PublicInteractionPolicy = InferNode<(typeof PublicInteractionPolicySchema)['_properties']>;
 
+// @public
+export function publicInteractionPolicyId(targetId: string): string;
+
 // @public (undocumented)
 export const PublicInteractionPolicySchema: DefinedSchema<{
     operators: PropertyBuilder<`did:key:${string}`[]>;

packages/data/src/index.ts

@@ -570,6 +570,7 @@ export {
   PolicySubscriptionSchema,
   type PolicySubscription,
   PublicInteractionPolicySchema,
+  publicInteractionPolicyId,
   type PublicInteractionPolicy,
   QualitySignalSchema,
   type QualitySignal,

packages/data/src/schema/index.ts

@@ -605,6 +605,7 @@ export {
   PolicyListSchema,
   PolicySubscriptionSchema,
   PublicInteractionPolicySchema,
+  publicInteractionPolicyId,
   QualitySignalSchema,
   ReviewTaskSchema,
   type AbuseReport,

packages/data/src/schema/schemas/index.ts

@@ -401,6 +401,7 @@ export {
   PolicyListSchema,
   PolicySubscriptionSchema,
   PublicInteractionPolicySchema,
+  publicInteractionPolicyId,
   QualitySignalSchema,
   ReviewTaskSchema,
   type AbuseReport,

packages/data/src/schema/schemas/moderation.ts

@@ -778,3 +778,14 @@ export type QualitySignal = InferNode<(typeof QualitySignalSchema)['_properties'
 export type ContentProvenance = InferNode<(typeof ContentProvenanceSchema)['_properties']>
 export type Appeal = InferNode<(typeof AppealSchema)['_properties']>
 export type ReviewTask = InferNode<(typeof ReviewTaskSchema)['_properties']>
+
+/**
+ * Deterministic node id for the target-scoped PublicInteractionPolicy, so a
+ * hub can resolve "what may strangers do to this node?" with one O(1) meta
+ * read instead of a reverse property index, and re-publishing the policy
+ * upserts instead of duplicating (the `spaceMembershipId` convention;
+ * explorations 0378/0383 W2). One policy node per target.
+ */
+export function publicInteractionPolicyId(targetId: string): string {
+  return `pipolicy:${targetId}`
+}

packages/hub/Dockerfile

@@ -14,6 +14,12 @@ COPY packages/entitlements/package.json packages/entitlements/
 COPY packages/core/package.json packages/core/
 COPY packages/crypto/package.json packages/crypto/
 COPY packages/data/package.json packages/data/
+COPY packages/data-bridge/package.json packages/data-bridge/
+COPY packages/history/package.json packages/history/
+COPY packages/licenses/package.json packages/licenses/
+COPY packages/plugins/package.json packages/plugins/
+COPY packages/runtime/package.json packages/runtime/
+COPY packages/trust/package.json packages/trust/
 COPY packages/identity/package.json packages/identity/
 COPY packages/slack-compat/package.json packages/slack-compat/
 COPY packages/sqlite/package.json packages/sqlite/
@@ -35,6 +41,12 @@ COPY packages/entitlements/ packages/entitlements/
 COPY packages/core/ packages/core/
 COPY packages/crypto/ packages/crypto/
 COPY packages/data/ packages/data/
+COPY packages/data-bridge/ packages/data-bridge/
+COPY packages/history/ packages/history/
+COPY packages/licenses/ packages/licenses/
+COPY packages/plugins/ packages/plugins/
+COPY packages/runtime/ packages/runtime/
+COPY packages/trust/ packages/trust/
 COPY packages/identity/ packages/identity/
 COPY packages/slack-compat/ packages/slack-compat/
 COPY packages/sqlite/ packages/sqlite/
@@ -60,6 +72,12 @@ COPY --from=builder /build/packages/entitlements/package.json packages/entitleme
 COPY --from=builder /build/packages/core/package.json packages/core/
 COPY --from=builder /build/packages/crypto/package.json packages/crypto/
 COPY --from=builder /build/packages/data/package.json packages/data/
+COPY --from=builder /build/packages/data-bridge/package.json packages/data-bridge/
+COPY --from=builder /build/packages/history/package.json packages/history/
+COPY --from=builder /build/packages/licenses/package.json packages/licenses/
+COPY --from=builder /build/packages/plugins/package.json packages/plugins/
+COPY --from=builder /build/packages/runtime/package.json packages/runtime/
+COPY --from=builder /build/packages/trust/package.json packages/trust/
 COPY --from=builder /build/packages/identity/package.json packages/identity/
 COPY --from=builder /build/packages/slack-compat/package.json packages/slack-compat/
 COPY --from=builder /build/packages/sqlite/package.json packages/sqlite/
@@ -92,6 +110,12 @@ COPY --from=builder /build/packages/entitlements/dist packages/entitlements/dist
 COPY --from=builder /build/packages/core/dist packages/core/dist/
 COPY --from=builder /build/packages/crypto/dist packages/crypto/dist/
 COPY --from=builder /build/packages/data/dist packages/data/dist/
+COPY --from=builder /build/packages/data-bridge/dist packages/data-bridge/dist/
+COPY --from=builder /build/packages/history/dist packages/history/dist/
+COPY --from=builder /build/packages/licenses/dist packages/licenses/dist/
+COPY --from=builder /build/packages/plugins/dist packages/plugins/dist/
+COPY --from=builder /build/packages/runtime/dist packages/runtime/dist/
+COPY --from=builder /build/packages/trust/dist packages/trust/dist/
 COPY --from=builder /build/packages/identity/dist packages/identity/dist/
 COPY --from=builder /build/packages/slack-compat/dist packages/slack-compat/dist/
 COPY --from=builder /build/packages/sqlite/dist packages/sqlite/dist/

packages/hub/README.md

@@ -195,3 +195,24 @@ wins over the env-generated one. Backup freshness is published on `GET /health`
 ```bash
 pnpm --filter @xnetjs/hub test
 ```
+
+> Note: run tests through the **root** vitest config (`pnpm vitest run
+> packages/hub/test/<file>`) — the per-package filter breaks project
+> resolution.
+
+## Shipping a hub change (the PR tax)
+
+`@xnetjs/hub` is `private: true`, so hub-only changes take **no changeset**
+(confirm with `node scripts/changeset/publishable-pathspec.mjs`). Two things
+ARE required (exploration 0383 W0):
+
+1. **A changelog fragment** whenever behaviour is user-visible:
+   `node scripts/changelog/new.mjs --title "…" --summary "…" --tags platform,sync`
+   (valid tags are `KNOWN_TAGS` in that script). Pure refactors/CI can use the
+   `skip-changelog` PR label instead.
+2. **Expect one `electron-e2e` rerun.** The `xnet://` deep-link case
+   (`electron-smoke.spec.ts:161`) times out flakily and the lane runs
+   `--fail-on-flaky-tests`, so a single timeout reds the PR on identical code.
+   Before debugging, check `git log --oneline HEAD..origin/main` — if your
+   delta is docs-only or unrelated, it is the flake:
+   `gh run rerun <run-id> --failed`.

packages/hub/package.json

@@ -32,6 +32,7 @@
     "@xnetjs/crypto": "workspace:*",
     "@xnetjs/data": "workspace:*",
     "@xnetjs/identity": "workspace:*",
+    "@xnetjs/runtime": "workspace:*",
     "@xnetjs/telemetry": "workspace:*",
     "@xnetjs/slack-compat": "workspace:*",
     "@xnetjs/sync": "workspace:*",