You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Phase 2 of the firehose rework (after #7). Adds the follows hose (kind 3) and retires the legacy raw-upsert path for kind 3. First real exercise of the HOSES switch (HOSES=profiles,follows).
Background
The follows collection uses a derived shape — { pubkey, follows: [hex…], created_at, count } — not raw events. That matters: the in-degree query (followerCount → countDocuments({ follows: pubkey })) and enrichCounts both depend on the top-level follows array + count. The legacy upsertEvent fallback (current phase-1 path for kind 3) writes the raw event instead, which has no follows array — so follower counts silently don't accrue for events it writes. Moving kind 3 to a proper hose fixes that.
The old followshose.js also harvested relay URLs from the kind-3 content (legacy NIP-65 relay map, {"wss://…":{read,write}}) into the relays collection. We keep that discovery side-effect.
Scope
src/hoses/event.js: extract the shared structural + schnorr-signature check (eventId, verifySignature) used by every hose; refactor the profiles hose to use it (DRY).
src/hoses/follows.js (kind 3):
Verify the event signature (kind 3 content is optional — unlike kind 0, it is not required to be JSON).
Derive { pubkey, follows: [hex…], created_at, count } from p tags (validate 64-hex, dedupe).
Latest-wins upsert (the old followshose blindly replaced — an out-of-order older event could clobber a newer list; we gate on created_at).
Owns its indexes: pubkey, follows (reverse lookup), created_at.
Relay harvesting: parse content as a relay map, canonicalize URLs (trailing slash on origins; ws/wss only), $setOnInsert into the relays directory so discovery never overwrites health metrics. Best-effort — never fails ingest.
src/indexer.js: register the follows hose. The switch then removes kind 3 from the legacy set automatically; HOSES=profiles,follows becomes a meaningful selection.
Phase 2 of the firehose rework (after #7). Adds the follows hose (kind 3) and retires the legacy raw-upsert path for kind 3. First real exercise of the
HOSESswitch (HOSES=profiles,follows).Background
The
followscollection uses a derived shape —{ pubkey, follows: [hex…], created_at, count }— not raw events. That matters: the in-degree query (followerCount→countDocuments({ follows: pubkey })) andenrichCountsboth depend on the top-levelfollowsarray +count. The legacyupsertEventfallback (current phase-1 path for kind 3) writes the raw event instead, which has nofollowsarray — so follower counts silently don't accrue for events it writes. Moving kind 3 to a proper hose fixes that.The old
followshose.jsalso harvested relay URLs from the kind-3 content (legacy NIP-65 relay map,{"wss://…":{read,write}}) into therelayscollection. We keep that discovery side-effect.Scope
src/hoses/event.js: extract the shared structural + schnorr-signature check (eventId,verifySignature) used by every hose; refactor the profiles hose to use it (DRY).src/hoses/follows.js(kind 3):{ pubkey, follows: [hex…], created_at, count }fromptags (validate 64-hex, dedupe).created_at).pubkey,follows(reverse lookup),created_at.contentas a relay map, canonicalize URLs (trailing slash on origins; ws/wss only),$setOnInsertinto therelaysdirectory so discovery never overwrites health metrics. Best-effort — never fails ingest.src/indexer.js: register the follows hose. The switch then removes kind 3 from the legacy set automatically;HOSES=profiles,followsbecomes a meaningful selection.planIngesttwo-hose cases already exist from Firehose phase 1: validated profiles hose (kind 0) + extensible hose seam #7.Out of scope (later phases)
Relay-lists hose (kind 10002) · relay-health prober · deploy/retire legacy server hoses.
Acceptance
npm testgreen incl. new follows-hose tests.relayswithout clobbering existing health docs.