RFC: bulk historical backfill for archive imports, and an `imported_from` provenance convention

[Ferchmin]

RFC: bulk historical backfill for archive imports, and an imported_from provenance convention

Context

I'm building an open-source importer that moves Slack workspace history into a
Buzz community (public channels, threads, reactions, files). It is explicitly an
archive importer, not an identity migration — Buzz has no email primitive to
match identities on, so history cannot be imported "as" the original person.
Messages are published under deterministic, clearly-labelled per-user archive
keys, and the tool says so loudly.

Searched first, per CONTRIBUTING. Closest existing work:

• feat(integrations): reference Slack Connect channel bridge #2822 (Slack Connect channel bridge) — a live forward bridge, not history import.
• Snapshot import has no update path: reimport duplicates, and the team it created cannot be deleted #3085 (snapshot import) — .team.json agent/persona snapshots, not messages.
• RFC: Stable extension points for downstream integrations and distributions #2754 (RFC: stable extension points for downstream integrations) — same class
  of boundary question; this is a concrete instance of it.

I found no existing message-history backfill support. If something is in flight
that I missed, point me at it and I'll drop this.

The requirement

To import history at all, events need created_at set from the original Slack
timestamp. Everything else about the import (kind:9 messages with h tags,
kind:7 reactions, NIP-10 threading, kind:9007 channel creation) works today
through POST /events with no relay changes at all. Timestamps are the only
blocker.

Why I am not proposing a scoped POST /events capability

My first instinct was an authenticated import role permitted to publish past
created_at. Reading the code, I don't think that can exist without breaking a
correctness property, and I'd rather say so up front than send a PR that has to
be explained away.

There are two layers:

1. Ingest envelope — MAX_TIMESTAMP_DRIFT_SECS = 900 in
   crates/buzz-relay/src/handlers/ingest.rs. Rejects
   |created_at - now| > 900. Gating this on a scope would be easy.

2. Commit-time floor — migrations/0021_created_at_fence_floor.sql plus
   crates/buzz-db/src/replica_fence.rs. A DEFERRABLE INITIALLY DEFERRED
   constraint trigger aborts, at COMMIT, any insert of a channel-bearing
   events row with created_at < clock_timestamp() - 960s. The relay's writer
   pool arms it on every connection via the buzz.created_at_floor GUC
   (crates/buzz-db/src/lib.rs, connect_pool / after_connect).

Layer 2 is what makes the replica-freshness fence provable: cursor pages are
served from a replica only for rows older than
min(oldest_xact_start, S) - floor - margin. A backfilled channel row that
committed below the fence would make replica-served keyset pages silently
incomplete — holes, not staleness.

So relaxing layer 1 alone means every backfilled message aborts at COMMIT. And
relaxing layer 2 for import writes removes the invariant the read-routing proof
rests on. That isn't a scoped capability; it's dropping a guarantee.

Migration 0021's own comment already prescribes the alternative:

any operational backfill of channel rows must run on a connection without the
GUC — i.e. outside the relay's writer pool — and the operator must hold the
replica breaker closed from before the backfill transaction begins until its
WAL is replayed on the replica

Which reads to me as: historical backfill is an operator-plane operation in
Buzz, and no client-facing endpoint should be able to do it. If that's the
intended reading, I'd like to confirm it and then build against it.

Proposal — two separable pieces

(a) imported_from provenance convention — docs only, ready now

A standard tag identifying an event as imported and where from:

["imported_from", "slack", "<team_id>/<channel_id>/<slack_ts>"]

Three properties worth having regardless of (b):

• Already works. events.tags is JSONB with no allowlist, so this passes
  ingest unchanged today.
• Already indexed. idx_events_tags_gin (migration 0004, jsonb_path_ops)
  makes tags @> '[["imported_from","slack"]]' an index probe, so importers get
  cheap idempotency and clients get a cheap "this is archived content" signal.
• Zero new surface. No schema change, no relay code, no new kind.

The value is purely that it's standard — #2822's bridge, this importer, and any
future Discord/Teams/Mattermost importer should agree on one spelling, and
clients should be able to render an "imported" affordance without special-casing
each tool.

I have a docs/nips/NIP-IM.md written in the style of NIP-IA, covering the tag
shape, the third-element key format, client rendering guidance, and explicit
non-goals (does not assert authorship, does not imply the archive key is the
original person). Happy to open that PR immediately — just say go. Holding
it because CONTRIBUTING asks for discussion before new conventions.

(b) Operator backfill path — needs your direction

Sketch, respecting the fence rather than working around it:

• A buzz-admin import subcommand (or a separate buzz-backfill binary) that
  writes through its own pool with the floor GUC unset, deliberately outside
  the relay's writer pool.
• An explicit operator hold that forces the replica fence closed for the
  duration and reopens it once the backfill's WAL is replayed. ReplicaFence
  has a CLOSED sentinel but, as far as I can tell, no operator-settable hold —
  that's the one piece of genuinely new relay surface this needs.
• Reuse of ingest's validators, since this path bypasses ingest entirely
  (signature verification, content bounds, kind/scope rules, h-tag channel
  resolution). Duplicating them would rot; extracting them is the real work.

Open questions I'd rather you answer than guess at:

1. Is operator-plane-only the intended reading of 0021, or is there appetite for
   a narrower in-band mechanism I've missed?
2. Should backfill be a buzz-admin subcommand, a separate binary, or out of
   scope for this repo entirely (i.e. importers are expected to talk to Postgres
   themselves under documented rules)?
3. Would you want the extracted ingest validators as a reusable seam
   (buzz-relay::handlers::ingest::validate-shaped), independent of backfill?
4. Is a fence operator-hold API something you'd accept, or does the fence need
   to stay purely probe-driven?

Known consequence, either way

Buzz's audit log is hash-chained with prev_hash per community
(crates/buzz-audit/src/entry.rs). Backfilled events chain in import order,
not chronological order, so anything reading that chain as a timeline will
disagree with message dates. That seems inherent to backfilling into an
append-only chain — I plan to document it rather than paper over it, but flagging
in case it changes your view of (b).

What I'm not asking for

Not asking anyone to build this. I'm happy to do the work on (b) if the shape is
agreed; mainly I need to know whether the direction is one you'd merge before I
build it, per CONTRIBUTING's "open an issue first" guidance.