feat(publish): the publishing spine — content-v4 to HTML, feeds, and xnet publish (0362) - #575
Merged
crs48 merged 6 commits intoJul 19, 2026
Conversation
added 6 commits
July 18, 2026 20:19
…ap and head tags The critical path from exploration 0362: no block-to-HTML renderer existed anywhere in the repo. renderPost() walks the Yjs tree directly (no DOM, no BlockNote) and is deterministic so static builds are reproducible. Live embeds degrade through explicit shell/link tiers, both carrying a visible snapshot date rather than implying freshness (0344's honesty label). RSS generalises site/src/lib/blog-feed.ts over a PublishedPost, keeping the Dublin Core dc:creator multi-author byline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: xNet Test <test@xnet.dev>
A post is a Page with editorial metadata rather than a new document type, keeping 0346's substrate claim intact: publishing is a lens over the page, not a fork of it. publishedAt absence is the single source of truth for draft state, and publishedFrontier pins the published version per 0329 so editing the live page does not change what readers see until re-publish. Publication is separately followable (Leaflet's primitive) so a publication survives its authors changing accounts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: xNet Test <test@xnet.dev>
… pinning Publishing assigns a permanent URL and pins the version readers see, so both decisions live in one pure module. Slugs are sticky: once assigned they are never regenerated from a changed title, because the slug is a promise to every inbound link. Re-publishing keeps the original publishedAt and moves only the frontier pin (0329). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: xNet Test <test@xnet.dev>
The BATNA command: emits plain HTML, RSS, sitemap and robots.txt into a directory servable from anything, with no hub and no xNet runtime in the read path. Drafts are excluded by construction and reported, so an author is never left wondering where a post went. Covered by an end-to-end test that renders a real Yjs page, writes the site and serves it over plain HTTP — the BATNA guarantee as an executable assertion rather than a promise. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: xNet Test <test@xnet.dev>
…ment @xnetjs/publish is private so it needs no changeset; data and cli are publishable and both changes are additive, hence minor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: xNet Test <test@xnet.dev>
safeUrl strips control characters by code point rather than a control-character regex class, which is itself a lint hazard. Also records implementation notes on the exploration: what landed (phase 1), what did not, and the one deviation that matters — publishedFrontier is stored but not yet honoured at render time. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: xNet Test <test@xnet.dev>
Contributor
|
Preview removed for PR #575. |
crs48
deleted the
claude/0362-publishing-on-xnet-ghost-substack-and-the-owned
branch
July 19, 2026 03:55
This was referenced Jul 19, 2026
crs48
added a commit
that referenced
this pull request
Jul 19, 2026
… body (0367) (#580) Answers the mechanical questions 0366 deferred: what the index actually contains, how publication is programmed, how flexible it is, and how it scales. ## The index is a lens, not a content type Seven shipped schemas already qualify as publishable — `Page` (now with `slug`/`publishedAt`/`publishedFrontier`), `Publication` (already `followable`), `Post`, `Course`, `Event`, `Space` (kind `community`), `Profile` — plus registry entries and `.xnetpack` bundles. Two of those landed last week in #575 and #576. So "is it plugins or blogs or communities?" resolves to **whatever declares itself publishable**, and the set is non-trivial without inventing a schema. ## The programming model already exists `packages/data/src/schema/lens.ts` ships `SchemaLens { source, target, forward, backward, lossless }` with BFS path composition and builders for rename/remove/convert. It exists to migrate between schema versions — but **an xNet-node → lexicon mapping is structurally the same object with `lossless: false`**. So a schema declares `publish: { lexicon, lens, rkey }` and a runtime outbox does the rest. **No feature code ever imports an atproto client.** That is the direct answer to "can't xNet just be taught to speak atproto so the programming still feels like xNet?" `@xnetjs/publish` already proves the lifecycle: `publishPost()` is pure, pins a frontier, assigns a sticky slug that survives title edits, and preserves the original `publishedAt` across republishes; `unpublishPost()` deliberately keeps the slug so republishing restores the URL. The PDS is simply a second output alongside static HTML. ## The card and the body The record carries the **card** — title, slug, excerpt, author, timestamps, content hash, URL, roughly 1–4 KB. The hub keeps the **body** — the `content-v4` Yjs doc, blobs, comments, grants, history. This split is **forced, not chosen**, for two independent reasons: - `NodePayload` is a sparse per-property CRDT delta with per-key LWW; atproto commits whole CBOR records into an MST. They don't reconcile, so you must project from materialized `NodeState`, never from the change log. - Verified from `packages/pds/src/rate-limits.ts`: **5,000 points/hour per DID, create = 3 points → ~0.46 creates/second**. `applyWrites` *sums* points, so batching buys atomicity, not budget. Publishing a post is comfortable; syncing a document is impossible. ## Narrow indexing is first-party supported Three findings that make the scale story much better than 0365/0366 assumed: - **Jetstream `wantedCollections` accepts NSID prefixes** (max 100 entries) — `net.x.*` is one subscription. `subscribeRepos` has no filtering at all. - **`com.atproto.sync.listReposByCollection`** enumerates every DID holding a collection. Measured live: `sh.tangled.repo` → 4,288 DIDs in **0.9 seconds**. Network-wide discovery is sub-second; there is no crawl. - **Bluesky shipped Tap** (`indigo/cmd/tap`), whose Collection Signal mode auto-discovers and backfills repos using *your* lexicon and POSTs to a webhook. Statusphere's entire ingest became ~60 lines. Beta, but exactly our use case. **Correction to 0366:** Constellation *cannot* do collection-first enumeration — every read path needs a link target as the first key, and it has no backfill mechanism. It remains a valid cost proof (16.7B linking records on a Pi 4b) but not a discovery mechanism. ## Edge cases A 26-row matrix covering the happy paths and the awkward ones: editing after publish, withdrawal, public→private (refused), gated content projection (structurally impossible), whose PDS a community post lives on (the author's — atproto has no org accounts), bulk-importing 500 posts against a 0.46/s budget, users with no atproto identity at all, malformed inbound records, and the firehose echo clobbering local state. That last one is real prior art: **Leaflet hit it in production** — the echo of your own write arrives before your local insert commits and the truncated public record overwrites the fuller local copy. There's a sequence diagram for the round trip. ## Defects found in passing - **`publishedFrontier` is stored but never read** — `render.ts`, `site.ts` and `feed.ts` have zero references, so the frontier pin is decorative and editing a published post changes what readers see immediately. - **`search_index.body` is always `''`** — all three `doc_meta` triggers insert a literal empty string, making trigger-driven indexing title-only. - **Federated RRF is defeated by ordering** — `deduplicateByCid` runs before `reciprocalRankFusion` (`federation.ts:180-181`), so fusion degenerates to a rank-transform of one hub's ordering. - **`registerSchema` is a stub** that ignores its argument, so there's no live third-party schema path to project from. Also flagged: `DID` is hardcoded to `did:key` with runtime prefix checks, so `did:plc` fails validation today; and sharding is term-partitioned on 8 bits of hash, making multi-term cross-shard BM25 unsound. ## Notes - Docs-only; no changeset required. - Branch was synced with `main` first — this worktree was stale and predated #575/#576, so the doc is verified against `main` as merged. - Revenue position unchanged from 0366: reads and listing free; commercial consumers meter on the consumption side. The rule stated explicitly — meter what is expensive to *serve*, never what is valuable to *know*. - CI checks waived by request. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
crs48
added a commit
that referenced
this pull request
Jul 19, 2026
…#578) Closes the gap I flagged in [#575](#575): `publishedFrontier` was **stored but not honoured**, so a published post showed readers the author's latest draft rather than the version they published. ## Two real defects, not just a missing feature **1. The frontier shape was wrong.** I shipped `Record<string, string>`. The repo's actual contract in [`packages/history/src/frontier.ts`](packages/history/src/frontier.ts) is `Record<NodeId, { hash, yjsSnapshotRef? }>`. That second field is the whole point: without it a frontier pins only the **record** lane, so a published post's prose drifts with every keystroke. The pin would have looked correct and done nothing. **2. `frontierEquals` compared entries with `===`.** Once entries became objects that was reference equality, so every frontier reloaded from storage would have reported "edited since publish". Now compares contents. ## What closes it `resolvePublishedDoc()` returns the `Y.Doc` a reader should see — the snapshot named by `publishedFrontier[postId].yjsSnapshotRef`, or the live document when the post pins no document lane. The snapshot store lives in `@xnetjs/history`, which depends on `@xnetjs/data`. Importing it would drag the data layer into a package whose point is running in a bare static build, so **the resolver is injected**. A pin that can't be resolved (pruned past the history horizon) falls back to the live document **with a warning** — rendering current text under an old publication date is a smaller lie than rendering nothing, but it's still a lie, so the caller gets told. ## Proof The D2 guarantee is now an executable test: publish → edit the live doc → the reader still gets the published words; re-publish → the edit becomes visible; and the original `publishedAt` survives. 75 tests in `@xnetjs/publish` (7 new). Validation item **"Editing a published post does not alter the published version until re-published"** is now checked. Exploration status: 13/41 implementation, 5/18 validation — the doc stays `[_]`; phases 2–5 are untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements Phase 1 of exploration 0362 — the critical path, and only that.
Why
Exploration 0362 found that xNet can mark a page
publicbut cannot publish it, and that the blocker was concrete: grepping forblocksToHTMLLossy,ServerBlockNoteEditoror any block→HTML path returned zero hits. Everything else in the doc is downstream of a renderer that did not exist.What landed
packages/publish(@xnetjs/publish) — MIT, depends only onyjs.renderPost():content-v4→ HTML, walking the Yjs tree directly. No DOM, no BlockNote, deterministic.shell/linktiers, both carrying a visible snapshot date rather than implying freshness (0344's honesty label).buildStaticSite().packages/data—PublicationSchemaand publishing fields onPageSchema. A post is a Page with editorial metadata, not a new document type, keeping 0346's substrate claim intact.publishedAtabsence is the single source of truth for draft state.packages/cli—xnet publish static.The BATNA proof
The Charter's BATNA test is asserted, not promised: an end-to-end test renders a real Yjs page, writes the site, serves it over plain HTTP with no xNet process running, and checks the draft 404s. Verified by hand too — output served from
python -m http.serverand read in a browser.74 new tests. Lint 0 errors, typecheck 97/97.
Deliberately not done
Phases 2–5 (anchor tenant, hub SSR, subscribers/ESP, paid reading) are untouched — they need infrastructure decisions and external validators unavailable here. Charter §6 stays Aspirational: nothing here yet gives an author a portable audience.
One deviation worth review:
publishedFrontieris stored but not yet honoured at render time —renderPost()renders the current document, not the document materialised at the pin. That validation item is left unchecked; rendering-at-a-frontier needsmaterializeMultipleAtfrompackages/history. Recorded in the doc's Implementation Notes.🤖 Generated with Claude Code