RFC: Completing EmDash comments — from "it works" to best-in-class engagement

[marcusbellamyshaw-cell]

Status: Ideas Discussion (per CONTRIBUTING.md — features need an Ideas Discussion + Approved for PR label before a PR). The Tier 1 branch is built and validated; holding the PR for that label.
AI disclosure: This proposal was researched and drafted with AI assistance (Claude Opus 4.8, model id claude-opus-4-8). All repository claims below were verified against the source by reading the cited files; the competitive survey was AI-assisted research; the Tier 1 prototype was implemented and tested with the same assistance.
Backward compatibility: Every change proposed here is additive — new tables, new opt-in widget props, new optional plugin descriptor fields. No existing migration, route, component prop, or hook signature changes. Forward-only migrations, defaults preserve today's behavior.

---

TL;DR

EmDash already ships a comment system in core — and it's a genuinely good "lightweight self-hosted" foundation (threaded, moderated, Turnstile-protected, hook-extensible, with an AI-moderation plugin already proving the extension model). The gap between what exists and "the best commenting system on the internet" is not a rewrite. It's a focused set of additive core primitives (reactions, commenter notifications, real-time, a public-component slot) plus a plugin ecosystem on the hooks that already exist.

This RFC proposes the dividing line and a tiered, backward-compatible roadmap:

• Core owns the substrate — data model, public ingest, identity, transport, notifications. A plugin literally cannot build these today (plugins get KV/plugin-state, not indexed D1 tables; no real-time binding; no public-component registry).
• Plugins own policy & intelligence — moderation decisions, AI/spam scoring, import adapters, Q&A/membership formats, analytics. This surface is already strong and proven.

One-liner: Best-in-class comments = a richer core comment substrate + a thriving plugin ecosystem on the existing hooks. The line is data/transport/identity → core; policy/intelligence/presentation → plugin.

---

Demo — working prototype (Tier 1: reactions + "Best" sort)

To de-risk the proposal rather than just describe it, I've already built and validated Tier 1 (reactions + a Reddit-style Wilson "Best" sort) end-to-end against demos/simple. The branch is ready to PR the moment this is Approved for PR.

Before — comments today (chronological, no reactions):

After — opt-in <Comments reactions sort="best" />: one-tap likes with live counts, and the most-liked comment rises to the top (Wilson lower-bound, so a couple of likes can't outrank a heavily-liked comment); replies stay chronological:

What landed in the prototype (all additive, behavior-preserving defaults):

• New _emdash_comment_reactions table (migration 042), deduped per voter by a salted IP hash — the same privacy primitive as comment ip_hash.
• A public, honeypot- + rate-limited endpoint GET/POST /_emdash/api/comments/:collection/:contentId/reactions, registered in the Astro integration (verified live against the demo: toggle on/off and aggregate counts return the expected JSON).
• <Comments> gains two opt-in props — reactions (like button + live counts) and sort="best". Posting is progressively enhanced by a tiny inline script that is emitted only when reactions are enabled, so non-reaction pages ship zero extra JS. Counts and ordering are server-rendered (SEO + first paint); the client script only handles toggling.

Validation: lint clean (0 diagnostics), typecheck clean, 118 tests passing — 15 new (Wilson math, toggle/dedupe, counts, rate-limit, best-sort ordering) plus the existing comment/migration/client/API suites. Includes a changeset (emdash: minor).

This is offered as a concrete first step (§5, Tier 1) — happy to adjust scope/shape based on maintainer feedback before opening the PR.

---

1. Motivation

One of EmDash's stated reasons to exist is to beat WordPress. WordPress and Disqus — the two defaults of the open web — share the same failure: a slow, low-engagement, spam-prone commenting experience. Comments are where social engagement and time-on-page are won or lost, and both incumbents leave that value on the table:

• Disqus delivers the engagement features (portable identity, reply notifications, threading, reactions) but at a ruinous cost: ~1.5 MB of third-party JS, 50–100+ third-party requests on a page that had ~10, behavioral tracking + data sale as the actual business model, ad/affiliate injection, comments hidden in an un-indexable iframe, and wholesale blocking by privacy extensions. Publishers who left Disqus have reported engagement jumping (one widely-cited WPBeginner figure: +304% comments after dropping it).
• WordPress native owns the data but is a documented spam magnet, has no real-time, weak commenter-to-commenter notifications, dated full-page-reload UX, and moderation trapped in a slow dashboard — survivable only by bolting on a paid stack (Akismet + wpDiscuz + Thrive + an SMTP plugin + a cache plugin).

EmDash's architecture is purpose-built to beat both: first-party on the publisher's own Cloudflare zone, data in their own D1, SSR for SEO and speed, Workers/Turnstile/Workers-AI/Email at the edge. It can deliver Disqus's engagement loop with WordPress's data ownership and none of the tracking/perf/SEO baggage. The foundation already does part of this. This RFC is about finishing the job.

---

2. What EmDash already has (verified against source)

This framing matters: we are completing a system, not proposing a new one. Confirmed by reading the repo:

Data model — _emdash_comments (packages/core/src/database/migrations/027_comments.ts):

• Threading via self-referential parent_id (ON DELETE CASCADE).
• Anonymous or account-linked authors: author_name, author_email, author_url, author_user_id → users.id (ON DELETE SET NULL).
• Moderation status (default pending), ip_hash (privacy), user_agent, moderation_metadata (JSON).
• Indexes on (collection, content_id, status), parent_id, (status, created_at), author_email, author_user_id.
• Per-collection settings: comments_enabled, comments_moderation (default first_time = auto-approve returning commenters — the Isso trust pattern), comments_auto_approve_users (default on), comments_closed_after_days (default 90).

Rendering — Comments.astro + CommentForm.astro (emdash/ui):

• Fully server-rendered (SEO + fast first paint), 2-line drop-in, semantic <ol>/<article> with ec--prefixed classes; threaded prop.
• Posting hydrates via a tiny inline script (delegated submit listener) — not a heavy framework island.

Ingest + anti-spam — POST /_emdash/api/comments/:collection/:contentId:

• X-EmDash-Request: 1 CSRF header, honeypot field, IP-hash rate limiting, optional Cloudflare Turnstile (turnstileSiteKey prop on CommentForm).

Moderation pipeline — packages/core/src/plugins/hooks.ts + comments/moderator.ts:

• comment:beforeCreate (mutate or reject), exclusive comment:moderate (one provider returns ModerationDecision = { status: "approved" | "pending" | "spam", reason? }), comment:afterCreate (fire-and-forget, carries content + contentAuthor), comment:afterModerate.
• An admin moderation UI + API (api/admin/comments/*).
• @emdash-cms/plugin-ai-moderation already exists and claims comment:moderate exclusively, running Cloudflare Workers AI (Llama Guard). The plugin extension model for comments is real and shipping.

Notifications — comments/notifications.ts: emails the content author when a comment is approved.

This is roughly "good lightweight self-hosted tier (Isso/Remark42) + AI moderation." Solid. Not yet best-in-class.

---

3. The gap to best-in-class

From a survey of Disqus, Coral (Vox), Hyvor/Commento, Remark42/Cusdis/Isso, Giscus, WordPress, Substack/Ghost, and Reddit/HN/Discourse, "best" sorts into four layers. Legend: ✅ exists · ⚠️ partial · ❌ missing (all statuses verified against source).

Layer 1 — Friction-killers (table stakes)

Capability	Status
Comment with name+email, no account; auto-approve returning + logged-in users	✅
SSR + progressive enhancement (anti-Disqus on Core Web Vitals)	✅
Turnstile + honeypot + IP rate-limit	✅
Rich text / markdown body (sanitized)	❌ plain text only (escapeHtml + auto-link URLs, white-space: pre-wrap)
Persistent anonymous visitor identity → edit/delete-your-own, "notify me" without an account	❌
Import from Disqus / WordPress (kills switching cost)	❌

Layer 2 — The engagement flywheel (the actual goal: time-on-page + return visits)

Capability	Status
Reply notifications to commenters / thread subscriptions	❌ author-only today — the single highest-leverage retention mechanic, missing for the people who matter most
Reactions / likes / votes (one-tap participation for lurkers)	❌ no reactions/votes table exists
Sort-by-Best (Reddit Wilson lower-bound) / Top / New	❌ (needs vote data)
Real-time live updates ("X new replies", live thread)	❌ polling only — no WebSocket/SSE/Durable Object path
Web Push, @mentions, featured/pinned comments	❌

Layer 3 — Trust & safety

Capability	Status
Hook-based moderation, exclusive provider, AI toxicity, Turnstile	✅ (Turnstile + plugin-ai-moderation)
Graduated sanctions (mute/timeout/ban), Banned-vs-Suspect word filters, shadowban, trust-weighted community flagging	⚠️ partial / plugin-extensible
Design rule (industry consensus): AI scores sort the queue or nudge before posting — never silent auto-reject (Coral)	💡 adopt as norm

Layer 4 — Moonshots / differentiators

Capability	Status
Q&A / AMA mode (Coral) — comments as a programmable engagement format	❌
Membership-gated commenting as a conversion CTA (Substack/Ghost flywheel)	❌
Lightweight, privacy-respecting trust levels (Discourse, minus heavy telemetry)	❌
Cross-site "active/trending discussions" module (HN gravity decay)	❌

---

4. Core vs plugin — the architectural decision

The honest answer is hybrid, and the dividing line is precise and evidence-based.

Why some of this must be core (a plugin cannot build it today)

Verified plugin limits:

• Storage: plugins get KV + plugin-state only (ctx.kv, _plugin_state) — not their own indexed D1 tables/migrations. Reactions/votes need an indexed relational table to sort-by-best at scale; that belongs in the core schema.
• No real-time transport: plugins cannot bind a Durable Object / WebSocket / SSE channel. Live threads + browser reply-notifications need a core primitive.
• No visitor-identity primitive: there is no persistent anonymous token, edit/delete token, or optional social/magic-link identity for non-CMS visitors.
• No commenter notification store: afterCreate can trigger mail, but the subscription table + unsubscribe tokens + identity are core concerns.
• No public-component registry / render-slots: a plugin can only inject raw HTML/script strings (page:fragments) or Portable-Text block components. It cannot ship a typed, prop-driven SSR-then-hydrate Astro island that auto-appears under an article. The "paste one line and a rich widget appears" delivery needs a small core extension point. (WidgetRenderer.astro's componentMap is also hardcoded to core widgets.)

Why the rest should be a plugin (and already can be)

Verified plugin capabilities:

• Public, unauthenticated routes: PluginRoute.public?: boolean — "Mark this route as publicly accessible (no authentication required). Public routes skip session/token auth and CSRF checks." So a reactions endpoint, import adapter, or webhook is plugin-doable.
• The full comment hook pipeline + exclusive-provider selection (proven by plugin-ai-moderation).
• KV storage, admin pages/widgets, cron, Workers AI, email, page:fragments, Portable-Text block components, page:metadata (schema.org Comment/Discussion JSON-LD).

So: AI/spam moderation, analytics, badges, import-from-Disqus, Q&A-mode UI, membership gating, alternative themes → plugins.

---

5. Proposed roadmap (tiered, additive, independently shippable)

Each tier is a separate Discussion-able unit; ROI descends down the list. Data sketches follow EmDash conventions (text/ULID PKs, integer booleans, idx_{table}_{col}, register migration in runner.ts, never interpolate into SQL).

Tier 1 — Reactions + Sort-by-Best (core; highest ROI, cleanest PR)

The single highest-ROI engagement primitive: one-tap participation for the silent majority, plus the signal needed to surface quality.

• New core table _emdash_comment_reactions (next migration 042_*): id, comment_id → _emdash_comments.id (cascade), reaction (text; default set like, extensible), voter_hash (IP+UA hash or visitor token, for dedupe), created_at; unique index on (comment_id, voter_hash, reaction); index on comment_id.
• Ingest: a public, rate-limited, Turnstile-gated POST (mirrors the existing comment ingest). Could ship as a core route or — to validate the pattern — as a first-party plugin using public: true routes, with the table the one piece that needs core.
• Ranking: store up/down (or positive-only "Respect", per Coral's anti-pile-on stance — recommend positive-only default), compute Wilson lower-bound for a Best sort; offer Top/New alternates. Cheap to store/recompute per reaction.
• Widget: new opt-in props on Comments.astro (reactions, sort) — default off → zero behavior change.
• Backward-compat: new table + new optional props. Nothing existing changes.

Tier 2 — The retention loop: commenter notifications + lightweight visitor identity (core)

The mechanic every system in the survey ranks #1 for repeat visits — and the one EmDash is missing for commenters.

• Visitor identity primitive: a persistent, signed anonymous token (cookie) → enables edit/delete-your-own within a window and notify-me without forcing an account. Optional magic-link/social upgrade for a verified badge (core already renders an isRegisteredUser ✓ badge).
• Subscriptions: _emdash_comment_subscriptions (content_id, subscriber_email or visitor token, scope thread|content, signed unsubscribe_token, created_at).
• Delivery: reuse the existing comment:afterCreate event (it already carries content + thread context) to email subscribers via the existing email pipeline; honor per-subscriber unsubscribe. (Generalizes notifications.ts beyond author-only.)
• Backward-compat: new tables, new opt-in form checkbox ("notify me of replies"). Author notifications unchanged.

Tier 3 — Real-time + comment-aware caching (core)

Makes threads feel alive and fixes a correctness gap.

• Live updates: an optional Durable Object (or SSE Worker route) for "new reply" delivery + in-browser notifications, with graceful polling fallback for Node/self-host.
• Caching: today a server-cached article page (Astro.cache.set(cacheHint)) serves a stale comment list. Add a comment-aware cache boundary / hole-punch so new approved comments invalidate correctly.
• Backward-compat: real-time is opt-in; cache fix is a correctness improvement with no API change.

Tier 4 — Embed ergonomics: public-component registry / render-slots (core; cross-cutting)

A general extension point that unlocks one-line rich embeds for comments and benefits other plugins.

• A definePublicComponent / named <EmDashSlot name="after-content" /> mechanism, mirroring the existing virtual:emdash/block-components generator with a new virtual:emdash/public-components module collected from a new optional descriptor field (e.g. publicComponentsEntry).
• Optionally wire WidgetRenderer.astro to resolve component-type widgets from plugin registrations instead of the hardcoded map.
• Backward-compat: purely additive registry; no existing component changes.

🔗 Shared primitive with the white-label RFC. This same "plugins/config can drop a typed, SSR-then-hydrate component into a named slot" extension point is what the companion proposal — First-class white-labeling for the EmDash admin & login (emdash-whitelabel-login-admin-proposal.md, Discussion #1493) — needs for its component-override tier. These are not two separate asks: a single public-component / render-slot registry serves both the public-page surface (comments and other plugin widgets) and the admin/login surface (branding and component overrides). Maintainers should evaluate it once, as one foundational extension point. If accepted there first, this tier reuses it; if accepted here first, that proposal reuses it.

Tier 5 — Content quality: rich text + deeper threading (core)

• Markdown/rich body with strict server-side sanitization (allowlist) + lazy syntax highlighting (Giscus's lesson; today body is plain text). New opt-in render mode.
• Threading depth + collapse: today threading is exactly one level (comments → replies). Add configurable depth, subtree collapse with child-count badges, and "continue thread" lazy loading (Reddit pattern) — render shallow, lazy-load deep.

Plugin lane — no core changes required

Buildable today on hooks + public: true routes + KV: import-from-Disqus/WordPress, Q&A/AMA mode, membership-gated commenting, trust levels, advanced moderation (Banned-vs-Suspect word lists, shadowban, trust-weighted flagging), analytics widgets.

---

6. Design constraints (the "do-not" list, drawn from the cautionary tales)

• Zero third-party requests; tiny first-party payload. Publish and CI-enforce an embed-size budget. (Disqus = ~1.5 MB; the bar is single-digit KB over the existing SSR HTML.)
• No tracking, no ads, no data sale, no affiliate injection. First-party on the publisher's own zone — no consent banner needed.
• Render in-page (SSR), never an iframe — preserve SEO + Core Web Vitals (Disqus/Ghost lose here).
• AI never silently auto-rejects — nudge-before-post and queue-sort only (Coral).
• No public downvote pile-ons / opaque shadow-moderation as default — prefer positive-only reactions + transparent, appealable moderation.
• Reliable transactional email (SPF/DKIM/DMARC via Cloudflare Email) — WordPress's worst re-engagement gap.
• Signed, expiring tokens for any email-link moderation / unsubscribe / edit capability URL.

---

7. Backward compatibility & process

• Additive only. New tables (042+), new optional widget props, new optional descriptor fields, new opt-in routes. No change to existing migrations, the _emdash_comments schema, the comment hook signatures, or Comments.astro/CommentForm.astro defaults.
• Forward-only migrations; defaults preserve current behavior (reactions off, notifications author-only unless a subscriber opts in, real-time off, plain-text body unless rich mode enabled).
• Per CONTRIBUTING.md: this is a feature set → Ideas Discussion + Approved for PR before any PR; each tier lands as its own PR with a changeset (minor), tests-first, no messages.po churn, AI disclosure on the PR.

8. Open questions for maintainers

1. Appetite for reactions/votes in core vs. a first-party plugin + a single core table? (The table is the only hard core dependency.)
2. Positive-only "Respect" reaction (Coral) vs. up/down voting as the default?
3. Is a public-component registry (Tier 4) something core wants generally, or should comment widgets stay author-hand-wired?
4. Real-time via Durable Objects — acceptable as an optional dependency with a polling fallback for Node/self-host?
5. Preferred sequencing — is Tier 1 (reactions) the right first PR to validate the pattern?

---

Prepared for the EmDash community. Repo claims verified against source at the time of writing; competitive analysis AI-assisted (claude-opus-4-8).

[ascorbic]

I like this. I'd be open to have the central parts in core. I do think reactions should be opt-in though. I would say upvote only for now. I think the cmponent registry ideas are scope creep and shouldn't be included here. I don't think real-time is worth the extra complexity.

I'll approve reactions for PR. The dsecription in the tier 1 section looks good. Thanks!

[marcusbellamyshaw-cell]

Thanks for the quick feedback! Confirmed on all points: reactions will be strictly opt-in (the reactions prop on <Comments> defaults to off, so existing installs are unaffected), upvote-only to start, component registry dropped from this RFC entirely, and real-time out of scope.

I'll open the Tier 1 PR — reactions + Wilson "Best" sort — against the branch I already have. The changeset is already there (emdash: minor). I'll drop the Tier 4 section from the PR description so the scope is unambiguous.

Tiers 2–3 (commenter notifications, cache fix) I'll bring back as separate discussions when the time feels right — no pressure to commit to those now.