RelationalText

Rich text as a relation. A document is a string and a table of typed annotations over byte ranges — nothing more. Every format is a view. Conversions are morphisms.

---

The model

A RelationalText document has two parts: plain text and a flat list of facets, each covering a byte range and carrying one or more typed features:

{
  "text": "\uFFFCHello, world.",
  "facets": [
    {
      "index": { "byteStart": 0, "byteEnd": 3 },
      "features": [{ "$type": "org.relationaltext.richtext.block", "name": "paragraph", "parents": [] }]
    },
    {
      "index": { "byteStart": 8, "byteEnd": 13 },
      "features": [{ "$type": "org.relationaltext.richtext.mark", "name": "bold", "expandStart": true, "expandEnd": true }]
    }
  ]
}

Feature types are AT Protocol lexicon IDs — namespaced strings like org.relationaltext.richtext.mark or app.bsky.richtext.facet#mention. Any application can define its own feature types; unknown types are preserved, not dropped.

The text field is always valid as plain text on its own. The facets are the overlay.

---

Conversion engine: panproto

Cross-format conversion is powered by panproto, a schema-driven conversion engine based on generalized algebraic theories. The conversion pipeline:

1. Each format adapter (WASM binary) parses raw text into a Document with format-specific facets
2. Protolens chain — panproto auto-discovers the structural alignment between source and target schemas, builds a lens from the schema diff
3. panproto_lens::get — applies the lens via wtype_restrict with vertex remap, field transforms, and complement tracking
4. Value-dependent rules (matchAttrs, template names) are applied post-restrict

Lenses compose (A→B + B→C = A→C) and invert automatically when lossless. A lens graph finds the shortest conversion path between any two registered formats.

---

Layers: annotation substrate

Semantic annotations (beyond text formatting) use the Layers annotation model (pub.layers.annotation). A LayeredDocument wraps a document with typed annotation layers:

import { LayeredDocument } from 'relational-text/layered-document'

const layered = LayeredDocument.fromDocument(doc)
const withNER = layered.addLayer({
  expression: 'doc',
  kind: 'span',
  subkind: 'entity-mention',
  annotations: [
    { uuid: { value: '...' }, anchor: { textSpan: { byteStart: 0, byteEnd: 5 } }, label: 'PER' }
  ],
  createdAt: new Date().toISOString(),
})

Annotation layers support:

• Span, token-tag, relation, tree, graph, tier, and document-tag kinds
• Knowledge references (Wikidata, FrameNet, custom sources)
• Ontology type references with role slots
• Discontiguous spans via additionalSpans
• Temporal and spatial anchoring

---

Format support

All 38 formats use panproto's protolens pipeline for cross-format conversion.

Category	Formats
Markup	CommonMark, GFM, HTML
JSON editors	Quill Delta, ProseMirror, TipTap, Lexical, Slate, Contentful, Sanity, Notion
Social / messaging	Bluesky, Mastodon, Slack, Discord, Telegram, WhatsApp, LinkedIn, Threads
Documents	BBCode, Org-mode, OPML, Apple News Format
Notebooks / PKM	Jupyter, Obsidian, Roam, Logseq
Markdown variants	MultiMarkdown, Pandoc, GitLab Flavored, MDX, MyST, Markdoc
Wiki / CMS	Confluence, JIRA, DokuWiki, MediaWiki, Textile
Specialized	Fountain (screenplay)

---

Architecture

crates/
  relationaltext-core/      document model, HIR, lens engine, Layers data model,
                             panproto bridge, position tracking
  relationaltext-wasm/       WASM bindings (document ops + lens + LayeredDocument)
  relationaltext-sqlite/     SQLite extension for facet queries
  relationaltext-format-adapters/  per-format WASM import/export adapters
formats/
  org.w3c.html/              lexicon, lenses, WASM adapter
  org.commonmark/            lexicon, lenses, shared markdown WASM adapter
  com.recipe/                recipe annotation system (Layers-based)
  ...                        38 format directories, each self-contained
packages/
  relational-text/           TypeScript SDK (WASM-backed)
    src/
      core.ts                Document class, lexicon registration
      lens.ts                LensSpec, applyLens, LensGraph
      registry.ts            async from()/to() dispatcher via lens graph
      layered-document.ts    LayeredDocument class (Layers annotations)
      layers.ts              RT ↔ Layers conversion via panproto
      knowledge.ts           KnowledgeResolver (Wikidata, custom sources)
      annotation-overlay.ts  Framework-agnostic annotation range computation
      concept-index.ts       Cross-document concept linking
apps/
  demo/                      Next.js demo with 38-format editor + recipe annotator

The Rust core has no built-in knowledge of any specific format. Everything is registered at runtime from lexicon JSON. Cross-format conversion goes through panproto's protolens pipeline (Rust/WASM).

---

Usage

import { from, to } from 'relational-text/registry'

const doc = await from('markdown', '# Hello\n\nThis is **bold** text.')
const html = await to('html', doc)
// → <h1>Hello</h1><p>This is <strong>bold</strong> text.</p>

import { LayeredDocument } from 'relational-text/layered-document'
import { KnowledgeResolver, createDefaultResolver } from 'relational-text/knowledge'

// Annotations with knowledge references
const resolver = createDefaultResolver() // built-in Wikidata support
const entity = await resolver.resolve({ source: 'wikidata', identifier: 'Q14806' })

---

Getting started

Prerequisites: Rust, wasm-pack, Node.js 20+, and pnpm 10+.

pnpm install
pnpm build        # builds WASM + TypeScript
pnpm test         # runs the full test suite (~2355 tests)

Demo app

An interactive demo with TipTap and Lexical editor panes, live conversion across 38 formats, and a recipe annotation demo with Layers-based annotation editor:

pnpm demo         # starts Next.js dev server at http://localhost:3000

---

License

Apache-2.0 — see LICENSE.