Skip to content

docs: typed documentation model + pxf docs build — the doc-pack compiler (#170) - #174

Merged
trendvidia merged 1 commit into
mainfrom
docs/doc-pack-170
Jul 25, 2026
Merged

docs: typed documentation model + pxf docs build — the doc-pack compiler (#170)#174
trendvidia merged 1 commit into
mainfrom
docs/doc-pack-170

Conversation

@trendvidia

Copy link
Copy Markdown
Owner

Layer 1 of the application-documentation platform (#170): the typed documentation model in proto/docs/v1/ and pxf docs build, the compiler that turns authored topics into a doc pack — the documentation analog of the lowered image (#164).

topics/*.pxf ─┐
schema image ─┼─► pxf docs build ─► doc pack ─► bundle publish ─► runtime help/search
registry data ┘                              └─► pxf openapi (#173), static HTML (#171)

Design record: docs/DOC-PACK.md.

The model

Stock proto3 — like schema/v1/report.proto — because runtimes and editors across every port read it and must parse it without a v1.2-capable parser. The structural rules a v1.2 @validate would carry live in the compiler, which is also where #170 asks for the load-bearing ones to live: no authoring tool can decide to skip them.

File Defines
topic.proto TopicFile/Topic keyed by (key, locale) — never the file path; typed prose; five anchor kinds; Review; Translation; Redirect; the Audience taxonomy
pack.proto DocPack, CompiledTopic, ResolvedAnchor, SearchIndex, PackProvenance
registry.proto WidgetCatalog — protowire's typed mirror of the appviewer registry export (appviewer#33)

Prose is typed — blocks and inline runs, never a markdown blob. Every consumer needs structure (the help panel renders it, the HTML exporter emits DOM from it, the indexer walks it), and a renderer that re-parses prose is a renderer that disagrees with its siblings.

Per the binding format rule of 2026-07-25, the only JSON in the pipeline is the appviewer export adapter at the integration boundary. It drops out unchanged when appviewer emits the typed message natively.

Anchor-stability contract

Anchor kinds are classified by how well they survive evolution, because the failure mode is proven rather than hypothetical: protocompile v0.18.0 re-keyed elementPath[fqn#ordinal] and broke exactly this class of persisted reference (trendvidia/protolsp#260).

Stable by construction — schema FQNs (including type aliases from the 50403 carrier), registry widget IDs, topic keys, routes.

Derived and fragile — descriptor paths. DescriptorPathAnchor is therefore authored in element terms (element FQN, annotation FQN, ordinal); the compiler derives the canonical §8.3.1 path through the shared fdp.DescriptorPath formatter, verifies it against the image's own embedded source map (50404), and stamps the resolved anchor with the image digest. A source can never spell a path, so a toolchain that re-keys the grammar invalidates these anchors instead of silently serving stale ones.

A dangling anchor is a compile error. A moved target is a Redirect, whose chains the compiler resolves to their terminus — cycles rejected, kind changes rejected — so no consumer ever walks a chain and none can loop.

The revisor gate

Approval needs all three of:

  1. state == REVIEW_STATE_APPROVED,
  2. a revisor who is not the author — always an error otherwise, since self-approval defeats the gate,
  3. an approved_digest that still matches the content — a bare approved flag is a claim nobody can check.

The content digest covers title, summary and body only, so retagging a topic or re-recording an approval does not churn it. State and digest drift warn while authoring — goed runs this compiler on its diagnostics debounce (goed#321) — and refuse under --release, which additionally requires an explicit audience tier. Translation staleness is a compiler signal on the same terms (--stale-translations-fatal).

pxf docs digest is new relative to the issue, and load-bearing: without it the canonical encoding is knowable only by reading this compiler, so the authoring flow could not produce a digest the gate would accept — and a gate nobody can satisfy is a gate everybody routes around.

Index and determinism

The pack embeds a prebuilt inverted index. Indexing lives in the compiler because it needs parsed content, and the index must be consumable beyond appviewer (goed help, client-side search on fully static hosting, docs-site export) with no search service anywhere in that list. Occurrences record their field class and positions but no ranking — a runtime help panel and a docs site rank differently and both are right — and Tokenization records how the index was built so consumers tokenize queries identically.

Output is byte-stable and timestamp-free. PackProvenance records image and catalog digests, per-source digests, the tool version, and whether release policy applied, so a publisher can check the pack rather than trust the pipeline that handed it over.

Tests

testdata/docs/: a valid corpus exercising every block and anchor kind — anchors resolve against a real image built from testdata/schema-extensions/01_basic.proto, so the tests run the actual lowering pipeline rather than a hand-written descriptor set — plus 13 invalid fixtures each pinned to a specific diagnostic (an unlisted fixture fails the suite) and 3 policy fixtures asserting warn-by-default / fail-at-release. go build, go vet, go test ./... and -race all clean.

Two scoping calls worth review

  • Audience consistency between a topic and a schema element is not enforced. The image carries no visibility tier. That is the schema-side half of the same rule and belongs to cmd/pxf: pxf openapi — OpenAPI boundary renderer over the lowered image + doc pack (RFC-001 #080, M8) #173, which has to settle artifact filtering vs. descriptor stripping regardless. Topic→topic consistency is enforced (a public topic may not anchor or link to a more restricted one).
  • No plain-text Block shortcut. PXF's list syntax for repeated fields makes deep prose verbose to hand-author (blocks = [{ paragraph { runs = [...] } }]). A shortcut would mean two ways to say the same thing and every renderer handling both. This does raise the stakes on goed#321 being the real editor.

CLI-only and additive: no grammar, wire-format, or report change.

Closes #170. Unblocks #171, #173, trendvidia/appviewer#364, trendvidia/goed#321.

…ler (#170)

Layer 1 of the application-documentation platform: the typed doc model
in proto/docs/v1 and the compiler that turns authored topics into a doc
pack — the documentation analog of the lowered image (#164). Consumers
read the pack and nothing else: they never re-resolve anchors,
re-tokenize prose, or re-derive descriptor paths.

Model (stock proto3, like report.proto, so every port and editor parses
it without a v1.2 parser):

- topic.proto: TopicFile/Topic keyed by (key, locale) — never the file
  path; typed prose (blocks + inline runs, never a markdown blob, since
  every consumer needs structure); five anchor kinds; Review;
  Translation; Redirect; the Audience taxonomy, defined once here and
  inherited by every renderer including #173's API filtering
- pack.proto: DocPack, CompiledTopic, ResolvedAnchor, SearchIndex,
  PackProvenance
- registry.proto: WidgetCatalog, protowire's typed mirror of the
  appviewer registry export (appviewer#33)

Per the binding format rule (2026-07-25), the only JSON in the pipeline
is the appviewer export adapter at the integration boundary; it drops
out unchanged when appviewer emits the typed message natively.

Anchor-stability contract. Kinds are classified by how well they survive
evolution, because the failure mode is proven: protocompile v0.18.0
re-keyed elementPath[fqn#ordinal] and broke exactly this class of
persisted reference (trendvidia/protolsp#260). Schema FQNs, widget IDs,
topic keys and routes are stable by construction. Descriptor paths are
derived and fragile, so DescriptorPathAnchor is authored in *element*
terms and the path is re-derived every build through the shared §8.3.1
fdp.DescriptorPath formatter, verified against the image's own source
map (50404), and stamped with the image digest — a source can never
spell a path, so a re-keyed grammar invalidates anchors instead of
silently serving stale ones. A dangling anchor is a compile error; a
moved target is a Redirect whose chains the compiler resolves to their
terminus (cycles rejected, kind changes rejected).

Revisor gate, as compiler policy rather than IDE machinery. Approval
needs all three of: APPROVED state, a revisor who is not the author
(always an error otherwise), and an approved_digest that still matches
the content — a bare approved flag is a claim nobody can check. The
content digest covers title/summary/body only, so retagging or
re-recording an approval does not churn it. State and digest drift warn
while authoring — goed runs this compiler on its diagnostics debounce
(goed#321) — and refuse under --release, which additionally requires an
explicit audience tier. Translation staleness is a compiler signal on
the same terms (--stale-translations-fatal).

pxf docs digest is new relative to the issue and load-bearing: without
it the canonical encoding is knowable only by reading this compiler, so
the authoring flow could not produce a digest the gate would accept, and
a gate nobody can satisfy is a gate everybody routes around.

The pack embeds a prebuilt inverted index — indexing needs parsed
content, and the index must be consumable beyond appviewer (goed help,
client-side search on static hosting, docs-site export) with no search
service anywhere in that list. Occurrences record field class and
positions but no ranking: a help panel and a docs site rank differently
and both are right.

Output is byte-stable and timestamp-free; PackProvenance records image
and catalog digests, per-source digests, and whether release policy
applied, so a publisher can check the pack rather than trust the
pipeline that handed it over.

testdata/docs pins the acceptance bar: a valid corpus exercising every
block and anchor kind against an image built from 01_basic.proto, 13
invalid fixtures each tied to a specific diagnostic (an unlisted fixture
fails the suite), and 3 policy fixtures asserting warn-by-default /
fail-at-release.

Two scoping notes. Audience consistency between a topic and a *schema
element* is not enforced — the image carries no visibility tier; that is
the schema-side half of the same rule and belongs to #173, which must
settle artifact filtering vs. descriptor stripping regardless. And PXF's
list syntax for repeated fields makes deep prose verbose to hand-author;
no plain-text Block shortcut was added, because two ways to say the same
thing means every renderer must handle both.

CLI-only and additive: no grammar, wire-format, or report change.
@trendvidia
trendvidia merged commit fa34662 into main Jul 25, 2026
6 checks passed
@trendvidia
trendvidia deleted the docs/doc-pack-170 branch July 25, 2026 20:38
trendvidia added a commit that referenced this pull request Jul 25, 2026
… (#175)

Resolves the three design gaps GH #93 left open for the OpenAPI
boundary renderer, now that #170 (PR #174) has landed the doc pack it
reads. Decision record: docs/RFC-001-issues.md §#80.

Gap 1 — operation surface. @http gains four defaulted parameters:
summary, operation_id, tags, security (§5.2, annotations.proto). Binding
rules are pinned with them: {name} path segments bind to same-named
top-level request fields; remaining fields bind to the query string for
bodyless methods and to the request body otherwise; operation_id
defaults to <Service>_<Method>, unique by construction; summary falls
back to the first sentence of @description. tags and security take list
literals of strings, because the grammar admits no repeated parameter
type (§5.1 paramType ::= qualifiedIdent) — list-shaped values ride `any`
plus a Literal.list (§8.1), verified against the reference parser before
the text was written. Security-scheme *definitions* stay in generator
config, per the §9.4/#112 argument that keeps deployment topology out of
descriptors.

The alternative — a generator-owned openapi.* library with zero spec
involvement — was considered and not chosen; the cost is recorded rather
than elided: OpenAPI vocabulary now lives in the library every port
mirrors and the draft describes, and it can only grow additively. What
makes it tolerable is that the parameters carry no validation semantics
and impose no port obligation beyond carrying them through the §8.1
carrier.

No `responses` parameter. Responses are derived — success from the
return type, errors from @error_code plus §7 — and authored per-status
entries would need a list of message literals: a shape the carrier
represents (LiteralValue.literal) but the reference parser rejects at an
annotation argument today. Specifying it would put the spec ahead of the
toolchain that can express it.

Gap 2 — audience/visibility tiers. Taxonomy is protowire.docs.v1.Audience
(#170). Artifact filtering only, never descriptor stripping (stripping
has spec implications about image completeness and needs its own pass).
Tiers come from generator configuration, not a new canonical annotation,
because visibility is deployment policy — same reasoning as #112 — with
doc-pack topics contributing their own tier so docs and API surface
cannot disagree. Transitive inconsistency (public reaching internal) is
an error: a dangling $ref or a silently inlined restricted definition
are both worse than refusing.

Gap 3 — availability version. x-since is derived from protoregistry
history, omitted when no registry is configured. No canonical @SInCE is
added: an authored availability claim is unverifiable and nothing would
check it, while the registry already knows.

Additive throughout: the v1.2.0 two-argument @http keeps its meaning and
no existing schema changes shape, pinned by new fixture
21_http_operation.proto, which compiles the bare form beside fully
parameterized uses. No grammar production, extension number, wire, or
report change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: typed documentation model + pxf docs build — the doc-pack compiler, validator, and search indexer

1 participant