Skip to content

feat: carry an audio prompt's transcript text#30

Merged
wavekat-eason merged 1 commit into
mainfrom
feat/prompt-transcript
Jul 20, 2026
Merged

feat: carry an audio prompt's transcript text#30
wavekat-eason merged 1 commit into
mainfrom
feat/prompt-transcript

Conversation

@wavekat-eason

@wavekat-eason wavekat-eason commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Why

Today an audio prompt in a flow document is a bare reference — prompt: { audio: vprompt_… }. The words the clip actually speaks are dropped: they exist where the clip was authored, but nowhere in the document. So a read-only viewer can play a prompt clip yet cannot show what it says — there's no transcript to display alongside playback.

The text already exists upstream (it's what the clip was synthesized from). This change gives the document a place to carry it.

What

Add an optional transcript to the Prompt audio variant — the words the clip was synthesized from, carried for display (what a caller hears) and for traces.

  • Schema (schema/flow.v1.schema.json): Prompt audio object gains an optional transcript. Regenerated the TS model + bundled schema via pnpm gen; the crate-local copy syncs through build.rs, and typify emits Prompt::Audio { audio, transcript: Option<String> } with #[serde(default, skip_serializing_if)] so it round-trips and stays absent when unset.
  • Helpers (model_ext.rs, model.ts): new Prompt::transcript() / promptTranscript() return the spoken words for either prompt kind (a text prompt is its own transcript; an audio prompt returns its transcript, or None when omitted).
    • as_text() / promptText() are unchanged — they still mean "text to synthesize" and return None for an audio prompt. Playback branching and the 2000-char length cap are deliberately untouched; an audio transcript is display text, not something to synthesize or bound.

Why transcript, not text

promptText() deliberately returns null for an audio prompt (it means "text to synthesize"). A field also named text would sit right beside a helper that returns null for it — promptText(p) === null while p.text is a non-empty string. transcript removes that contradiction, matches the accessor name, and reads as the user-facing concept ("the transcript of what the clip says"). It intentionally does not mirror the platform's source_text column name, which describes provenance rather than the viewer-facing meaning.

  • Conformance: a new valid case (audio-prompt-transcript) exercises transcript present on one node and absent on another — additive, always structurally valid.

Compatibility

Fully additive and optional:

  • Older documents (no transcript) stay valid.
  • Refs the platform didn't generate from text simply omit it.
  • Older readers ignore the unknown field (structural tolerance is a documented property of both twins).

Downstream (separate PRs, not in this one)

This is the shared-contract half. To reach the screen it needs:

  1. The platform to emit transcript alongside a generated clip when it produces/freezes the flow document (source text already stored), and pick up the new @wavekat/flow-schema.
  2. The voice daemon to bump wavekat-flow, read prompt.transcript() for each audio prompt, thread it into the flow-detail asset view, and render it next to the play control.

Testing

  • cargo test -p wavekat-flow — 35 unit (incl. new as_text/transcript/YAML round-trip in model_ext) + 3 conformance, green.
  • pnpm --filter @wavekat/flow-schema test — 86 pass (new prompt.test.ts; conformance corpus picks up the new fixture); typecheck clean.
  • cargo fmt --all --check + cargo clippy clean.
  • Re-running pnpm gen produces no drift.

🤖 Generated with Claude Code

An audio prompt was a bare `{ audio: vprompt_… }` ref — the words the
clip speaks lived only where it was authored and were dropped from the
document. So a viewer can play a clip but cannot show what it says.

Add an optional `transcript` to the audio prompt object: the words the
clip was synthesized from, carried for display (what a caller hears) and
for traces. Advisory only — playback still uses `audio`, never this —
and optional, so older documents and non-generated refs stay valid and
older readers ignore it.

- schema: `Prompt` audio variant gains optional `transcript` (regenerated
  the TS model + bundled schema; the crate-local copy syncs via build.rs).
- helpers: new `Prompt::transcript()` / `promptTranscript()` return the
  spoken words for either prompt kind. `as_text()` / `promptText()` are
  unchanged — they still mean "text to synthesize" (audio → None), so
  playback branching and the length cap are untouched. Naming `transcript`
  (not `text`) avoids a footgun: `promptText()` returns None for an audio
  prompt, so a `text` field sitting beside it would read as contradictory.
- conformance: a new valid case exercises the field present and absent.

Lets the platform emit the transcript alongside a generated clip and the
voice daemon display it next to playback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hj93UCb3RKjF6mBwzhsp9f
@wavekat-eason
wavekat-eason force-pushed the feat/prompt-transcript branch from 6b8d121 to 62e8ad8 Compare July 20, 2026 08:26
@wavekat-eason
wavekat-eason marked this pull request as ready for review July 20, 2026 08:32
@wavekat-eason
wavekat-eason merged commit 1e531e4 into main Jul 20, 2026
3 checks passed
@wavekat-eason
wavekat-eason deleted the feat/prompt-transcript branch July 20, 2026 08:33
@github-actions github-actions Bot mentioned this pull request Jul 20, 2026
wavekat-eason added a commit that referenced this pull request Jul 20, 2026
#30 added an optional `transcript` to the audio-prompt model (regenerated
the TS model + bundled schema, added `promptTranscript()`), but the
hand-written safe-subset parser in `parse.ts` was never taught the field.
So `decodePrompt` both warned "unknown field \"transcript\" is ignored by
the engine" and dropped it — `promptTranscript()` returned null for any
audio prompt decoded via `parseFlow`.

Teach `decodePrompt` the field: add it to the known-field list (no more
spurious warning) and carry it onto the returned prompt (validated as a
string, so a non-string transcript is an `expected_string` error). The
Rust twin already carries it — serde ignores unknowns and `Audio` has the
column — so this only closes the TS-side gap.

Surfaced by the platform baking transcripts into published flows: every
audio prompt logged a warning on re-parse.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant