Self-contained scenario documents (v1): parse, validate, verify, execute exampleBot:greetbot

[trakhimenok]

Summary

Implements https://chatwright.dev/formats/scenario-document/v1 in runtime-ts, ported behaviourally from runtime-go's scenario/ package (chatwright.dev/runtime v0.4.0):

• Parse (document-parse.ts, document-shape.ts, document-secrets.ts) — pure, in-memory, no I/O. Rejects inline secrets, credential-bearing URLs, executable-string members, unknown members, unsupported format/schema/capabilities. Never echoes an offending value; every rejection names a JSON pointer + rule id.
• Validate (document-validate.ts, document-fidelity.ts) — structural rules (budgets, chat/actor references, provider shape, fidelity/environment/data-sensitivity vocabulary, regex subset). One deliberate, declared divergence: bot-transport support is the exact mirror image of Go's (runtime-ts refuses "http" by name, accepts "iframe"; runtime-go is the reverse) — both satisfy unsupported-transport-is-refused-by-name.
• Verify (document-verify.ts) — the independent journal re-check, ported byte-for-byte: UNMET_PREFIX and the "; " join are pinned constants, tested for literal string equality against the Go source.
• Build (document-build.ts, examplebots.ts) — maps a validated document onto an executable Run, narrowly scoped to bot.exampleBot === "greetbot" (the one transport-neutral cross-runtime fixture). Everything else is refused by name, not approximated.

Cassette decision

runtime-ts has no cassette engine (Go's key is sha256(providerConfig + "\x00" + json.Marshal(prompt)) — Go struct field order, not reproducible without inventing a canonical-JSON contract). Chose model-free parity: the shared greetbot document's cassette+replay provider is satisfied by substituting the repository's own deterministic GreetbotProvider policy, recorded in BuiltScenario.providerOverrides — never silently presented as "the cassette ran". Every other provider kind is refused by name.

Gaps (declared, not papered over)

Recorded in docs/runtime-parity.md's new "Part 3" section:

1. Bot-transport mirror image (above) — by design, not a gap.
2. Hand-written unknown-member scan (no TS DisallowUnknownFields equivalent) — reports every violation vs Go's first-only; same accept/reject outcome.
3. Cassette playback — model-free substitution, as above.
4. url-addressed bots (http/iframe) validate but Build refuses them by name — not wired in this task.
5. failurePolicy/ceiling validate identically but are not enforced at execution time — pre-existing runtime-ts run package gap (predates this PR), so ceiling-trip-attributes-run-and-part is unmet in TS today.
6. Bot roster platformIdentity.firstName doesn't carry bot.name in runtime-ts (fixed TelegramCodec identity) — pre-existing Session property, not scenario-document-specific.
7. Live model providers not wired (secret resolution via process.env would only suit a Node host; SecretResolver/EnvOnlySecretResolver are exported ready for a follow-up).

Test plan

• npm run typecheck — clean (see CI)
• npm test — 187/187 passing (52 new: 36 parse/validate parity cases ported from validate_test.go/parse_security_test.go, 6 byte-parity verify cases ported from verify_test.go, 10 conformance cases against a byte-identical copy of runtime-go's own testdata/greetbot-language-onboarding.json)
• Conformance fixture diffed byte-identical against runtime-go/scenario/testdata/greetbot-language-onboarding.json at copy time

⚠️ Studio vendors this package by pinned commit SHA (scripts/vendor-runtime.mjs) — picking this up needs a separate SHA bump in Studio, not done here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SkkrXdtf8mU2GRo2hHsHT1

[Copilot]

Pull request overview

This PR adds first-class support in runtime-ts for the self-contained scenario-document/v1 format: parsing + security scans, structural validation, independent journal verification, and a narrowly-scoped Build path that can execute the shipped exampleBot:greetbot conformance fixture.

Changes:

• Introduces scenario-document/v1 types plus parse/shape/secret scans and structural validation that emit machine-readable issues.
• Adds byte-parity independent journal verification (verify) with conformance tests against the shared greetbot fixture.
• Implements a constrained Build path that executes documents only when bot.exampleBot === "greetbot", recording provider overrides when substituting for cassette replay.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/testkit/greetbot-bot.ts	Updates documentation to reflect greetbot’s role as the shipped exampleBot:greetbot transport.
src/scenario/testdata/greetbot-language-onboarding.json	Adds the shared, portable scenario-document greetbot fixture.
src/scenario/issues.ts	Adds Issue/Report types and rejection error rendering for parser/validator findings.
src/scenario/index.ts	Re-exports new scenario-document modules from the scenario package surface.
src/scenario/greetbot.ts	Aligns greetbot success-criteria prose with the portable document fixture.
src/scenario/examplebots.ts	Adds an exampleBot registry and factory for greetbot.
src/scenario/document.ts	Defines the scenario-document/v1 TypeScript shape and vocabulary types.
src/scenario/document-verify.ts	Implements compiled verify evaluation with ordered expectations and pinned formatting.
src/scenario/document-verify.test.ts	Adds byte-parity tests for verify evaluation and pinned string constants.
src/scenario/document-validate.ts	Adds structural validation rules, including runtime-ts’s transport support divergence.
src/scenario/document-shape.ts	Adds forbidden-member scan and unknown-member rejection via a shape-aware walk.
src/scenario/document-secrets.ts	Adds secret shape checking, secretRef cross-checks, and credential-bearing URL rejection.
src/scenario/document-parse.ts	Implements the pure, in-memory parse pipeline (scans → decode → validate).
src/scenario/document-parse.test.ts	Adds parity/security/AC coverage for parsing and validation behaviour.
src/scenario/document-json.ts	Adds shared JSON-walk helpers (sorted keys, JSON pointer escaping, object guard).
src/scenario/document-fidelity.ts	Adds fidelity vocabulary validation and deterministic resolution heuristics.
src/scenario/document-conformance.test.ts	Adds end-to-end conformance: parse → build → execute → verify for the greetbot fixture.
src/scenario/document-build.ts	Builds an executable Run for the greetbot exampleBot and records provider overrides.
README.md	Updates feature matrix to reflect partial support for portable scenario documents.
docs/runtime-parity.md	Documents scenario-document/v1 support, intentional deviations, and declared parity gaps.

Comments suppressed due to low confidence (1)

src/scenario/issues.ts:91

• This doc comment says ScenarioRejectionError.message never includes any document-supplied value, but it concatenates Issue.message (which can include document values like an unknown member name or unsupported capability). Adjust the comment to match the actual guarantee (e.g. never echoing secret/credential values).

 * The error thrown (returned as `RejectionError` in Go) for a {@link Report}
 * with at least one `SeverityError` issue. `message` renders every
 * error-severity issue as `"<pointer>: <code>: <message>"`, one per line —
 * never the report's warnings, and never any document-supplied value.
 */

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.