feat: support transcription replay and API key validation

[jpr5]

Final Timing Contract Coverage

Commit b5ce7df is test coverage only; it does not change replay behavior. The fake-clock writer test now proves no frame arrives one millisecond before each scheduled 40/20 ms boundary, each expected frame arrives at its boundary in order, and the stream stays open until the final frame. The route test spies on the real writer boundary and asserts the fixture forwards its exact recordedTimings object and replaySpeed: 2.

Focused timing tests passed 20/20 repeated runs. Full pre-push validation passed: format, lint, TypeScript, 4,794 tests (3 skipped), build, and export checks.

Timing Determinism Correction

Commit 4b6cedb fixes a completion-gate flake in the replay-speed integration test without changing replay behavior. The real HTTP test had a 200 ms ceiling even though its actual 8-frame schedule is 180 ms before HTTP, event-loop, and timer-resolution overhead: role + 6 content + finish frames receive [40, 20, 20, 20, 20, 20, 20, 20] ms at 2x speed. The upper bound is replaced by a fake-timer writer assertion of that exact schedule; the HTTP test remains to prove the route wires replaySpeed into the writer.

Local Red-Green Proof

Red, clean c45bc66, actual target test repeated 30 times:

10/30 failures
expected 204 to be less than 200
expected 203 to be less than 200
expected 259 to be less than 200

Green, 4b6cedb, same actual target test repeated 30 times:

summary: 30/30 passes

The deterministic writer contract and real route both pass. Full suite: 166 files, 4,794 passed, 3 skipped; format, lint, TypeScript check, build, and export checks pass.

Nested GA Output Correction

Commit c45bc66 accepts and round-trips nested GA session.audio.output.voice and its complete PCM format object without changing Beta or legacy session translation. The recorder integration now explicitly proves fixture persistence and replay of terminal languages and usage metadata after the client closes.

Direct Local Red-Green Proof

A standalone real local recorder and WebSocket probe ran against clean ead88d4 before the change, then against this commit.

Red (ead88d4):

RECORDER {"persisted":true,"savedResponse":{"transcription":{"text":"typed terminal","languages":[{"code":"en"}],"usage":{"input_tokens":4,"output_tokens":5}}},"replayStatus":200}
WEBSOCKET {"betaDefault":null,"betaNested":{"type":"server_vad","threshold":0.7},"betaLegacy":{"type":"semantic_vad"},"gaFormat":{"type":"audio/pcm","rate":24000},"gaOutput":{"format":null,"voice":null}}

Green (c45bc66):

RECORDER {"persisted":true,"savedResponse":{"transcription":{"text":"typed terminal","languages":[{"code":"en"}],"usage":{"input_tokens":4,"output_tokens":5}}},"replayStatus":200}
WEBSOCKET {"betaDefault":null,"betaNested":{"type":"server_vad","threshold":0.7},"betaLegacy":{"type":"semantic_vad"},"gaFormat":{"type":"audio/pcm","rate":24000},"gaOutput":{"format":{"type":"audio/pcm","rate":24000},"voice":"alloy"}}

Focused real integration coverage: 168 tests passed. Full pre-push suite: 4,793 passed, 3 skipped.

Post-Rebase Corrections

ead88d4 closes the rebase-review gaps without changing the release version: typed transcription terminals are now treated as complete on client close; Beta session events retain root turn_detection; GA retains the documented PCM rate.

Direct Local Red-Green Proof

The same standalone probe used a real upstream SSE recorder proxy, client abort, persisted-fixture local replay, and live WebSocket connections.

Red (89d4e5c):

RECORDER {"persisted":false,"replayStatus":502,"replayBody":"{\"error\":{\"message\":\"Proxy to upstream failed: connect ECONNREFUSED 127.0.0.1:53771\",\"type\":\"proxy_error\"}}"}
WEBSOCKET {"gaFormat":{"type":"audio/pcm"}}

Green (ead88d4):

RECORDER {"persisted":true,"replayStatus":200,"replayBody":"data: {\"type\":\"transcript.text.delta\",\"delta\":\"typed terminal\"}\n\ndata: {\"type\":\"transcript.text.done\",\"text\":\"typed terminal\",\"languages\":[{\"code\":\"en\"}]}\n\n"}
WEBSOCKET {"betaDefault":null,"betaNested":{"type":"server_vad","threshold":0.7},"betaLegacy":{"type":"semantic_vad"},"gaFormat":{"type":"audio/pcm","rate":24000}}

The red JSON omits Beta keys because JavaScript omits undefined object properties. The green output verifies default null, nested GA update, and legacy flat update independently.

Release / Rebase Status

Rebased the transcription implementation and its contract coverage onto current main (9ce3107) without changing the intended runtime behavior. The PR now releases @copilotkit/aimock 1.38.0 and aligns its publish surfaces: root package, dated changelog entry, Helm appVersion, pytest default pin plus pytest package 0.5.1, and Claude plugin metadata/range.

Current branch commits:

• 3cf3956 feat: support OpenAI transcription replay
• cfbbe0a test: cover transcription replay contracts
• 89d4e5c chore: release v1.38.0

The retained proof below was captured against the original behavior branch; the rebased branch passed the focused 400-test transcription suite, full 4,746-test npm suite, format, lint, explicit TypeScript check, build, exports, frozen-lockfile validation, pytest 17/17, and pytest package build.

Summary

Aligns OpenAI transcription HTTP, Realtime, recorder, and replay behavior with the current nested GA wire contract.

• Dedicated intent=transcription sockets ignore regular realtime query models.
• GA session events serialize audio.input.format and audio.input.transcription; Beta keeps its flat shim.
• Current input-transcription configuration emits progressive transcript events, retains languages, preserves empty transcript fixtures, honors stream timing and interruption controls, and ends at typed transcript.text.done without a [DONE] sentinel.

Baseline vs Candidate Local Proof

The exact probe ran against a clean detached 8d425a4 worktree and the isolated candidate worktree. It used real local HTTP, WebSocket, recorder proxy, persisted fixture, and replay paths.

Red: clean 8d425a4

INTENT_MODELS gpt-realtime-2 wrong-model
CURRENT_GA {"voice":null,"input_audio_format":"audio/pcm","output_audio_format":null,"input_audio_noise_reduction":null,"input_audio_transcription":{"model":"gpt-transcribe"}} session.created,session.updated,input_audio_buffer.committed
INVALID_MODEL Model gpt-realtime-2 does not support session type transcription
HTTP interrupt 200 "data: {\"type\":\"transcript.text.delta\",\"delta\":\"ab\"} ... data: {\"type\":\"transcript.text.done\",\"text\":\"abcdefgh\"}\n\nevent: done\ndata: [DONE]\n\n"
HTTP missing 404 "{\"error\":{\"message\":\"No fixture matched\",\"type\":\"invalid_request_error\",\"code\":\"no_fixture_match\"}}"
HTTP error 422 "{\"error\":{\"message\":\"bad transcription\",\"type\":\"invalid_request_error\",\"param\":null,\"code\":null}}"
RECORDED_FIXTURES ... "response":{"transcription":{"text":"hello"}}

Green: candidate 6ae6ec4

INTENT_MODELS gpt-transcribe gpt-transcribe
CURRENT_GA {"input":{"format":{"type":"audio/pcm"},"noise_reduction":null,"transcription":{"model":"gpt-transcribe"}},"output":{"format":null,"voice":null}} session.created,session.updated,input_audio_buffer.committed,conversation.item.added,conversation.item.input_audio_transcription.delta...,conversation.item.input_audio_transcription.completed
INVALID_MODEL Model wrong-model does not support session type transcription
HTTP interrupt FETCH_ERROR TypeError: terminated
HTTP missing 404 "{\"error\":{\"message\":\"No fixture matched\",\"type\":\"invalid_request_error\",\"code\":\"no_fixture_match\"}}"
HTTP error 422 "{\"error\":{\"message\":\"bad transcription\",\"type\":\"invalid_request_error\",\"param\":null,\"code\":null}}"
RECORDED_FIXTURES ... "response":{"transcription":{"text":"hello","languages":[{"code":"en"}]}}

Validation

prettier --check  PASS
eslint            PASS
tsc --noEmit      PASS
Focused protocol suites: 312 tests PASS
Full pnpm test and pnpm build completed after the GA conformance update with no failure output.

Fourth-Pass Red-Green

Red: terminal reported four stale GA expectations (legacy audio transcription field and missing ).

Green:
RUN v3.2.4 /private/tmp/aimock-live-transcribe-pr.0mGErr

✓ src/tests/ws-realtime.test.ts (83 tests) 7614ms

Test Files 1 passed (1)
Tests 83 passed (83)
Start at 14:39:20
Duration 8.13s (transform 235ms, setup 0ms, collect 300ms, tests 7.61s, environment 0ms, prepare 44ms) passed 83/83. Direct clean-baseline vs candidate VAD probe changed into nested , progressive live deltas, and journal interruption before completion. Full format, lint, typecheck, 145-file test suite, and build passed.

Fourth-pass proof: red terminal ws-realtime test had four stale GA audio expectation failures. Green exact file: 83 of 83 passed. Clean baseline VAD emitted an error; candidate returned nested audio.input.turn_detection server_vad, progressive deltas, and truncateAfterChunks journal interruption. Full format, lint, typecheck, 145-file tests, and build passed.

Fifth-pass red-green: baseline emitted GA session type conversation, top-level turn_detection, and accepted model mutation. Candidate emits realtime for normal GA sessions, keeps VAD under audio.input, rejects established model mutation, and preserves explicit transcription/translation session types. Red conformance test at ws-api-conformance line 618 is now green (66/66); full format, lint, tsc, tests, and build passed.

Sixth-Pass Red-Green

Red: clean baseline 2871866 accepted a real WebSocket mixed model/type update and persisted gpt-4o-transcribe transcription must-not-mutate.

Green: candidate returned error invalid_session_config; its next valid update reported gpt-realtime-mini realtime safe, proving rejection occurs before any session mutation. Live-transcription interruption now uses socket destruction and journals the interruption.

Validation: focused ws-realtime suite 83/83 passed. Full format check, lint, typecheck, test suite, and build passed locally.

P2 Coverage Hardening

Commit f2ffc7e adds tests only; it does not change runtime behavior. The clean 71452a7 baseline already demonstrated the documented intent=transcription strict no-match and ErrorResponse paths, versioned model matching, abrupt interrupted socket termination with journal details, and scheduled inter-frame delivery.

The new direct tests cover those established paths, HTTP stream=true JSON errors, and JSON/SSE recorder-to-local-replay preservation of languages and usage. Applied as a test-only diff to detached 71452a7, the focused suite passed 132/132; the candidate focused suite also passed 132/132. Full format, lint, typecheck, test, and build passed on the candidate.

API-key validation and proxy isolation

This PR also adds opt-in, server-wide inbound API-key validation for the pending 1.38.0 Node release, plus the aimock-pytest 0.5.2 helper support. No major-version change is included.

• Validates Authorization: Bearer, X-API-Key, X-Goog-API-Key, Api-Key, and xi-api-key before fixture, proxy, control, and WebSocket handling.
• Leaves /health, /ready, and /metrics public; validates auth-enabled bare OPTIONS requests and WebSocket upgrades before mount routing.
• Treats test credentials as inbound-only: direct-recording paths remove them before egress and use configured provider credentials. Missing provider credentials fail locally with 502, without calling upstream.
• Adds api_key and --aimock-api-key support to the pytest helper, injected through the child environment rather than command-line arguments.

Local API-key red-green proof

Observed against the real server before the implementation:

HTTP_RED 200 {"id":"chatcmpl-EjY1_yI49ir9kGx0","object":"chat.completion","created":1785787642,"model":"gpt-4","choices":[{"index":0,"message":{"role":"assistant","content":"fixture-content","refusal":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":2,"completion_tokens":4,"total_tokens":6}}
WS_RED HTTP/1.1 101 Switching Protocols

The same raw HTTP/WebSocket probe after the implementation:

HTTP_CANDIDATE wrong-key 401 {"error":{"message":"Invalid API key","type":"authentication_error","code":"invalid_api_key"}}
HTTP_VALID 200 fixture-content
WS_CANDIDATE wrong-key HTTP/1.1 401 Unauthorized
HTTP_CANDIDATE test-key 200 {"id":"chatcmpl-ULZ9ObI2WG8hJhas","object":"chat.completion","created":1785788256,"model":"gpt-4","choices":[{"index":0,"message":{"role":"assistant","content":"fixture-content","refusal":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":2,"completion_tokens":4,"total_tokens":6}}
HTTP_VALID 200 fixture-content
WS_CANDIDATE test-key HTTP/1.1 101 Switching Protocols

Value checks

• HTTP completion: a wrong key changed from 200 to 401; the configured key remains 200 with the recorded fixture.
• Realtime WebSocket: a wrong key changed from 101 Switching Protocols to literal 401; the configured key still upgrades with 101.
• Generic recorder/proxy: an inbound test key is stripped and replaced with the configured provider key; with no provider key it returns local 502 and the fake upstream receives zero requests.
• Pytest helper: the child and control request both receive the configured key; PYTHONPATH=src AIMOCK_CLI_PATH=../../dist/cli.js pytest -q -p aimock_pytest.plugin passed all 18 tests.

Validation

pnpm format:check                         PASS
pnpm lint                                 PASS
pnpm exec tsc --noEmit                    PASS
pnpm build                                PASS
pnpm test:exports                         PASS
pnpm test                                 167 files passed, 4802 passed, 3 skipped
pytest helper integration                 18 passed
git diff --check                          PASS

CR repair: request-scoped API-key egress

This follow-up removes the process-global record-config auth marker. The egress policy is now tied to the authenticated incoming request, so servers that reuse a RecordConfig cannot affect one another. It also accepts bearer and key schemes case-insensitively, removes Authorization case-insensitively before OpenRouter off-origin unsigned-URL fetches, and applies the same no-implicit-provider-key policy to AG-UI, fal, and video egress.

The release remains Node 1.38.0 and pytest 0.5.2; no major or additional root-version bump is included. PyPI publishing is now gated on the pinned npm release existing first, enforcing npm 1.38.0 before pytest 0.5.2.

Local red-green proof

Clean f3a21fb baseline, exercised against live local server sockets and a local proxy upstream:

LOWERCASE_HTTP_RED 401
LOWERCASE_WS_RED HTTP/1.1 401 Unauthorized
SERVER_ISOLATION_RED 502 NO_EGRESS
OFF_ORIGIN_RED expected 'Bearer provider-key' to be undefined

The identical raw HTTP/WebSocket/proxy probes after the repair:

LOWERCASE_HTTP_GREEN 200
LOWERCASE_WS_GREEN HTTP/1.1 101 Switching Protocols
SERVER_ISOLATION_GREEN 200 Bearer external-client-key
OFF_ORIGIN_GREEN undefined

Coverage added

• Real raw duplicate credential lines and lowercase bearer / key WebSocket upgrade coverage.
• Two-server shared-RecordConfig isolation regression.
• Foreign-host OpenRouter capture that asserts a configured provider Authorization is absent.
• Direct authenticated AG-UI and fal egress: 502 with zero upstream requests when no provider key exists; OpenRouter video path is covered by the foreign-host capture.
• CLI config-file auth plus AIMOCK_API_KEYS override integration.
• Real pytest subprocess using --aimock-api-key and the plugin fixture.

Final validation

pnpm format:check                         PASS
pnpm lint                                 PASS
pnpm exec tsc --noEmit                    PASS
pnpm test                                 167 files passed, 4806 passed, 3 skipped
pnpm build                                PASS
pnpm test:exports                         PASS
pytest helper integration                 19 passed
pnpm install --frozen-lockfile --ignore-scripts PASS
actionlint publish-pytest.yml             PASS
docker buildx (native default context)    PASS
git diff --check                          PASS

Pytest CI repair

The release test subprocess now disables automatic plugin discovery before it
explicitly loads aimock_pytest.plugin. This prevents installed pytest entry
point discovery from registering the same plugin a second time.

Local installed-package red-green proof (Python 3.12):

RED: ValueError: Plugin already registered under a different name: aimock=<module 'aimock_pytest.plugin' ...>
GREEN: 1 passed, 13 deselected in 0.43s
GREEN full installed-package suite: 19 passed in 5.85s

The repeated final gate is green: format, lint, TypeScript, full Node suite
(167 files, 4,806 passed, 3 skipped), build, exports, and source-tree pytest
integration (19 passed).

Publish-workflow security repair

actions/setup-node v7 enabled package-manager caching by default in the PyPI
publish workflow. Caching is unnecessary for this small npm view release
ordering check, so the workflow now explicitly sets
package-manager-cache: false.

RED: error[cache-poisoning] at publish-pytest.yml:25
GREEN: zizmor: No findings to report
GREEN: actionlint publish-pytest.yml PASS

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@345

commit: 29645e1