Conversation
commit: |
jpr5
force-pushed
the
ws-drift-detection
branch
3 times, most recently
from
d82515c to
2a5266c
Compare
The real OpenAI Responses WS API expects a flat message format:
{ type: "response.create", model: "...", input: [...] }
The handler previously required fields nested under a "response"
object, which doesn't match the real API. Updated the handler and
all existing WS tests to use the flat format.
…emini Live TLS WebSocket client (ws-providers.ts) connects to real provider WS endpoints using node:tls with RFC 6455 framing, ping/pong, and connection-scoped message cursors for multi-step protocols. 4 verified drift tests: - OpenAI Responses WS: text + tool call - OpenAI Realtime: text + tool call (gpt-4o-mini-realtime-preview) Canaries: - Realtime: checks gpt-4o-mini-realtime-preview still exists in model listing API, with hints for the GA replacement - Gemini Live: checks model listing API for text-capable bidiGenerateContent models; full drift tests skipped until Google ships a non-audio Live model Supporting changes: - sdk-shapes.ts: Realtime + Gemini Live event shapes - helpers.ts: collectMockWSMessages(), classifyGeminiMessage, GEMINI_WS_PATH - models.drift.ts: filter markdown anchor fragments from model scraper
DRIFT.md: WS coverage table with verified/unverified status, Gemini Live explanation, cost estimate (25 API calls), "Adding a New Provider" WS step. README.md: fix Gemini Live response shape example, update model name, add unverified warning, fix Responses WS example to use flat format. docs/index.html: add unverified note to Gemini Live in feature list and comparison table. CHANGELOG.md: 1.3.3 patch notes. vitest.config.drift.ts: increase testTimeout to 60s for WS protocols.
jpr5
force-pushed
the
ws-drift-detection
branch
from
2a5266c to
e5870ed
Compare
jpr5
added a commit
that referenced
this pull request
Apr 3, 2026
## Summary
- **Fix Responses WS input format**: handler now accepts the flat
`response.create` format matching the real OpenAI API (previously
required a non-standard nested `response: { ... }` envelope)
- **4 verified WS drift tests**: OpenAI Responses WS (text + tool call)
and OpenAI Realtime (text + tool call), triangulated against real APIs
- **Model canaries**: Realtime preview model availability check (detects
deprecation, suggests GA replacement); Gemini Live text-capable model
availability check (enables drift tests when Google ships one)
- **Gemini Live**: protocol implemented per docs, documented as
unverified — no text-capable `bidiGenerateContent` model exists yet
- TLS WebSocket client (`ws-providers.ts`) with RFC 6455 framing,
ping/pong, connection-scoped cursors
- SDK shapes for Realtime and Gemini Live event sequences
- Fix README Gemini Live response shape example and Responses WS example
## Test plan
- [x] `pnpm test` — 540 unit tests pass
- [x] `pnpm test:drift` without keys — all 27 tests skip gracefully
- [x] `pnpm test:drift` with keys — 25 pass, 2 skip (Gemini Live
text/tool)
- [x] `pnpm run format:check` — clean
- [x] `pnpm run lint` — clean
- [x] 5 rounds of CR→fix loop with full pr-review-toolkit suite
🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
response.createformat matching the real OpenAI API (previously required a non-standard nestedresponse: { ... }envelope)bidiGenerateContentmodel exists yetws-providers.ts) with RFC 6455 framing, ping/pong, connection-scoped cursorsTest plan
pnpm test— 540 unit tests passpnpm test:driftwithout keys — all 27 tests skip gracefullypnpm test:driftwith keys — 25 pass, 2 skip (Gemini Live text/tool)pnpm run format:check— cleanpnpm run lint— clean🤖 Generated with Claude Code