feat(gateway): add OpenAI Responses API wire protocol support#263
Merged
Conversation
Add POST /v1/responses route to handle the OpenAI Responses API, enabling Lore memory features for openai, openai-codex, and azure-openai-responses providers that use this protocol instead of /v1/chat/completions. - New ingress/egress translator (translate/openai-responses.ts) that normalizes Responses API input items to GatewayRequest and converts GatewayResponse back to Responses API format (streaming + non-streaming) - New SSE stream accumulator (stream/openai-responses.ts) that parses Responses API events into GatewayResponse - Third protocol branch in forwardToUpstream() preserving openai-responses wire protocol through the pipeline - Pi plugin: add openai + 5 newly-discovered compatible providers - 27 new tests covering translator and stream accumulator
…d_tokens tracking Fix pipeline to correctly accumulate upstream responses for all three wire protocols (Anthropic, OpenAI Chat Completions, OpenAI Responses API). Previously, all responses were parsed as Anthropic format, silently losing all content and usage data when the upstream was OpenAI. - Add effectiveProtocol to UpstreamResult so pipeline can dispatch to the correct accumulator (Anthropic, OpenAI Chat Completions, or Responses API) - Add accumulateOpenAINonStreamJSON and accumulateResponsesNonStreamJSON for non-streaming upstream responses - Add accumulateNonStreamOpenAIStream for OpenAI Chat Completions SSE - Route streaming openai-responses through accumulateResponsesSSEStream - Parse prompt_tokens_details.cached_tokens from OpenAI usage responses and map to existing cacheReadInputTokens (semantic equivalent) - Emit prompt_tokens_details.cached_tokens in Chat Completions and Responses API egress responses when cache data is available - Prompt ordering already optimal for OpenAI automatic prefix caching (system/tools before messages in both builders)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #239
Adds
POST /v1/responsesroute to handle the OpenAI Responses API wire protocol, enabling Lore memory features (LTM injection, gradient transforms, temporal capture, recall) for providers that use this protocol. Also fixes protocol-aware upstream response accumulation and adds OpenAI cached token tracking.Wire Protocol Support
translate/openai-responses.ts): Ingress parser normalizes Responses API input items (message, function_call, function_call_output) toGatewayRequest; egress builder convertsGatewayResponseback to Responses API format (both streaming SSE and non-streaming JSON)stream/openai-responses.ts): Parses Responses API SSE events (response.output_text.delta,response.function_call_arguments.delta,response.completed, etc.) intoGatewayResponse, reusingparseSSEStreamfrom the Anthropic moduleforwardToUpstream()that preserves"openai-responses"from ingress (prevents model-prefix routing from downgrading to"openai"), routes to correct upstream URLProtocol-Aware Response Accumulation
Previously
accumulateNonStreamResponseonly parsed Anthropic-format responses, silently losing all content and usage data when the upstream was OpenAI. Fixed by:effectiveProtocoltoUpstreamResultso the pipeline dispatches to the correct accumulatoraccumulateOpenAINonStreamJSON,accumulateResponsesNonStreamJSON,accumulateNonStreamOpenAIStream(for OpenAI Chat Completions SSE)openai-responsesrouted throughaccumulateResponsesSSEStreamOpenAI Cached Token Tracking
prompt_tokens_details.cached_tokensfrom OpenAI usage responses (all three paths: non-stream JSON, stream SSE, Responses API SSE) and map to existingcacheReadInputTokensprompt_tokens_details.cached_tokensin Chat Completions and Responses API egress responsesPi Plugin
"openai"+ 5 newly-discovered compatible providers (zai,minimax,minimax-cn,kimi-coding,vercel-ai-gateway) toGATEWAY_PROVIDERSVerification
Deferred
openai-codexandazure-openai-responsesPi provider redirection (need URL pattern investigation)