0.0.86

Otoroshi LLM Extension — v0.0.86

Added

• Native /responses endpoint on providers — new responses provider option (false by default). When
  it is on, /responses requests go to the provider's own Responses API instead of being translated to chat
  completions, which unlocks reasoning items, previous_response_id continuity and native file inputs.
  Available on OpenAI and OpenAI-compatible providers, Azure OpenAI (api_version v1 only, otherwise the
  option is ignored and a warning is logged), Groq, x.ai and Ollama (v0.13.3+). The payload is rebuilt from
  the conversation held by the gateway, chat-only options are renamed (max_tokens → max_output_tokens,
  response_format → text.format) or dropped, parameters a provider documents as unsupported are dropped
  instead of being forwarded, and the provider payload is returned as is. Wasm functions, MCP connectors and
  search engines keep working, declared as native tools with the tool loop running on function_call items.
• Full gateway pipeline on /responses — guardrails, prompt contexts, persistent memory, model
  restrictions, semantic and simple caches, cost tracking, budgets, auditing and metrics now apply to
  /responses exactly as they do to /chat/completions, on both the native and the translated path.
  Audit events and metrics distinguish the endpoint (responses/blocking, responses/streaming,
  ai.responses.*).
• Document content parts — {"type":"file"} on /chat/completions and input_file on the Responses
  proxies. PDF and plain text, inline as a data-uri (file_data) or by url (file_url), forwarded in the
  format of the target provider: a file part for OpenAI-compatible providers, a document block for
  Anthropic. The media type is deduced from the data-uri or the filename.
• Admin UI — Native responses API toggle on the providers supporting it.

Fixed

• File content parts were silently dropped for every provider and every proxy — the part was replaced by
  an empty text block, so the model answered as if nothing had been attached (#188, #186). Unusable or
  unsupported content parts are now logged instead of vanishing.
• Anthropic sampling parameters — temperature / top_p / top_k are now also stripped for the whole
  Claude 5 generation (opus, sonnet, fable, mythos), which rejects them with a 400. Model names carrying a
  date suffix (claude-opus-4-5-20251101) or a platform prefix (anthropic.claude-opus-5 on Bedrock) are
  matched too.
• Responses streaming — tool calls are now rebuilt from the argument deltas and returned as
  function_call output items, token usage is reported on the final response.completed event, and empty
  text deltas are no longer emitted.
• Responses request translation — text.format maps to response_format, reasoning.effort to
  reasoning_effort, and tools are converted to the chat {"type":"function","function":{…}} shape.
  Responses-only parameters are stripped before reaching a chat completions provider.

Release Infos

• the documentation is available here
• release is available here

Contributors

• @mathieuancelin