Skip to content

v0.2.2

Latest

Choose a tag to compare

@provos provos released this 06 Sep 21:56
87b36fd

OpenAI Responses API

llm_from_config(provider="openai") now talks to the Responses API, where OpenAI's reasoning models allow reasoning together with function tools (Chat Completions only accepts tools with reasoning_effort="none"). Requests are stateless; the reasoning and function-call items of a tool turn are replayed with their encrypted content on the next round, so a model keeps its chain of thought across tool calls. Structured outputs use a strict json_schema text format. Pass openai_api="chat" for the Chat Completions wrapper.

Reasoning effort and usage accounting

  • llm_from_config(effort=...) reaches OpenAI as reasoning_effort; reasoning tokens are reported in TokenUsage.reasoning_tokens.
  • TokenUsage.cache_creation_tokens counts prompt-cache writes (Anthropic bills them at the cache-write rate).
  • For Anthropic, reasoning_tokens is estimated when a response carries thinking blocks: output tokens not accounted for by the visible text and tool calls.

Fixes

  • Chat Completions: parse() refuses non-strict function tools (any tool with an optional parameter); the wrapper falls back to create() with a strict json_schema response format.
  • Chat Completions: models that only accept function tools with reasoning_effort="none" get one retry with that value, remembered per model.
  • Provider rate limits (HTTP 429) are retried with a growing backoff instead of failing the request.
  • supports_structured_output understands dotted model versions such as gpt-5.6-luna.