Skip to content

v0.7.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Jul 01:42
· 1148 commits to master since this release
v0.7.0
c52c833

Highlights

Anthropic Cache Control Preservation

Anthropic's block-level cache_control now round-trips through the IR pipeline via a new cache_hint field on 7 IR types (TextPart, ImagePart, FilePart, ToolCallPart, ToolResultPart, ReasoningPart, ToolDefinition). Non-Anthropic converters ignore the field — zero cross-format leakage. (#362, #366)

Structured System Messages + Flatten Transform

system_instruction is now list[TextPart] across the IR, preserving block boundaries for cache_hint passthrough. OpenAI Chat converter outputs structured content arrays; a new flatten_system_content() body-level transform handles downstream compatibility. Auto-enabled for Gemini models via gateway config. (#364, #365, #370, #371)

ConversionPipeline + Routing Extraction

The gateway internals were restructured around a ConversionPipeline class and a ResolvedRoute + Router protocol, cleanly separating format conversion, routing, and transport concerns. (#322, #323, #325, #327, #329, #331, #332, #334)

Transform System Overhaul

  • Body-level transforms renamed: from_transforms → pre_ir_transforms, to_transforms → post_ir_transforms — making pipeline position explicit instead of ambiguous direction names
  • Backward compatible: old names still work (with DeprecationWarning), all 11 built-in provider transforms migrated
  • New primitives: flatten_system_content(), message-level transforms, model-conditional transforms
  • (#333, #368, #369)

Other Features

  • In-memory content capture for request/response inspection (#360)
  • Enhanced admin panel — settings popup with password, token, log level, retention, error detail toggle (#347, #358)
  • Hybrid profiling — phase timing + on-demand pyinstrument (#339)
  • Observability layer extracted from gateway admin for reuse (#342)
  • stream_connect_ms added to streaming profile
  • cache_write_tokens field for OpenAI SDK 2.45+ compatibility

Bug Fixes

  • Return correct HTTP status for streaming upstream errors (#346)
  • Capture error responses in non-streaming path
  • Split CaptureState.status() to avoid Lock reentrancy
  • Fix ty type-check errors with openai SDK 2.45.0 and ty 0.0.54 (#343, #367)

Breaking Changes

  • system_instruction IR type changed from str to list[TextPart]. Code that constructs IRRequest with a string system_instruction must wrap it: [{"type": "text", "text": "..."}]
  • from_transforms / to_transforms are deprecated (still functional). Use pre_ir_transforms / post_ir_transforms.

Full Changelog: v0.6.12...v0.7.0

Assets 6
Loading