Skip to content

v0.1.13-update.2

Choose a tag to compare

View all tags
@asdek asdek released this 12 May 12:48
v0.1.13-update.2
18a7017
This commit was signed with the committer’s verified signature.
asdek Dmitry Ng
GPG key ID: 955A172DFDB77CC3
Verified
Learn about vigilant mode.

Highlights

This release introduces significant enhancements to the streaming capabilities, reasoning features, and tool calling functionality across multiple LLM providers. It focuses on improving both API consistency and the robustness of interactions with various AI models.

Breaking Changes

  • Streaming API Interface: Changed the signature of StreamingFunc from func(ctx context.Context, chunk []byte) error to func(ctx context.Context, chunk streaming.Chunk) error
  • Removed StreamingReasoningFunc: This function has been consolidated into the new streaming architecture
  • Reasoning Configuration: Added explicit reasoning configuration options that may affect existing code using reasoning capabilities
  • Tool Call IDs: Changed tool call ID format for better cross-provider compatibility

Major Improvements

New Packages

  • Streaming Package: Added a dedicated package for handling streaming responses from language models

    • Unified interface for processing text, reasoning, and tool call chunks
    • Type-safe chunk structures with proper content type discrimination
    • Helper functions for properly invoking streaming callbacks by chunk type
    • Support for handling tool call data in streaming mode with proper state tracking

  • Reasoning Package: Added support for extracting and processing step-by-step thinking from models

    • Pattern-matching extraction for <think> and <thinking> tags in model responses
    • Stateful ChunkContentSplitter for handling streaming reasoning content
    • Support for both incremental streaming and complete content processing
    • Integration with all major LLM providers (OpenAI, Anthropic, Ollama)

LLM Client Enhancements

Ollama Client

  • Enhanced streaming support with proper chunk processing for text and tool calls
  • Added reasoning content extraction for models with thinking capabilities
  • Improved tool call handling with robust ID generation and argument parsing
  • Better error handling during streaming operations with detailed error messages
  • Support for concurrent tool calls with proper state tracking

Anthropic Client

  • Added support for parallel tool calls with proper tool ID tracking
  • Improved streaming API with sophisticated delta handling for various content types
  • Enhanced support for the thinking capability through dedicated API parameters
  • Better tool call state management during streaming with improved payload handling
  • Improved error handling with retries for rate limit errors

OpenAI Client

  • Added comprehensive test coverage for DeepSeek API and OpenRouter API, ensuring compatibility
  • Added support for multiple models through the same client interface
  • Improved handling of multi-tool call responses with better state management
  • Enhanced streaming response processing with proper content type discrimination
  • Added support for controlling reasoning effort with token budgeting
  • Fixed streaming tool call processing to properly accumulate arguments

Improved API Consistency

  • Standardized streaming interfaces across all LLM providers
  • Unified reasoning content extraction and processing
  • Consistent tool call format and handling across providers
  • Better type safety for message processing and tool interactions
  • Normalized effort levels for reasoning capabilities across providers

Comprehensive Testing

  • Added extensive test matrix covering 20+ models across multiple providers
  • Specific test cases for parallel tool calls functionality
  • Tests for reasoning capabilities with various model configurations
  • Cross-provider compatibility tests ensuring API consistency
  • Tests for streaming functionality with all content types

Documentation Updates

  • Added detailed package-level documentation for streaming and reasoning packages
  • Updated examples to demonstrate the new streaming API
  • Added examples for using the reasoning capabilities
  • Updated tool call examples to show parallel tool usage
  • Improved error messages and logging

Borrowed Upstream Pull Requests

This release incorporates code and ideas from the following upstream pull requests:

  • Support for reasoning in Anthropic and OpenAI requests from #1228 by @ppipada
  • Improved streaming API with proper content type discrimination from #950 by @diemus

Dependencies Updates

Same as in v0.1.13-update.1.

Contributors

Full Changelog: v0.1.13-update.1...v0.1.13-update.2

Contributors

asdek, ppipada, and diemus
Assets 2
Loading