v0.4.0 - Features from ai-lib-rust

ai-lib-python v0.4.0

This release completes the feature parity initiative, incorporating all key features from the Rust reference implementation (ai-lib-rust).

✨ New Features

🔀 Model Routing & Load Balancing (routing/)

A complete model management and selection system:

• ModelInfo - Comprehensive model information structure

  • ModelCapabilities: Chat, code generation, multimodal, function calling, tool use
  • PricingInfo: Cost calculation per 1K tokens
  • PerformanceMetrics: Speed and quality tier classification

• Selection Strategies

  • RoundRobinSelector - Cyclic selection
  • WeightedSelector - Speed + quality weighted
  • CostBasedSelector - Cheapest model
  • PerformanceBasedSelector - Fastest model
  • QualityBasedSelector - Highest quality
  • RandomSelector - Random selection

• ModelManager - Centralized model management

  • Filter by capability, cost, context window
  • Recommendation by use case
  • JSON config import/export

• ModelArray - Load balancing across endpoints

  • Health-based endpoint selection
  • Multiple balancing strategies

• Pre-configured Catalogs

  • create_openai_models() - GPT-4o, GPT-4o-mini, o1-preview
  • create_anthropic_models() - Claude 3.5 Sonnet, Haiku, Opus

📊 Resilience Signals & Preflight (resilience/)

Unified runtime state aggregation for orchestration decisions:

• SignalsSnapshot - Aggregated runtime signals

  • InflightSnapshot - In-flight request state
  • RateLimiterSnapshot - Rate limiter state
  • CircuitBreakerSnapshot - Circuit breaker state
  • health_score - Composite health metric (0.0-1.0)

• PreflightChecker - Unified request gating

  • Rate limiter pre-check
  • Circuit breaker pre-check
  • Backpressure permit acquisition
  • Response header rate limit extraction

• PreflightContext - Async context manager

🔄 FanOut Stream Operator (pipeline/)

Stream splitting and array expansion:

• FanOutTransform - Expand arrays into separate events

  • Configurable array path
  • Metadata preservation
  • Auto-detection of common patterns (choices, candidates, results)

• ReplicateTransform - Duplicate each event

• SplitTransform - Predicate-based filtering

📝 User Feedback Collection (telemetry/)

Typed feedback events for RLHF and quality evaluation:

• Feedback Types

  • ChoiceSelectionFeedback - Multi-candidate selection with latency
  • RatingFeedback - 1-5 star ratings
  • ThumbsFeedback - Up/down feedback
  • TextFeedback - Free-form text
  • CorrectionFeedback - Edit tracking
  • RegenerateFeedback - Regeneration events
  • StopFeedback - Stop generation events

• FeedbackSink Implementations

  • NoopFeedbackSink - Discard (default)
  • InMemoryFeedbackSink - Store in memory
  • ConsoleFeedbackSink - Print to console
  • CompositeFeedbackSink - Multi-sink routing

⏹️ Stream Cancellation (client/)

Cooperative cancellation for streaming operations:

• CancelToken - Cancellation token

  • Cancel with reason (user_request, timeout, error)
  • Callback registration
  • Async wait support

• CancelHandle - Public cancel interface

• CancellableStream - Wrap async iterators

• create_cancel_pair() - Factory function

🔧 ToolCall Assembler (utils/)

Assemble streaming tool call fragments:

• ToolCallAssembler - Single-turn assembly

  • Handle started events
  • Accumulate argument fragments
  • JSON parsing with fallback

• MultiToolCallAssembler - Multi-turn tracking

✅ Protocol Validation Enhancements (protocol/)

• Protocol Version Validation

  • SUPPORTED_PROTOCOL_VERSIONS: 1.0, 1.1, 1.5, 2.0
  • Deprecation warnings for old versions

• Strict Streaming Validation

  • Decoder format check
  • Content path requirements
  • Tool call path requirements
  • AI_LIB_STRICT_STREAMING environment variable

📈 Test Coverage

• 155 new tests added
• 492 total tests passing
• All new modules have comprehensive test coverage

📦 Installation

pip install ai-lib-python==0.4.0

🔗 Related

• ai-lib-rust - Rust reference implementation
• ai-protocol - Protocol specification