Repository navigation
Release b188
feat: Support stream usage statistics
- Added
stream_optionstoChatCompletionRequestandCompletionRequestmodels. - Automatically inject
stream_options: {"include_usage": true}inbuild_request_body_genericfor streaming requests if not provided. - Enhanced
stream_logger_taskto parsetimingsfromllama.cppresponses and convert them to standard OpenAIusageformat for logging.