Skip to content

v0.9.2

Choose a tag to compare

@assareh assareh released this 01 Dec 07:18
· 21 commits to main since this release

Fixed

  • Malformed Tool Token Detection - Detect and retry when models output raw internal tokens
    • Some models (especially Hermes/ChatML-style) output raw tokens like <|start|>assistant<|channel|>... instead of proper responses
    • New _contains_malformed_tool_tokens() method detects these patterns
    • Final response generation now retries once with a stern instruction if malformed tokens detected
    • Falls back to error response if retry also fails
    • Prevents garbage output from reaching users