v0.7.1 - Tool Loop Timeout & Graceful Completion

What's New

Tool Loop Timeout

• New TOOL_LOOP_TIMEOUT config option (default: 120 seconds, 0 = no timeout)
• Prevents runaway tool loops from blocking requests indefinitely
• Environment variable: <PREFIX>_TOOL_LOOP_TIMEOUT

Graceful Tool Loop Completion

When the tool loop hits max iterations or timeout, the server now:

• Makes one final backend call WITHOUT tools
• Forces the LLM to synthesize a proper response from all gathered tool results
• Replaces the previous canned "I apologize" error message with actual useful content

This ensures users always get a meaningful response, even when the LLM gets stuck in a tool-calling loop.

Installation

# Using uv
uv sync

# Or pip
pip install git+https://github.com/assareh/llm-api-server.git@v0.7.1

Full Changelog

https://github.com/assareh/llm-api-server/blob/main/CHANGELOG.md