Skip to content

fix: increase default LLM timeout to 300s for local models#144

Merged
NotYuSheng merged 2 commits intomainfrom
fix/llm-timeout-default
Apr 1, 2026
Merged

fix: increase default LLM timeout to 300s for local models#144
NotYuSheng merged 2 commits intomainfrom
fix/llm-timeout-default

Conversation

@NotYuSheng
Copy link
Copy Markdown
Owner

Summary

  • The default LLM_TIMEOUT of 60s was too short for local LLMs (e.g. LM Studio with Qwen2.5-14B), causing story generation to fail with "LLM server is not responding" even when the model was actively processing
  • Added LLM_TIMEOUT=300 to .env.example and docker-compose.yml, raising the default to 5 minutes
  • Users can override via the LLM_TIMEOUT env var if their hardware needs more time

Test plan

  • Set LLM_TIMEOUT=300 in .env, restart backend with docker compose up -d backend
  • Trigger story generation on a PCAP file with a local LM Studio model
  • Confirm story generates successfully without timeout error

🤖 Generated with Claude Code

The default 60s timeout was too short for local LLMs (e.g. LM Studio
with Qwen2.5-14B) generating full story JSON, causing "LLM server is
not responding" errors even when the model was actively processing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a configurable LLM_TIMEOUT environment variable in .env.example and docker-compose.yml, defaulting to 300 seconds to accommodate slower local models. Review feedback points out that the implementation is incomplete because the internal application defaults in application.yml and LlmConfig.java are still set to 60 seconds, which may lead to inconsistent behavior. Additionally, it is suggested to update the documentation in .env.example to clarify that 300 seconds is a recommended value rather than the system-wide fallback.

Comment thread docker-compose.yml
Comment thread .env.example Outdated
- application.yml: change ${LLM_TIMEOUT:60} -> ${LLM_TIMEOUT:300}
- LlmConfig.java: change hardcoded fallback 60_000ms -> 300_000ms
- .env.example / .env: clarify comment — 300s is recommended, not the
  internal default (they now match since application.yml is updated)

Addresses review feedback on PR #144.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@NotYuSheng NotYuSheng merged commit 1bdc787 into main Apr 1, 2026
@NotYuSheng NotYuSheng deleted the fix/llm-timeout-default branch April 1, 2026 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant