Skip to content

v2.2.1 — Increase Bedrock request timeout for large prompts

Choose a tag to compare

View all tags
@tverney tverney released this 04 Apr 21:03
· 29 commits to main since this release
v2.2.1
f78c6c8

Fixes timeout errors when processing large consolidation/extraction prompts (50K-100K+ chars) against Bedrock.

What changed

  • Added explicit NodeHttpHandler with configured timeouts:
    • connectionTimeout: 10 seconds (establish TCP connection)
    • requestTimeout: 5 minutes (wait for LLM response)
  • The AWS SDK v3 has no default request timeout, but the underlying HTTP agent can close idle sockets prematurely. The explicit handler ensures long-running Converse API calls complete successfully.

Why

Large prompts (e.g. 94K chars from 35 session files) can take several minutes for the LLM to process and respond. Without an explicit timeout configuration, the request could be terminated prematurely by the default socket behavior.

Assets 2
Loading