Timeout Audit: Multiple unbounded async waits and inadequate timeout defaults across ACP, model API, tools, and storage layers

[Million-mo]

Timeout Audit: Unbounded Async Waits & Inadequate Defaults

Summary

A comprehensive timeout audit identified 6 critical unbounded async waits (potential permanent hangs), 7 high-risk areas with inadequate or missing timeout defaults, and several moderately risky patterns across the codebase. These issues can cause sessions, agents, or the entire server to hang indefinitely under adverse conditions (network partitions, subprocess deadlocks, slow LLM APIs, remote filesystem stalls).

---

🔴 CRITICAL — Unbounded Async Waits (Permanent Hang Risk)

1. LLM API Call — No wall-clock timeout on model inference

• File: src/agentpool/agents/native_agent/agent.py:694-711 → src/agentpool/agents/native_agent/turn.py:190
• Problem: PydanticAI Model instances (OpenAI, Anthropic, etc.) are created via llmling_models.infer_model() with no explicit HTTP timeout. usage_limits caps tokens/requests, not wall-clock time. If the upstream LLM API hangs (network partition, rate-limit stall, server hang), agentlet.iter() blocks indefinitely.
• Fix: Inject ModelSettings(timeout=httpx.Timeout(N)) into agentlet.iter() or _resolve_model_string(). Add an inference_timeout field to BaseAgentConfig.

2. ACP send_request() — No response timeout on JSON-RPC

• File: src/acp/connection.py:139
• Code: return await future
• Problem: The future is resolved only when the remote peer sends a JSON-RPC response. If the subprocess crashes without responding (or is overwhelmed), the caller hangs indefinitely. This is the common path for ALL ACP JSON-RPC calls (initialize, session/create, session/prompt, tool calls).
• Fix: Wrap with asyncio.wait_for(future, timeout=N) (default 120s, configurable).

3. ACP _receive_loop() — No read timeout

• File: src/acp/connection.py:154
• Code: async for chunk in self._text_reader:
• Problem: No timeout on reading from subprocess stdout. If the subprocess stops writing but keeps the pipe open (deadlock, infinite loop), the receive loop blocks forever.
• Fix: Add asyncio.wait_for() around reads, or add an inactivity watchdog.

4. ACP MessageSender._loop() — No write timeout

• File: src/acp/task/sender.py:94
• Code: await self._writer.send(item.payload)
• Problem: Writing to subprocess stdin can block if the subprocess stops reading (deadlock, backpressure). No timeout on send().
• Fix: Add a timeout around self._writer.send().

5. ACP _drain_stderr_to_log() — No timeout on stderr read

• File: src/acp/transports.py:782
• Code: async for line_bytes in process.stderr:
• Problem: Unbounded stderr read. If subprocess writes gigabytes of stderr or pipe gets stuck, background task hangs forever, preventing spawn_stdio_transport() cleanup.
• Fix: Add bounded read strategy (max lines/bytes per yield or timeout).

6. Read/Grep Tools — No timeout on filesystem operations

• Files:
  • src/agentpool/tool_impls/read/tool.py:168,186 — _cat_file() with no timeout
  • src/agentpool/tool_impls/grep/tool.py:128,144 — subprocess ripgrep and fsspec grep with no timeout
• Problem: On remote filesystems (S3, SSH, Docker), these can hang indefinitely.
• Fix: Wrap with asyncio.wait_for() (default 60s for read, 120s for grep).

---

🟡 HIGH — Inadequate Defaults or Conditional Risk

7. SQLAlchemy Engine — No connection/pool timeout

• File: src/agentpool_config/storage.py:53-80
• Problem: create_async_engine() called without connect_timeout, pool_timeout, pool_recycle, or pool_pre_ping. Database connection hangs are possible.
• Fix: Add connect_timeout=10, pool_timeout=30, pool_recycle=3600, pool_pre_ping=True.

8. ACP prompt() Call — No timeout

• File: src/agentpool/agents/acp_agent/acp_agent.py:464
• Problem: self._api.prompt() has no timeout wrapper. ACP subprocess hang → entire agent blocks.
• Fix: Wrap with asyncio.wait_for(timeout=300).

9. ACP-MCP Transport Timeout — 600s (10 min) is too long

• File: src/agentpool_server/acp_server/session.py:495
• Code: AcpMcpTransport(conn, timeout=600.0)
• Problem: 10-minute timeout masks hung MCP tools. Users wait 10 minutes for failure.
• Fix: Reduce to 120s.

10. MCP Server Default Timeout — 600s (10 min)

• File: src/agentpool_config/mcp_server.py:72
• Problem: Default timeout: float = 600.0 for both init handshake and per-request read timeout.
• Fix: Reduce default to 120s.

11. ACP Process Kill — Unbounded wait() after kill

• File: src/agentpool/agents/acp_agent/acp_agent.py:407
• Code: await self._process.wait() (after process.kill())
• Problem: If process refuses to die after kill() (zombie process), agent cleanup hangs forever.
• Fix: Add a second timeout (e.g. 3s) or retry loop.

12. Continuous Run — No per-iteration timeout

• File: src/agentpool/agents/base_agent.py:637
• Problem: await self.run(...) in continuous mode has no per-iteration timeout. If run() hangs (infinite LLM wait), the whole continuous loop freezes.
• Fix: Add configurable per-iteration timeout.

13. Team Member — Default no timeout

• File: src/agentpool_config/teams.py:73
• Problem: member_timeout: None means a hung member agent blocks the entire team indefinitely.
• Fix: Consider a default timeout (e.g. 300s).

---

🟢 WELL-GUARDED (No Action Needed)

Component	Location	Timeout
Session close cascade	session_controller.py:884,897	10s turn_lock + 2s complete_event
Child done events	run.py:390	30s
get_tools()	agent.py:855, turn.py:151	5s
Elicitation	context.py:167,430	300s configurable
ACP subprocess terminate	acp_agent.py:404	5s
MCP cleanup	mcp_server/manager.py:552	5s
ACP stdio shutdown	transports.py:855	2s × 2 stages
Download file tool	download_file/tool.py:52	30s httpx
EventBus publish	event_bus.py:500+	Non-blocking put_nowait
Title generation	storage/manager.py:828	15s
OpenCode route	session_routes.py:1960	30s
WS heartbeat	transports.py:483	Configurable pong_timeout

---

Proposed Fix Priority

Priority	Issue	Impact
P0	#2 ACP send_request()	Common path for ALL ACP IPC calls
P1	#1 LLM API timeout	All native agent runs
P1	#3 ACP receive loop	All ACP agents
P1	#4 ACP sender write timeout	All ACP agents
P1	#7 SQLAlchemy engine	All storage operations
P2	#5 ACP stderr drain	ACP subprocess lifecycle
P2	#6 Read/Grep tools	Remote filesystem usage
P2	#8 ACP prompt()	ACP agent runs
P2	#9-10 MCP timeout defaults	MCP tool calls
P2	#11 Kill wait timeout	ACP process cleanup
P3	#12 Continuous run timeout	Background agents
P3	#13 Team member default	Team execution

---

Related

• Branch fix/expert-interview-v0.3.1-patch addresses a session TTL sweeper issue where last_active_at was not updated on receive_request(). That fix is already present in develop/agentic at session_controller.py:1180-1182.
• Branch fix/session-ttl-last-active-at is the same fix on the refactored codebase.