feat(api): bound ANSWER reader by wall-clock timeout + document ask/write#196
Merged
Merged
Conversation
…rite Two follow-ups to #195: - ANSWER's reader callable was previously unbounded - a hung local LLM would lock the executor thread indefinitely. Wrap the call in a daemon thread with `worker.join(timeout=N)`. On timeout the call returns with data["answer"] == "", data["error"] describing the timeout, and a meta["warnings"] entry; a warning is logged so the leak is visible. Threshold is configurable via `GraphStore(reader_timeout_seconds=...)`, default 60s. Constructor rejects non-positive values up-front. Trade-off: Python cannot cancel a running thread, so a genuinely stuck reader leaks one daemon per call. Acceptable - daemons don't block process exit, and demanding cancellable readers is too much to require from callers. - README: Replaced no docs for `g.ask` / `g.write` (added in #195) with a small section showing the chat-app shape: ingestor= factory for NL writes, reader= for synthesis, the new reader_timeout_seconds knob, and a pointer to dry-run for question-shape routing through Bonsai. Verified end-to-end in container: fast reader returns cleanly; slow reader (5s) is bounded at 0.3s with proper error + warning surfaced; exception readers are caught; reader_timeout_seconds=0 raises ValueError at construction.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ANSWER's reader callable was unbounded — a hung local LLM blocked the executor thread forever. Now wrapped in a daemon-thread +worker.join(timeout=N). On timeout the result still comes back withdata[\"answer\"] == \"\",data[\"error\"]describing the timeout, and ameta[\"warnings\"]entry. Threshold viaGraphStore(reader_timeout_seconds=...), default 60s. Constructor rejects non-positive values.g.ask/g.write(introduced in feat(api): auto-fetch Bonsai GGUF + GraphStore.ask/write surface #195) — the chat-app shape:ingestor=factory,reader=, the new timeout knob, and how to drive question-shape routing through Bonsai viadry_run=True.Trade-off
Python cannot cancel a running thread, so a genuinely stuck reader leaks one daemon thread per call. Daemons don't block process exit and a warning is logged on each leak so it's observable. Demanding cancellable readers is too much to require from callers. Net: failures become observable (was: silent hangs).
Test plan
errorkey, no warnings.reader_timeout_seconds=0.3: returns at 0.3s withdata[\"error\"]+meta[\"warnings\"]; warning logged.data[\"error\"]populated with the exception, no timeout reported.reader_timeout_seconds=0rejected at construction withValueError.pip install graphstore[pro]UX from feat(api): auto-fetch Bonsai GGUF + GraphStore.ask/write surface #195 still works (no regression).