fix: give GeminiBackend a request timeout so a stuck socket can't hang forever

[duckyquang]

Summary

GeminiBackend.complete called generate_content with no request deadline, so a wedged socket
blocked the call forever. That's what froze the long MedMCQA batch runs — a worker sat at 0% CPU
for ~3.5h with its cache frozen, unrecoverable short of a kill. google-genai takes its timeout in ms
via HttpOptions, so I added a timeout param to GeminiBackend (seconds, default 60) and build
the client with it: a hung request now raises after the deadline instead of never returning.

The important part is what this unblocks. Every arm script already wraps GeminiBackend in
RetryBackend, which retries on any exception — so they weren't missing a retry, they were stuck on
a call that never raised. With a deadline, a wedged request raises and the existing RetryBackend
retries it. So this one change makes the whole battery self-healing against socket hangs, no
per-script edits needed.

Result

Verified against the live API: a 1 ms timeout raises ConnectTimeout in 0.03s (no hang), a normal
60s call is unaffected. Full suite green (710 passed), including new tests for the ms conversion,
the finite default, and real-client construction with the timeout wired in.

Notes

• The one Gemini construction that is not RetryBackend-wrapped is build_backend in
  runner.py (the benchmaxxing run CLI path, on the feat: benchmaxxing run entry point for the stage runners #251 stack, not on main). Worth wrapping once
  this lands and the stack rebases — doing it before the timeout exists would be a no-op, since
  RetryBackend can't retry a call that hangs.
• timeout=None disables the deadline.

[sebasmos]

Re-verified 31 Jul at d7c17c0b. The timeout works; the dependency floor still does not.

I proved the mechanism rather than reading it. Pointing the backend at an unroutable address:

setting	http_options.timeout	behaviour
timeout=4.0	4000	raises ConnectTimeout after 4.0s
timeout=None	None	still hanging when killed at ~70s

So the original finding is confirmed end to end: timeout=None disables the httpx deadline rather than falling back to a client default, and your fix genuinely closes it. The stall that froze a 3.5h run is a real bug and this is the right fix for it.

The one blocker is unchanged. pyproject.toml:25 is still models = ["google-genai>=0.3", ...]. I checked the actual google_genai-0.3.0 wheel rather than inferring: types.py has no HttpOptions at all. It exists only as a TypedDict in google/genai/_api_client.py, carrying base_url, api_version, headers and response_payload, with no timeout field. In 1.0.0 types.HttpOptions exists as a BaseModel with timeout. So on the floor this package declares, types.HttpOptions(timeout=ms) is an AttributeError, and a legal pip install produces a backend that cannot construct.

One line: google-genai>=1.0.

A second thing worth fixing while you are in there, and it answers @Agastya191's objection properly. tests/test_gateway.py::test_gemini_backend_builds_a_real_client_with_the_timeout does now assert the wiring (_http_options.timeout == 45000), which is exactly what was asked for. But it opens with pytest.importorskip("google.genai"), and google-genai sits in the models extra, not dev (pyproject.toml:27). There is no .github/workflows in this repo. It passed in my environment only because 1.39.1 happens to be installed; on a stock pip install -e .[dev] the test silently skips and the guard is gone precisely where CI would run it.

Either add google-genai to the dev extra, or assert the wiring against a stub so the test does not depend on the optional dependency being present.

Merges clean onto current main, 1081 passed, 7 skipped. Holding only on the floor bump.

[Agastya191]

Nice fix, duckyquang. The deadline is the right call, _timeout_ms keeping the seconds-facing API while google-genai gets milliseconds is clean, and the docstring note that the raised error is transient so RetryBackend picks it up saves the next reader a trip through the wrapper.

One problem is that none of the three new tests in test_gateway.py observe the http_options actually reaching the client. They assert self.timeout, which is a plain attribute set before the SDK branch and unaffected by the wiring, and test_gemini_backend_builds_a_real_client_with_the_timeout only checks that same attribute plus _client is not None, then importorskips away entirely since google-genai is not in the dev env. I deleted the ms / http_options / genai.Client(http_options=...) block outright and all 19 gateway tests still passed, so the regression this PR exists to prevent is not pinned by anything.

You have options, but I would monkeypatch genai.Client with a recorder and assert it received http_options.timeout == 60000, which tests the wiring rather than the attribute.

[duckyquang]

@sebasmos thanks for tracing the httpx timeout=None mechanism — that's exactly it, and it's a cleaner statement of why the hang was real than mine. All three fixed in d7c17c0:

1. The real-client test now asserts be._client._api_client._http_options.timeout == 45000, so it fails if the http_options= wiring is dropped. I checked the way you did — deleting the argument now fails the test instead of passing silently.
2. build_backend (the --backend gemini path) is wrapped in RetryBackend(tries=5, backoff=3.0), matching the arm scripts. You're right it's live on main now that feat: benchmaxxing run entry point for the stage runners #251 merged, and it was the one unwrapped production construction — so this converts that path from hard-fail to retry, which is the self-healing the summary claimed. Added a test.
3. _timeout_ms now raises on a non-positive timeout instead of mapping it to None; None stays the explicit disable, so a timeout=-60 typo can't quietly reinstate the unbounded behaviour.

Rebased onto main; full suite green locally. Default stays 60s.

[duckyquang]

Pushed f3ca20b, covers both of you.

• google-genai floor -> >=1.0. Confirmed the 0.3 wheel has no types.HttpOptions with a timeout field, so a clean install built a backend that couldn't construct. @sebasmos
• Wiring test: dropped the importorskip. It now stubs the google-genai modules and asserts genai.Client actually receives http_options.timeout, so it runs on a stock pip install -e .[dev] instead of skipping. @Agastya191 — your "delete the http_options block and tests still pass" check now fails; I sabotaged it the same way to confirm. Also added a test pinning the timeout=None branch (has to pass http_options=None, not HttpOptions(timeout=None)).

Mind another look when you get a sec?

[sebasmos]

Approving. Both objections resolved, and I verified the test is not vacuous rather than taking your word for it.

• pyproject.toml:25 is now google-genai>=1.0. That was the blocker: types.HttpOptions does not exist in the 0.3 wheel at all, so a legal install built a backend that could not construct.
• The importorskip is gone. test_gemini_backend_wires_the_timeout_into_the_client now stubs the google-genai modules and asserts genai.Client actually receives http_options.timeout, so it runs on a stock pip install -e .[dev] instead of skipping exactly where CI would live.

Sabotage check. I removed http_options=http_options from the genai.Client(...) call at gateway.py:162, which is the realistic regression, and got:

FAILED tests/test_gateway.py::test_gemini_backend_wires_the_timeout_into_the_client
1 failed, 19 passed, 1 skipped

So the guard catches the thing it exists to catch. My first attempt at that sabotage patched mid-expression and produced a collection error instead, which proved nothing; worth saying because a broken-module "failure" is not evidence of a working test.

Suite merged onto current main: 1118 passed, 7 skipped.

Also good: pinning the timeout=None branch to pass http_options=None rather than HttpOptions(timeout=None) is the right call, since the whole original bug was that None there disables the deadline rather than falling back to a default.

Nice fix. The stall that froze a 3.5h run was real and this closes it properly.