Skip to content

s3-blob-store.ts's fetch calls have no timeout, breaking the repo's bounded-fetch convention #8362

Description

@JSONbored

Context

This repo has an established "bounded-fetch convention" for self-host backend adapters: every
external network call must carry an explicit timeout so an unresponsive/partitioned dependency can't
hang the request indefinitely. src/selfhost/qdrant-vectorize.ts:31-34 states this explicitly in
its own code comment and implements it via signal: AbortSignal.timeout(QDRANT_FETCH_TIMEOUT_MS) on
every fetch call, matching the pattern already used by src/selfhost/ai.ts.

src/selfhost/s3-blob-store.ts's three client.fetch() calls — get (line 59), put (line 77),
and delete (line 84) — carry no signal/timeout at all. The underlying aws4fetch client's
retries: 3 option bounds retry count, not per-attempt hang time: a misconfigured or unreachable
S3-compatible endpoint (e.g. a bad S3_ENDPOINT in self-host config, or a network partition) can hang
each of these calls indefinitely, which in turn hangs the screenshot/visual-review pipeline that
depends on this blob store.

Requirements

  • Add signal: AbortSignal.timeout(<TIMEOUT_MS>) to all three client.fetch() calls in
    src/selfhost/s3-blob-store.ts (get, put, delete), mirroring
    src/selfhost/qdrant-vectorize.ts:31-34's pattern exactly (same style of named timeout constant,
    same signal-passing approach).
  • Define a named constant for the timeout value (do not use a magic number inline) — follow the
    naming convention of QDRANT_FETCH_TIMEOUT_MS (e.g. S3_FETCH_TIMEOUT_MS), and choose a duration
    consistent with the other self-host adapters' existing timeout constants — check their actual current
    values in the repo before picking a number, don't invent one out of context.
  • Do not change aws4fetch's retries: 3 option or any other client configuration — this issue is
    scoped to adding a per-attempt timeout only.

Deliverables

  • src/selfhost/s3-blob-store.ts's get, put, and delete methods all pass an
    AbortSignal.timeout(...)-based signal to client.fetch().
  • A new named timeout constant is defined and used consistently across all three call sites.
  • test/unit/selfhost-s3-blob-store.test.ts gains a test asserting a timeout/abort on a hung
    fetch surfaces as a rejected promise (mock a fetch that never resolves and assert the call
    rejects/aborts within a bounded test-fake time, following this repo's existing pattern for
    testing AbortSignal.timeout-based behavior elsewhere, e.g. in
    test/unit/selfhost-qdrant-vectorize.test.ts if such a test exists there).

Test Coverage Requirements

This repo's Codecov patch gate is 99%+ of changed lines and branches (src/** is covered). The new
signal argument on all three call sites must be exercised by the new/updated test(s) above.

Expected Outcome

An unreachable or unresponsive S3-compatible endpoint can no longer hang get/put/delete calls
indefinitely — they now time out and surface a clear failure, consistent with every other self-host
backend adapter's documented bounded-fetch convention.

Links & Resources

  • src/selfhost/s3-blob-store.ts:59,77,84 — the three fetch calls to fix
  • src/selfhost/qdrant-vectorize.ts:31-34 — the reference pattern to mirror
  • src/selfhost/ai.ts — another adapter already following this convention

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions