Skip to content

chore(selfhost): shared contract + tests for queue/adapter/vectorize backend pairs #4010

Description

@JSONbored

Context

An audit found that the self-host runtime's three swappable-backend pairs have no shared type contract, and at least two have already, concretely drifted:

  • Queue backends: DurableQueue (src/selfhost/sqlite-queue.ts:121) and PgDurableQueue (src/selfhost/pg-queue.ts:205) are two independently-declared interfaces, not one shared type. Every synchronous method on the sqlite side is Promise-wrapped on the postgres side, plus PgDurableQueue has an extra init() method the sqlite side lacks. src/server.ts:129 papers over this with a loose T | Promise<T> union type on Backend.queue — direct evidence the compiler is enforcing a looser contract than either concrete interface actually promises.
  • Storage adapters: d1-adapter.ts and pg-adapter.ts share no project-defined interface at all — both just as unknown as D1Database force-cast to Cloudflare's external ambient type, so nothing would catch either side silently changing shape.
  • Vectorize backends: qdrant-vectorize.ts, pg-vectorize.ts, and vectorize.ts (sqlite) each privately redeclare their own copy of VectorRecord/QueryOptions/Match — and vectorize.ts's QueryOptions already carries an extra returnMetadata field the other two don't have, a real, already-existing divergence.

No contract test exists for any of the three pairs (confirmed: each backend's test file is independent, none cross-imports the sibling's module or runs a shared parameterized suite).

Requirements

  1. For each pair, define one shared interface both implementations satisfy structurally (not just by convention) — reconciling the sync-vs-Promise mismatch in the queue pair (likely: make both async, since that's a strict superset and the D1/Workers path can already await trivially).
  2. For each pair, add a contract test that runs the identical assertion suite against both concrete implementations (a describe.each/parameterized harness, or two thin test files that both import a shared spec function).
  3. Fix the one already-manifested divergence (the returnMetadata field only present in vectorize.ts's QueryOptions) — decide whether it belongs on all three or none, and align them.

Deliverables

  • Shared interface definitions for each of the three pairs.
  • A contract test per pair.
  • Resolution of the returnMetadata field divergence.

Expected outcome

The next time one backend implementation changes, a mismatch with its sibling is caught by the compiler or the contract test suite — not discovered later as a silent production divergence.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions