Skip to content

spall-openapi: create the crate + a transport-agnostic request/response model #24

Description

@rustpunk

Context. The logic that turns an OpenAPI operation into an HTTP call — request building, auth injection, pagination — currently lives in the spall-cli binary, welded to reqwest (async, tokio). Other programs (e.g. the Clinker ETL engine, which uses synchronous ureq) need to reuse it. So it must move into a library with no HTTP client dependency that exchanges neutral types, and whose read contract is a lazy, auto-paginating stream of generic items — never a fully-buffered response.

Scope.

  • Create a new spall-openapi library crate that depends on spall-core's clap-free IR.
  • Define neutral, client-agnostic types: HttpRequestSpec { method, url, query, headers, body } and a response type.
  • Two-layer read contract:
    • Low-level (escape hatch): a single request → a streaming response body (Box<dyn Read>, a reader the caller owns), never a buffered Vec<u8>. For raw per-response access (saving a page verbatim, rate-limit debugging).
    • High-level (default): a lazy iterator of generic items across all pages — see spall-openapi: extract pagination strategies as transport-agnostic next-request computation #27. Each item is a generic parsed JSON value (spall's own neutral value), located per the operation's response shape. spall holds no consumer record or schema model: items are generic JSON, mapped to typed records by the consumer (e.g. Clinker, per its source-node schema).
  • No reqwest / ureq / tokio dependency in this crate.

Baseline note (post-#42). The CLI already has a structured OperationResult { status, raw: Option<Vec<u8>>, value } plus a threaded ResponseContext (the former LAST_RESPONSE global is gone). Evolve these rather than starting cold, but: (a) status is currently a reqwest::StatusCode — the neutral model must use a client-agnostic status (e.g. a u16); (b) raw + value still buffer the whole body — replace them with the two-layer streaming contract above. ResponseContext is the natural carrier for the streaming response.

Acceptance.

  • spall-openapi builds with no HTTP-client / async-runtime dependency (cargo tree confirms).
  • The low-level response delivers a large body incrementally (no forced full-buffer); a test drives a multi-hundred-MB body through the streaming reader at bounded memory.
  • The read contract carries no consumer-specific (record / schema) type — items are generic JSON values — and no reqwest type (status is neutral).

Depends on: #23.

Metadata

Metadata

Assignees

No one assigned

    Labels

    embeddingWork toward embedding spall as a library in other Rust programsenhancementNew feature or requestsize:M

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions