You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context. The logic that turns an OpenAPI operation into an HTTP call — request building, auth injection, pagination — currently lives in the spall-cli binary, welded to reqwest (async, tokio). Other programs (e.g. the Clinker ETL engine, which uses synchronous ureq) need to reuse it. So it must move into a library with no HTTP client dependency that exchanges neutral types, and whose read contract is a lazy, auto-paginating stream of generic items — never a fully-buffered response.
Scope.
Create a new spall-openapi library crate that depends on spall-core's clap-free IR.
Define neutral, client-agnostic types: HttpRequestSpec { method, url, query, headers, body } and a response type.
Two-layer read contract:
Low-level (escape hatch): a single request → a streaming response body (Box<dyn Read>, a reader the caller owns), never a buffered Vec<u8>. For raw per-response access (saving a page verbatim, rate-limit debugging).
High-level (default): a lazy iterator of generic items across all pages — see spall-openapi: extract pagination strategies as transport-agnostic next-request computation #27. Each item is a generic parsed JSON value (spall's own neutral value), located per the operation's response shape. spall holds no consumer record or schema model: items are generic JSON, mapped to typed records by the consumer (e.g. Clinker, per its source-node schema).
No reqwest / ureq / tokio dependency in this crate.
Baseline note (post-#42). The CLI already has a structured OperationResult { status, raw: Option<Vec<u8>>, value } plus a threaded ResponseContext (the former LAST_RESPONSE global is gone). Evolve these rather than starting cold, but: (a) status is currently a reqwest::StatusCode — the neutral model must use a client-agnostic status (e.g. a u16); (b) raw + value still buffer the whole body — replace them with the two-layer streaming contract above. ResponseContext is the natural carrier for the streaming response.
Acceptance.
spall-openapi builds with no HTTP-client / async-runtime dependency (cargo tree confirms).
The low-level response delivers a large body incrementally (no forced full-buffer); a test drives a multi-hundred-MB body through the streaming reader at bounded memory.
The read contract carries no consumer-specific (record / schema) type — items are generic JSON values — and no reqwest type (status is neutral).
Context. The logic that turns an OpenAPI operation into an HTTP call — request building, auth injection, pagination — currently lives in the
spall-clibinary, welded toreqwest(async, tokio). Other programs (e.g. the Clinker ETL engine, which uses synchronousureq) need to reuse it. So it must move into a library with no HTTP client dependency that exchanges neutral types, and whose read contract is a lazy, auto-paginating stream of generic items — never a fully-buffered response.Scope.
spall-openapilibrary crate that depends onspall-core's clap-free IR.HttpRequestSpec { method, url, query, headers, body }and a response type.Box<dyn Read>, a reader the caller owns), never a bufferedVec<u8>. For raw per-response access (saving a page verbatim, rate-limit debugging).reqwest/ureq/tokiodependency in this crate.Baseline note (post-#42). The CLI already has a structured
OperationResult { status, raw: Option<Vec<u8>>, value }plus a threadedResponseContext(the formerLAST_RESPONSEglobal is gone). Evolve these rather than starting cold, but: (a)statusis currently areqwest::StatusCode— the neutral model must use a client-agnostic status (e.g. au16); (b)raw+valuestill buffer the whole body — replace them with the two-layer streaming contract above.ResponseContextis the natural carrier for the streaming response.Acceptance.
spall-openapibuilds with no HTTP-client / async-runtime dependency (cargo treeconfirms).reqwesttype (status is neutral).Depends on: #23.