@pymthouse/gateway-web

Minimal Node.js client for Livepeer live-runner inference: single-shot HTTP calls and HTTP persistent sessions (reserve → call → stop).

Ports the dispatch path that the Python SDK service uses: discover runners, pick by the advertised runner mode, pay a 402 challenge via a remote signer, and return the runner's JSON (with a media URL extracted). Persistent runners keep the discovery /session URL, reserve a session, POST {app_url}{endpoint}, then POST {control_url}/stop.

runInference considers both modes and dispatches per runner. Single-shot capabilities POST the discovery URL as published — endpoint is rejected. Persistent apps must pass endpoint — it is not advertised and cannot be guessed from the app id. Missing endpoint fails before reserve so you are not billed for a session that then 404s.

runInference against a persistent runner is reserve → one call → stop. Each invocation pays a full reserve; the isolate cannot reuse the session across calls. Hold a session yourself with reserveSession / callSession / stopSession when you need more than one HTTP round-trip.

If the runner returns a fal queue receipt (IN_QUEUE / status_url, no media URL) instead of blocking until completion, runInference polls status_url then fetches response_url for the remaining timeout. Queue control URLs are never treated as media. A 401/403 on the poll URL leaves the handle on result.data / statusUrl so the caller can surface it. Pass onProgress for poll ticks.

Pass onPayment({ manifestId, phase }) on runInference / reserveSession to persist payment lineage before the signer charge (prepared) and after credentials return (accepted). The callback is per-request so concurrent callers stay isolated. It never receives payment bytes or signer state. Thrown errors abort without paid orchestrator failover.

Orchestrator failover: for each capability the gateway caches up to 5 distinct orchestrators (one runner per orch, merit-ranked). On retryable runner failures (5xx, timeouts, exhausted payment retries on that orch) it automatically tries the next cached orchestrator before giving up. Single-shot runners are tried first when an app is advertised under both modes.

This package does not implement BYOC /process/request/{cap}, gRPC GetOrchestrator, protobuf, WebSocket, LV2V/trickle, or training. See docs/stream-session-handoff.md for the streaming-handoff spike.

Published to the @pymthouse npm org (not @livepeer — no npm org access there).

Install

npm install @pymthouse/gateway-web

Node 20+. Runtime dependency: undici (needed so TLS verification can be disabled per request for self-signed orchestrator/runner certs — never set NODE_TLS_REJECT_UNAUTHORIZED=0).

Usage

import { createGateway } from "@pymthouse/gateway-web";

const gw = createGateway({
  signerUrl: "https://signer.pymthouse.com",
  signerHeaders: { Authorization: `Bearer ${process.env.PYMTHOUSE_API_KEY}` },
  // discoveryUrl defaults to `${signerUrl}/discover-orchestrators`
  insecureTls: true, // runner + discovery only; signer stays verified
  timeoutMs: 600_000,
  attributionSource: "pymthouse_gateway", // gateway stack recorded on every ticket
});

const res = await gw.runInference({
  capability: "image-generation/black-forest-labs/FLUX.1-dev",
  params: { prompt: "a dragon" },
});

console.log(res.url, res.mode, res.gatewayRequestId);

// Persistent HTTP apps: pass the app path. WebSocket / trickle apps are out of scope.
const hello = await gw.runInference({
  capability: "livepeer-example/hello-world",
  endpoint: "/hello",
  params: { name: "livepeer" },
});

reserveSession / callSession / stopSession are also on the gateway (and exported) for callers who want to hold a session across multiple HTTP calls. Pass startFunding: false to reserve without starting the 3s payment loop; paymentSession.snapshot() / LivePaymentSession.fromSnapshot() move that loop to another process.

Mint the bearer the same way Console does (mintUserSignerToken via @pymthouse/builder-sdk + app signer routing). Do not point this package at signer.daydream.live.

callRunner, discoverRunners, reserveSession, callSession, and stopSession are also exported for callers who want to drive the pieces directly.

Smoke

CONSOLE_ENV=/path/to/console/.env npm run smoke

Uses Console's PYMTHOUSE_* M2M vars to mint a signer JWT, then runs createGateway().runInference() against pymthouse discovery. Default capability is vllm/qwen3-coder-30b (chat completion). Override with CAPABILITY, MODEL, and PROMPT.

Usage attribution

Every paid call sends gatewayRequestId and attributionSource in the /generate-live-payment body, which PymtHouse records on the resulting ticket rows as gateway_request_id / attribution_source. That is what lets a caller join a job it made to what that job actually cost.

runInference generates a gatewayRequestId when you do not supply one and always returns it on the result. Failures after mint attach the same id on LivepeerGatewayError.gatewayRequestId so a paid-then-failed call is still joinable. Pass your own when you need the id before the call.

attributionSource belongs on createGateway because it names the gateway stack, not the job. PymtHouse documents the vocabulary as pymthouse_gateway | python_gateway | direct_api and defaults to direct_api; this package is pymthouse_gateway.

TLS

Runner and discovery hosts often use self-signed certs. TLS verification is skipped by default for those hosts (insecureTls defaults to true). Pass insecureTls: false to verify. Signer calls always verify TLS.

Storyboard

Storyboard depends on this package from npm (@pymthouse/gateway-web). It routes sdkPost("/inference", …) through the gateway when STORYBOARD_GATEWAY_WEB=1 (pymthouse signer URL required). All other SDK endpoints stay on the SDK service.

Releasing

CI runs lint, typecheck, tests, and a pack dry-run on every PR and push to main. Pushing a v*.*.* tag publishes to npm via trusted publishing and creates a GitHub Release. See docs/RELEASING.md.