Provider profiles: expose a serviceTier field to pin service_tier in request bodies (OpenRouter / OpenAI-compatible routes)

[GnaneshKunal]

Problem

OpenRouter (and OpenAI's own Responses endpoints) price and schedule requests by service tier: flex halves token costs in exchange for a slower pool, priority/fast/ultrafast pay multipliers for lower latency. Today, a DeepSeek Harness deployment routed through pi-ai (@deepseek-ai/dsh-llm-pi-ai) can only influence the tier indirectly, if at all:

• OpenRouter model-id variants cover routing models (:nitro, :floor), not the service_tier request parameter, and the variants' semantics do not line up with the tiers.
• The provider profile exposes per-route headers but no request-body fields, so service_tier is unreachable from configuration.
• The underlying adapter stack already implements the concept end-to-end, so the gap is purely a configuration surface.

Because request bodies are replay/cache-sensitive state, this has to be a configuration-level field rather than a per-call option. A tier affects pricing (flex halves, priority/fast double) and latency class for the whole route; it must be pinned once per provider profile and replayed identically on every request, the same way baseURL, headers, and retryPolicy already are. A per-call override would make the same conversation produce differently-priced request bodies, and a per-model suffix would leak provider-specific spelling into route-agnostic configuration.

Evidence and parity

1. pi-ai implements it. The installed @earendil-works/pi-ai (1.1.0) implements service tiers for its openai-responses API: the options type accepts serviceTier?: ResponseCreateParamsStreaming["service_tier"], writes params.service_tier = options.serviceTier into the request body, applies the flex/priority/fast cost multipliers to usage pricing, and resolves the served tier from the response ([pi-ai dist: api/openai-responses.js, api/openai-responses-shared.js]). The accepted value union is OpenAI's ServiceTier — auto, default, flex, scale, priority, fast, ultrafast (plus null).

2. opencode (same pi-ai lineage) already ships this as a config overlay. opencode exposes provider/model/variant-scoped body overlays merged into request bodies and documents "body": { "service_tier": "flex" } as the canonical example:

   • https://dev.opencode.ai/v2/docs/providers
   • https://github.com/anomalyco/opencode/blob/dev/packages/core/src/config/plugin/provider.ts

   So there is upstream precedent for treating the tier as configuration, not a call-site concern.

Proposed API

Add an optional, schema-validated serviceTier to PiAiProviderProfile (the providers dict in @deepseek-ai/dsh-llm-pi-ai config). Omitted means never send service_tier — the provider's own tier selection stays untouched (its served tier competes with a pinned one per the provider's own pricing rules).

export const SERVICE_TIERS = ['auto', 'default', 'flex', 'scale', 'priority', 'fast', 'ultrafast'] as const

/** One value of {@link SERVICE_TIERS}. */
export type PiAiServiceTier = (typeof SERVICE_TIERS)[number]

export interface PiAiProviderProfile {
  // ...existing fields...

  /**
   * Service tier pinned for every request on this route, sent as the
   * top-level `service_tier` request-body field of the OpenAI-compatible
   * protocols (chat completions and Responses) this route's models speak;
   * every other protocol ignores it. The value rides pi-ai's
   * `samplingParams` request-body passthrough because pi-ai's own
   * `SimpleStreamOptions` mapping drops a dedicated tier option, so a tier a
   * route names is the wire value, not an option pi-ai re-interprets.
   * Omission never sends the field, leaving the provider's own tier selection
   * — its served tier competes with the pinned one per the provider's pricing
   * rules — untouched.
   */
  serviceTier?: PiAiServiceTier
}

And the schema entry, in the same file (it renders as a form control in the Models settings automatically):

serviceTier: z.union(SERVICE_TIERS),

Example configuration:

- name: '@deepseek-ai/dsh-llm-pi-ai'
  config:
    providers:
      openrouter:
        apiKeyEnv: OPENROUTER_API_KEY
        serviceTier: flex

Implementation note (verified against the installed pi-ai 1.1.0)

The obvious passthrough — copying serviceTier into the SimpleStreamOptions object the adapter hands pi-ai — is a silent no-op. pi-ai's openai-responses streamSimple mapping builds its request from buildBaseOptions(...), which does not copy a serviceTier key (only the full stream/OpenAIResponsesOptions path honors it, and the adapter calls streamSimple). The wire seam pi-ai does honor on both OpenAI-compatible protocols is samplingParams: its entries are merged into the request body after the named fields ("Only applied by OpenAI-compatible adapters (completions, responses, Azure responses); other APIs ignore it" — pi-ai's own JSDoc). So the adapter maps the configured tier as:

...profile.serviceTier === void 0
  ? {}
  : { samplingParams: { service_tier: profile.serviceTier } },

This keeps the field config-level, honors omission exactly, works for chat-completions and Responses routes, and degrades safely on every other protocol (anthropic-messages etc. ignore it).

Compat notes

• Omitted = zero behavior change. No service_tier key is written to any request body, and resolution keeps the field absent (distinguishable from any sent value).
• The endpoint owns tier vocabulary. pi-ai passes the value through unvalidated, so a tier the route's endpoint does not serve fails at the provider, naming the field — the adapter does not second-guess per-provider vocabularies.
• Completions limitation. This works better than a dedicated pi-ai option would have: because it rides samplingParams, chat-completions routes get service_tier too (it is a valid OpenAI chat-completions parameter). The limitation that remains is on pi-ai's side: no applyServiceTierPricing-style cost multiplier is applied on the completions path, so usage costs on completions routes are not tier-adjusted; the wire field is still sent correctly. A dedicated pi-ai streamSimple option would be the cleaner long-term home; the samplingParams mapping is pinned to the installed 1.1.0 behavior and should be re-visited on pi-ai upgrades (this repo already carries a pi-ai patch and a drift-gate convention for exactly this).

Test/build evidence

• pnpm run typecheck — 5/5 gates pass.
• pnpm run build — 9/9 gates pass (host compiler, Typert, Host bundle, Desktop bundle, client compiler, Client bundle, Web build, aggregate checks).
• Package test suite vitest run packages/llm/llm-pi-ai/ — 18 files, 426 tests, all pass, including:
  • schema boundary: accepts all seven tier values, rejects garbage (turbo) at the write path, keeps an omitted tier absent after resolveProfiles;
  • profileOptions mapping: configured tier appears as samplingParams: { service_tier } on the options handed to pi-ai, absent entirely when unset;
  • wire-level openai-responses mock-server assertions: pinned flex produces service_tier: 'flex' in the request body with a clean stop finish; unset produces no service_tier key at all.
• Runtime proof against the built bundle (lib/index.js): Config accepts serviceTier: 'flex' and materializes it, rejects turbo, and PiAiAdapter resolves the profile carrying it.
• Repo lint (oxlint, staged config) — 0 warnings, 0 errors on all changed files.
• Boot check: dsh web --no-open --port 30814 --host 127.0.0.1 from this checkout serves HTTP 200 (via the boot-issued trust-token link; unauthenticated requests get the app's expected 401 fence), then shut down cleanly.

---

Update — client UI select added (Oct 15)

Browser verification surfaced a gap the first patch missed: the Settings → Models provider card is hand-written per adapter family (packages/client/ui-settings-models/src/client/ProviderEditor.tsx curates its fields; "the card names only the fields it can see"), so a new schema field alone is stored but never rendered. Follow-up commit c90ee87ed0 adds the missing client half:

• Service tier <select> on the pi-ai family's Customized settings fold (all pi-ai cards — a catalog route's profile names no route-level api, so gating on protocol would hide the control exactly on OpenRouter-style routes; the hint discloses that non-OpenAI protocols ignore the field)
• Choices are read from the rehydrated schema union (unionChoices() in store.ts) so client and host can't drift; Not set emits an unset path op, a pick emits a set op
• en+zh locales, package README, and card tests (choices render, stored-tier preselect, set/unset ops)

Verification: typecheck 5/5 · build 9/9 · oxlint clean · ui-settings-models 351/351 · llm-pi-ai 426/426 · visually confirmed on a live build (select renders preselected from a configured route).

Latest patch: refreshed single-commit patch at https://gist.github.com/GnaneshKunal/f80e41d36008a83967d8293ef7ad0db9 (holds both the git-am patch and the full diff) (see also the follow-up comment below).

[GnaneshKunal]

Single combined patch — host + client in one commit (c90ee87ed0):

• git am-able format-patch + full service-tier.diff, both files in one gist: https://gist.github.com/GnaneshKunal/f80e41d36008a83967d8293ef7ad0db9

What it covers:

• packages/llm/llm-pi-ai — serviceTier?: auto | default | flex | scale | priority | fast | ultrafast on PiAiProviderProfile (schema field + request-body passthrough via pi-ai's samplingParams seam), tests, READMEs en+zh, config catalog
• packages/client/ui-settings-models — Service tier select on the pi-ai provider card's Customized settings fold; choices read from the rehydrated schema union (no client/host drift), Not set emits an unset path op; en+zh locales, package README, card tests (choices render, stored-tier preselect, set/unset ops)

Verification: typecheck 5/5 · build 9/9 · oxlint clean · ui-settings-models 351/351 · llm-pi-ai 426/426 · the rendered select was visually confirmed on a live build (preselected from a configured route).

[GnaneshKunal]

Superseded — everything above is consolidated into the single combined patch (host + client, one commit). The gist linked in the first comment holds both the git am patch and the full diff.