[Plugin preview] DSH Provider Passport — request-dialect preflight for custom providers

[ArmyWas]

Hi, I’m sharing the public preview of DSH Provider Passport, an independent community plugin for custom OpenAI Chat Completions providers.

A generic “OpenAI-compatible” connectivity check can succeed while the first real Harness task still fails because a gateway rejects a request-shape detail such as developer vs system, max_completion_tokens vs max_tokens, store, reasoning_effort, or stream_options.include_usage.

The plugin adds a review-first preflight:

• runs only after explicit confirmation;
• sends at most 10 fixed requests, with at most 1 output token each;
• proposes the smallest observed set of existing Harness compat fields;
• applies changes only to the selected model;
• verifies the proposal through the real Harness LLM runtime;
• preserves cancellation, automatic recovery, and rollback.

Evidence before preview

• 9 automated unit/contract tests;
• packaged install → probe → apply → real-runtime verify → rollback lifecycle on DSH 0.1.1-rc.2 and 0.1.2-alpha.4;
• GitHub CI on Windows/Ubuntu with Node.js 22/24;
• clean reinstall and lifecycle verification from the public npm package.

This is intentionally a preview. Deterministic tests prove the mechanism, but not how often real third-party, enterprise, and self-hosted gateways need each compatibility field. I’m looking for opt-in, redacted reports rather than adding speculative features.

• Repository: https://github.com/ArmyWas/dsh-provider-passport
• Preview release: https://github.com/ArmyWas/dsh-provider-passport/releases/tag/v0.1.0-preview.1
• npm: https://www.npmjs.com/package/dsh-provider-passport
• Compatibility report: https://github.com/ArmyWas/dsh-provider-passport/issues/new?template=compatibility-report.yml

Install:

npx @deepseek-ai/dsh plugin --profile web add dsh-provider-passport@preview

The copied community report removes endpoint and model identity, credentials, headers, request text, response bodies, and model output. Nothing is uploaded automatically. Please never post API keys or private endpoint details.

---

中文简介：这个插件用于在真实任务开始前，发现自定义 OpenAI Chat Completions 接口与 Harness 请求格式之间的差异。它只在用户确认后进行有限探测，先展示最小兼容配置，再写入单个模型，并通过真实 Harness 运行时验证；失败时自动恢复，也支持回滚。目前希望收集经过脱敏的真实接口结果，以判断问题频率和下一步方向，而不是凭空增加功能。

Independent community project; not affiliated with or endorsed by DeepSeek.

[argszero]

Verified the probe surface against 4e84901e64 (dsh 0.1.2-alpha.4). Strong fit — and there are three things the "propose the smallest compat set" step needs to get right so it does not generate proposals that fail to apply.

1. All five named shape details are expressible as alpha.4 compat fields — on openai-completions. The offered set of the completions gate (packages/llm/llm-pi-ai/src/catalog.ts:217-242) maps 1:1 onto your examples:

• developer vs system → supportsDeveloperRole
• max_completion_tokens vs max_tokens → maxTokensField
• store → supportsStore
• reasoning_effort → supportsReasoningEffort
• stream_options.include_usage → supportsUsageInStreaming

So "propose the smallest observed set of existing Harness compat fields" is well-defined for your target class. Bonus stability property: each gate is a drift check (as const satisfies Record<keyof OpenAICompletionsCompat, …> at :242), so the proposal vocabulary is exactly the installed pi-ai's declared compat surface — it cannot silently lag the shipped version.

2. The proposal must be keyed by each model's resolved api — the offered set is protocol-specific. The same detail is not configurable on every protocol:

• openai-responses gate (:245-254) offers only supportsDeveloperRole, supportsStrictMode, supportsLongCacheRetention — no store, no reasoning_effort, no maxTokensField.
• anthropic-messages gate (:257-267) offers a different set entirely.

If the probe proposes a field the model's resolved protocol does not offer, the config is refused at load with a specific error — "sets compat X, which no wire protocol declares; the configurable switches are …" (assertOfferedCompatFields, catalog.ts:505-530). A gateway whose models resolve to openai-responses would receive a proposal that cannot apply. Worth a guard: run the "smallest set" search over the offered fields of the model's resolved api, not over the union of all gates.

3. Withheld fields are the trap. openRouterRouting, vercelGatewayRouting, zaiToolStream, sessionAffinityFormat, deferredToolsMode, … are withhold dispositions on the completions gate (:235-241). They are declared by pi-ai but refused on configuration with a different error — "not configurable here: pi-ai's installed catalog sets it for the vendors that need it". These are exactly the fields a probe is most likely to want to propose (a gateway that 500s without a routing header), so the plugin should surface them as "recognized but not settable — here is why", not as a proposal that silently fails to apply. That is also the right shape for your redacted reports: withheld-field frequency is the signal maintainers need to re-classify a field as offered.

Family linkage — this is the runtime half of the static compat gates:

• supportsDeveloperRole has a known static-default problem (community threads llm-pi-ai should expose supportsDeveloperRole for custom OpenAI-compatible gateways #3076/llm-pi-ai: expose `supportsDeveloperRole` in `PiAiCompatProfile` (reasoning models break on endpoints that reject the `developer` role) #551: pi-ai defaults it true on standard-looking URLs → custom gateways 400 on developer). Your probe observes the 400 and proposes the correction — that is the runtime complement of the static gate, and exactly the "deployment intent" side of the withhold-vs-offer discriminator discussed in 建议在 dsh-llm-pi-ai 开放 OpenRouter 路由锚定（compat.openRouterRouting） #5349 (static vendor capability vs. deployment intent).
• Protocol axis: your probe is completions-only, but one baseURL can serve anthropic-messages / openai-responses models (cf. Custom provider defaults to Chat Completions only, missing Anthropic / Response compatibility — some models undiscoverable #4947 — opencode-zen serves 25 models across 3 protocols). A model that answers only on /v1/responses fails the completions probe exactly like a compat mismatch would. Adding a per-model protocol probe (one 1-token request to each candidate endpoint, classified by which answers) would distinguish "wrong wire protocol" from "right protocol, wrong shape" — and feed the per-model api direction directly.

The opt-in redacted-report collection is the right call. Given the drift-gate design, the offered set is stable per DSH release, so cross-gateway frequency data is precisely what the maintainers need to decide which withheld fields become offered — and which compat fields deserve a probe in the first place.

Nice work — probe → propose → real-runtime verify → rollback is the right lifecycle. Happy to sanity-check the field mapping against a specific DSH release in the thread if that helps.

[ArmyWas]

Thanks @argszero — I used your source-level review as the release gate and have shipped v0.1.0-preview.2.

The safety changes follow the boundary you identified:

• A route is now eligible only when its editable settings explicitly declare api: openai-completions. Omitted APIs and other protocols are excluded before probing; the packaged E2E asserts the excluded route receives zero requests.
• Proposals are hard allow-listed to the five reviewed Completions fields: maxTokensField, supportsDeveloperRole, supportsStore, supportsReasoningEffort, and supportsUsageInStreaming.
• Catalog-withheld fields are never proposed or written. A Harness atomic settings-validation refusal is returned as a redacted no-write result.
• blocked now records base-request-shape-unresolved and explains that the cause may be another wire protocol or provider-managed behavior.
• I deliberately did not auto-switch to Responses/Anthropic/vendor routes from a failed Completions request; multi-protocol discovery remains a separate experiment.

Validation completed:

• 11/11 unit and contract tests.
• The public npm artifact dsh-provider-passport@0.1.0-preview.2 completed install → probe → apply → real Harness verify → rollback on DSH 0.1.1-rc.2 and 0.1.2-alpha.4.
• The ambiguous catalog-inherited route sent 0 probe requests in both runs.
• The real DSH Web card was manually checked for the skip notice, apply verification, and one-click rollback.
• GitHub CI passed on Windows/Linux with Node 22/24: https://github.com/ArmyWas/dsh-provider-passport/actions/runs/33583367703

npm: https://www.npmjs.com/package/dsh-provider-passport/v/0.1.0-preview.2

If you have time, I would still value a sanity-check of this exact interpretation against the installed catalog behavior. No obligation — your first review already prevented a real unsafe edge case. Thank you.