OpenAI Compatible Cache Matrix

OpenAI Compatible Cache Matrix

ChatHub's built-in openaicompatible provider supports both /v1/chat/completions and /v1/responses, but compatible gateways do not all use the same cache hints. Use this page as a settings checklist.

Quick Preset Checklist

Backticked values are dropdown/select values. Built-in presets hide the detailed verified matrix; switch to Custom to expand every cache and Responses-parameter option for provider-probe testing.

apikl.ai

Use this when the provider-probe confirms the apikl.ai behavior.

UI setting	Value
OpenAI-compatible API route	Responses API
OpenAI-compatible cache preset	apikl.ai
Hidden preset matrix	Chat prompt_cache_key + Session_id; Responses prompt_cache_key: Auto-generate, no Session_id, store: Default; omit max_tokens, omit max_output_tokens, omit truncation, send verbosity as text.verbosity

pptoken.org

Use this for the legacy ChatHub Responses state-cache behavior.

UI setting	Value
OpenAI-compatible API route	Responses API
OpenAI-compatible cache preset	pptoken.org
Hidden preset matrix	Chat cache off; Responses prompt_cache_key: Auto-generate, no Session_id, store: true; optional Responses parameter fields omitted by default

Custom

Use this when the provider-probe confirms a different combination.

UI setting	How to choose
OpenAI-compatible API route	Select Responses API to call /v1/responses; select Chat Completions to call /v1/chat/completions.
OpenAI-compatible cache preset	Custom
Chat prompt_cache_key	Check only if the chat probe confirms body prompt_cache_key helps.
Chat Session_id	Check only if the chat probe confirms the Session_id header helps.
Responses prompt_cache_key	Select Auto-generate only if the Responses probe confirms body prompt_cache_key helps; otherwise select Do not send.
Responses Session_id	Check only if the Responses probe confirms the Session_id header helps.
Responses store	Select the value confirmed by the probe: Default, true, or false.
Responses max_tokens	Check only if the provider accepts max_tokens on /v1/responses.
Responses max_output_tokens	Check only if the provider accepts max_output_tokens on /v1/responses.
Responses truncation	Select Do not send, auto, or disabled based on the probe.
Responses verbosity	Select Do not send, text.verbosity, top-level verbosity, or both based on the probe.

Selecting a built-in preset writes the full hidden matrix and defaults the route to Responses API. Selecting Custom expands the matrix; any manual matrix or parameter edit keeps the preset as Custom.

What Each Setting Sends

UI setting	API path	What ChatHub sends
OpenAI-compatible API route = Responses API	/v1/responses	Routes requests to Responses API.
OpenAI-compatible API route = Chat Completions	/v1/chat/completions	Routes requests to Chat Completions API.
Chat prompt_cache_key checked	/v1/chat/completions	Body prompt_cache_key: <derived stable key>.
Chat Session_id checked	/v1/chat/completions	Header Session_id: <derived stable key>.
Responses prompt_cache_key = Auto-generate	/v1/responses	Body prompt_cache_key: <derived stable key>.
Responses prompt_cache_key = Do not send	/v1/responses	No body prompt_cache_key.
Responses Session_id checked	/v1/responses	Header Session_id: <derived stable key>.
Responses store = Default	/v1/responses	No store field is sent.
Responses store = true	/v1/responses	Body store: true.
Responses store = false	/v1/responses	Body store: false.
Responses max_tokens checked	/v1/responses	Body max_tokens from the model max-output setting.
Responses max_output_tokens checked	/v1/responses	Body max_output_tokens from the model max-output setting.
Responses truncation	/v1/responses	Sends no field, truncation: "auto", or truncation: "disabled".
Responses verbosity	/v1/responses	Sends no field, text: { verbosity }, top-level verbosity, or both.

Probe Workflow

Use the repo-local chathub-provider-probe skill and dedicated probe environment variables:

export PROBE_OPENAICOMPATIBLE_API_KEY='...'
export PROBE_OPENAICOMPATIBLE_PROXY_URL='https://provider.example.com/v1'
export PROBE_OPENAICOMPATIBLE_MODEL='gpt-5.5'

Interactive cache detection must be round-bounded:

node skills/chathub-provider-probe/scripts/probe-openai-compatible.mjs --phase cache-round --runs 6

Rules:

• Use six runs for the first cache round and later interactive rounds.
• Test one strategy per unconfirmed endpoint per round.
• Pause after every round and ask the user to verify the request IDs in provider dashboard/logs.
• Keep /v1/chat/completions and /v1/responses status separate.
• If one endpoint is confirmed, lock that strategy and continue only the unconfirmed endpoint.
• Use cache-matrix only for non-interactive exhaustion after explicit approval.

Runtime Notes

• Provider cache config is stored as openAICompatCache; Responses parameter compatibility is stored as openAICompatResponsesParams.
• Legacy responseStateMode: "provider" normalizes to the pptoken.org Responses behavior when no matrix exists.
• Runtime-only fields are stripped before upstream request bodies are sent.
• Built-in presets apply their verified Responses parameter behavior. Custom applies the expanded parameter matrix exactly, so unknown providers can be tested freely.
• Responses Session_id-only mode must not force a body prompt_cache_key.
• Cache keys are derived from stable prompt parts such as model, system/developer messages, first user message, reasoning effort, and tools.