[Feature] Support TypeSafe Jev (System One decisions API) #6045
Gyarados4157
started this conversation in
Ideas
Replies: 1 comment
|
One proxy behavior worth locking down is avoiding silent failover from one Jev model or provider to another after a request is accepted. In Jev Social we persist the model identity, candidate set, and confidence beside the browser evidence; a cross-provider retry can make calibration and audit history ambiguous even when the response schema matches. It may be safer to retry only transport failures against the same model, then surface the actual provider and model plus the upstream request ID in response metadata. The caller can still own the low-confidence or unavailable-model fallback explicitly. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Motivation
TypeSafe's Jev (https://typesafe.ai, docs: https://docs.typesafe.ai/api) is a decision-only model: no chat completions, no streaming text. It is increasingly used in agent pipelines (routing, classification, scoring, moderation gates) alongside the chat models CLIProxyAPI already proxies.
Currently there is no way to put Jev behind CLIProxyAPI:
*-api-keyvariants). No typesafe/jev channel exists.openai-compatibilityupstream requires an OpenAI-compatiblePOST /v1/chat/completionsendpoint, which Jev does not speak.Jev API shape (for reference)
https://api.typesafe.aihttps://api.typesafe.ai/openapi.json, ReDoc at/redocPOST /v1/systemonewith{ state, questions, model }, where questions are typednoul/choice/score; response carries per-question probability + confidence + token usage.GET /v1/modelsTYPESAFE_API_KEY, created at console.typesafe.ai)typesafe/jev, Vercel AI Gatewaytypesafe-ai/jev), but none of them is/v1/chat/completionseither.Key point: Jev is not a chat model — there is no text generation and no streaming — so protocol translation from chat/completions does not make sense. What is needed is a passthrough provider, not a chat adapter.
Proposal
Add a new provider type (e.g.
typesafe-api-key, mirroringgemini-api-key/claude-api-key/codex-api-key), with:base-url(defaulthttps://api.typesafe.ai),api-key-entries,models(name/alias) soGET /v1/modelscan list Jev models,POST /v1/systemone(andGET /v1/models) to the configured upstream, with per-key load balancing / cooldown / retry consistent with other-api-keyproviders,Out of scope (at least initially): translating Jev decisions into chat completions; gateway-specific variants (OpenRouter/Cloudflare/Vercel) could come later via
base-urloverride if the request shape matches.Use case
Single local endpoint + single key management for both generative chat models and Jev decision calls in agent harnesses (e.g. Pi / Claude Code / Codex agents calling Jev MCP tools through the proxy for usage accounting, failover, and local routing).
Happy to provide a sample
state + questionsrequest/response pair or test against the live API if helpful.All reactions