·
6 commits
to main
since this release
Tool calling
The tools field is no longer silently dropped. tool_calls round-trips end to end on the public Worker (https://keylessai.thryx.workers.dev/v1) and the local proxy.
Added
- OpenAI tool calling —
POST /v1/chat/completionswith atoolsarray now returnsmessage.tool_calls(non-stream) ordelta.tool_callsSSE deltas (stream).tool_choice,parallel_tool_calls, androle: "tool"reply messages are all forwarded through the pipeline.finish_reasoncorrectly reports"tool_calls"when the model emits a call. - Provider capability flags — every provider exports
capabilities = { tools: bool }. Pollinations + ApiAirforce bothtrue; Pollinations-GET + Yqcloudfalse. Custom providers registered viaregisterProvider()default tofalsefor safety. - Tool-aware failover — when a request includes
tools, the router filtersFAILOVER_ORDERto providers that advertisecapabilities.tools. If none qualify, throwsToolsUnsupportedError(mapped to a 400 withcode: "tool_calls_unsupported") instead of silently degrading to a non-tool provider. providerSupportsTools(id)+ToolsUnsupportedErrorexported from the package surface.- Tool schema validation (
src/server/validate.js) —tools.length ≤ 128,function.name≤ 64 chars + charset[a-zA-Z0-9_-]+,tool_choiceshape ("auto" | "none" | "required" | {type, function}),parallel_tool_callsboolean. examples/tool-calling.js— runnable two-turn round-trip with the OpenAI Node SDK.- 27 new tests: 9 happy-path (
test/tools.test.mjs) + 18 adversarial (test/tools.extreme.test.mjs) — char-by-char streaming, parallel tool calls, prototype-pollution payloads, mid-stream errors, cache poisoning attempts, all-providers-circuit-open. 127/127 passing.
Changed
- Cache bypass for tool-bearing requests — proxy + Worker skip
defaultCacheentirely whenbody.toolsis present. Tool-call payloads are inherently non-idempotent (eachcall_idparticipates in a turn-by-turn round trip with the client). - Worker version bumped to 0.4.0 — visible at
GET /health.
Fixed
- Pollinations + ApiAirforce streamers previously parsed
delta.tool_callsfrom the upstream SSE but discarded it. They now emit{type: "tool_call_delta", index, id?, name?, arguments?}chunks that propagate through the router untouched.
Drop-in usage
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://keylessai.thryx.workers.dev/v1",
apiKey: "not-needed",
});
const res = await client.chat.completions.create({
model: "openai-fast",
messages: [{ role: "user", content: "What's the weather in NYC?" }],
tools: [{ type: "function", function: { name: "get_weather", parameters: { type: "object", properties: { city: { type: "string" } } } } }],
});
console.log(res.choices[0].message.tool_calls);Full diff: v0.3.0...v0.4.0