Skip to content

v0.4.0 — tool calling end-to-end

Latest

Choose a tag to compare

@lordbasilaiassistant-sudo lordbasilaiassistant-sudo released this 25 Apr 20:19
· 6 commits to main since this release
cd43f11

Tool calling

The tools field is no longer silently dropped. tool_calls round-trips end to end on the public Worker (https://keylessai.thryx.workers.dev/v1) and the local proxy.

Added

  • OpenAI tool callingPOST /v1/chat/completions with a tools array now returns message.tool_calls (non-stream) or delta.tool_calls SSE deltas (stream). tool_choice, parallel_tool_calls, and role: "tool" reply messages are all forwarded through the pipeline. finish_reason correctly reports "tool_calls" when the model emits a call.
  • Provider capability flags — every provider exports capabilities = { tools: bool }. Pollinations + ApiAirforce both true; Pollinations-GET + Yqcloud false. Custom providers registered via registerProvider() default to false for safety.
  • Tool-aware failover — when a request includes tools, the router filters FAILOVER_ORDER to providers that advertise capabilities.tools. If none qualify, throws ToolsUnsupportedError (mapped to a 400 with code: "tool_calls_unsupported") instead of silently degrading to a non-tool provider.
  • providerSupportsTools(id) + ToolsUnsupportedError exported from the package surface.
  • Tool schema validation (src/server/validate.js) — tools.length ≤ 128, function.name ≤ 64 chars + charset [a-zA-Z0-9_-]+, tool_choice shape ("auto" | "none" | "required" | {type, function}), parallel_tool_calls boolean.
  • examples/tool-calling.js — runnable two-turn round-trip with the OpenAI Node SDK.
  • 27 new tests: 9 happy-path (test/tools.test.mjs) + 18 adversarial (test/tools.extreme.test.mjs) — char-by-char streaming, parallel tool calls, prototype-pollution payloads, mid-stream errors, cache poisoning attempts, all-providers-circuit-open. 127/127 passing.

Changed

  • Cache bypass for tool-bearing requests — proxy + Worker skip defaultCache entirely when body.tools is present. Tool-call payloads are inherently non-idempotent (each call_id participates in a turn-by-turn round trip with the client).
  • Worker version bumped to 0.4.0 — visible at GET /health.

Fixed

  • Pollinations + ApiAirforce streamers previously parsed delta.tool_calls from the upstream SSE but discarded it. They now emit {type: "tool_call_delta", index, id?, name?, arguments?} chunks that propagate through the router untouched.

Drop-in usage

import OpenAI from "openai";
const client = new OpenAI({
  baseURL: "https://keylessai.thryx.workers.dev/v1",
  apiKey: "not-needed",
});
const res = await client.chat.completions.create({
  model: "openai-fast",
  messages: [{ role: "user", content: "What's the weather in NYC?" }],
  tools: [{ type: "function", function: { name: "get_weather", parameters: { type: "object", properties: { city: { type: "string" } } } } }],
});
console.log(res.choices[0].message.tool_calls);

Full diff: v0.3.0...v0.4.0