What version of Effect is running?
effect: 3.20.0, @effect/ai: 0.34.0, @effect/ai-openrouter: 0.9.0
What steps can reproduce the bug?
If the toolkit option is removed from LanguageModel.streamText, the program runs successfully.
The failure only occurs when a toolkit is present.
import { LanguageModel, Prompt, Tool, Toolkit } from "@effect/ai";
import { OpenRouterClient, OpenRouterLanguageModel } from "@effect/ai-openrouter";
import { FetchHttpClient } from "@effect/platform";
import { Config, Effect, Layer, Schema, Stream } from "effect";
const StepFun = OpenRouterLanguageModel.model("stepfun/step-3.5-flash:free");
const SafeUser = Schema.Struct({
firstName: Schema.String,
ageRange: Schema.Literal("18-24", "25-34", "35-44", "45+"),
});
const GetUser = Tool.make("GetUser", {
description: "Get non-sensitive information about the user",
success: SafeUser,
failure: Schema.Never,
});
const GetUserTools = Toolkit.make(GetUser);
const GetUserHandlers = GetUserTools.toLayer({
GetUser: () =>
Effect.succeed({
firstName: "John Doe",
ageRange: "25-34",
}),
});
const program = LanguageModel.streamText({
prompt: Prompt.make("Use GetUser tool and tell me about the current user."),
toolkit: GetUserTools,
}).pipe(Stream.runDrain, Effect.provide(StepFun));
const OpenRouter = OpenRouterClient.layerConfig({
apiKey: Config.redacted("OPENROUTER_API_KEY"),
}).pipe(Layer.provide(FetchHttpClient.layer));
program.pipe(Effect.provide([OpenRouter, GetUserHandlers]), Effect.runPromise);
What is the expected behavior?
LanguageModel.streamText should work when a Toolkit is provided and allow the model to call tools during streaming without throwing a MalformedOutput error.
What do you see instead?
(FiberFailure) MalformedOutput: { "module": "OpenRouterClient", "method": "streamRequest", "description": undefined, "cause": (parseJson <-> (@effect/ai-openrouter/ChatStreamingResponseChunk (Encoded side) <-> @effect/ai-openrouter/ChatStreamingResponseChunk))
Relevant part of the cause:
["type"] is missing
Actual streamed tool call chunk:
[{"index":0,"function":{"arguments":"{}"}}]
Additional information
No response
What version of Effect is running?
effect: 3.20.0, @effect/ai: 0.34.0, @effect/ai-openrouter: 0.9.0
What steps can reproduce the bug?
If the
toolkitoption is removed fromLanguageModel.streamText, the program runs successfully.The failure only occurs when a toolkit is present.
What is the expected behavior?
LanguageModel.streamTextshould work when aToolkitis provided and allow the model to call tools during streaming without throwing aMalformedOutputerror.What do you see instead?
(FiberFailure) MalformedOutput: { "module": "OpenRouterClient", "method": "streamRequest", "description": undefined, "cause": (parseJson <-> (@effect/ai-openrouter/ChatStreamingResponseChunk (Encoded side) <-> @effect/ai-openrouter/ChatStreamingResponseChunk))
Relevant part of the cause:
["type"] is missing
Actual streamed tool call chunk:
[{"index":0,"function":{"arguments":"{}"}}]
Additional information
No response