Skip to content

[@effect/ai-openai-compat] Invalid message ordering for parallel tool calls #6702

Description

@observerw

What version of Effect is running?

4.0.0-beta.101

What steps can reproduce the bug?

For strict openai-compat endpoint (e.g. https://api.deepseek.com/v1/chat/completions), tool call and output messages must be adjacent:

[
  {
    "role": "assistant",
    "tool_calls": [{ "id": "call_1" }, { "id": "call_2" }]
  },
  { "role": "tool", "tool_call_id": "call_1" },
  { "role": "tool", "tool_call_id": "call_2" }
]

rather than consecutive:

[
  { "role": "assistant", "tool_calls": [{ "id": "call_1" }] },
  { "role": "assistant", "tool_calls": [{ "id": "call_2" }] },
  { "role": "tool", "tool_call_id": "call_1" },
  { "role": "tool", "tool_call_id": "call_2" }
]

otherwise a error will occurred. e.g. from Deepseek:

An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'.
(insufficient tool messages following tool_calls message)

Reproduction:

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai-compat"
import { Config, Effect, Layer, Schema } from "effect"
import { Chat, Tool, Toolkit } from "effect/unstable/ai"
import { FetchHttpClient } from "effect/unstable/http"

const Tools = Toolkit.make(
  Tool.make("first", {
    description: "Return the first value",
    parameters: Schema.Struct({}),
    success: Schema.String
  }),
  Tool.make("second", {
    description: "Return the second value",
    parameters: Schema.Struct({}),
    success: Schema.String
  })
)

const ToolsLayer = Tools.toLayer(
  Effect.succeed(Tools.of({
    first: () => Effect.succeed("first"),
    second: () => Effect.succeed("second")
  }))
)

const ClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("DEEPSEEK_API_KEY"),
  apiUrl: Config.succeed("https://api.deepseek.com/v1")
}).pipe(Layer.provide(FetchHttpClient.layer))

const program = Effect.gen(function*() {
  const tools = yield* Tools
  const chat = yield* Chat.fromPrompt(
    "Call both `first` and `second` in parallel."
  )

  yield* chat.generateText({
    prompt: [],
    toolkit: tools,
    toolChoice: "required"
  })

  yield* chat.generateText({
    prompt: [],
    toolkit: tools
  })
})

program.pipe(
  Effect.provide([
    ToolsLayer,
    ClientLayer,
    OpenAiLanguageModel.model("deepseek-v4-flash")
  ]),
  Effect.runPromise
)

What is the expected behavior?

Chat.chat always produce adjacent tool call messages. This will help passing checks of some strict endpoints while not interfere other permissive endpoints.

What do you see instead?

No response

Additional information

toChatMessages in packages/ai/openai-compat/src/OpenAiLanguageModel.ts converts each consecutive function_call independently. It should combine them into one assistant message before emitting their outputs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions