Skip to content

Verify Cloudflare Workers AI via ChatOpenAI#558

Merged
brainlid merged 1 commit into
mainfrom
me-verify-cloudflare-workers-ai
May 25, 2026
Merged

Verify Cloudflare Workers AI via ChatOpenAI#558
brainlid merged 1 commit into
mainfrom
me-verify-cloudflare-workers-ai

Conversation

@brainlid

Copy link
Copy Markdown
Owner

Problem

Cloudflare Workers AI exposes an OpenAI-compatible /chat/completions endpoint, but it wasn't documented or test-covered in the library. Users wanting to point Elixir LangChain at Workers AI models (e.g. Moonshot Kimi K2.6) had no confirmation that the existing ChatOpenAI module would work, and no example to copy.

Solution

No new chat model module is needed — ChatOpenAI already supports a custom endpoint and api_key, which is exactly what Cloudflare's OpenAI-compatible gateway requires. This PR verifies that path end-to-end with live tests against @cf/moonshotai/kimi-k2.6 (both non-streaming and streaming-with-tool-calls), and documents the configuration in the README so users can adopt it without guesswork.

The streaming test is the load-bearing one: it exercises SSE frame decoding, incremental tool_call.arguments accumulation, and delta merge, and asserts that the chain assembled from streamed deltas matches the final Message produced by LLMChain.run/1. That confirms Cloudflare's stream format is compatible with the OpenAI delta handling already in the library.

Changes

  • README.md — Added Cloudflare Workers AI to the supported providers list and a full configuration example (endpoint URL, env vars, streaming/tool-calling notes)
  • .env.example — Added CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN
  • test/chat_models/chat_open_ai_test.exs — Two live tests tagged live_call: true, live_cloudflare: true:
    • Non-streaming sanity check (single-word reply)
    • Streaming + tool-calling check that asserts arguments parse correctly and that delta-merged output equals the synchronously-assembled message

Testing

Run the new live tests against Cloudflare with credentials in env:

mix test test/chat_models/chat_open_ai_test.exs --include live_cloudflare

Both tests pass against @cf/moonshotai/kimi-k2.6. Default mix test is unaffected since the tests are gated behind the live_cloudflare tag.

@brainlid brainlid merged commit ddc9ff7 into main May 25, 2026
2 checks passed
@brainlid brainlid deleted the me-verify-cloudflare-workers-ai branch May 25, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant