Verify Cloudflare Workers AI via ChatOpenAI#558
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Cloudflare Workers AI exposes an OpenAI-compatible
/chat/completionsendpoint, but it wasn't documented or test-covered in the library. Users wanting to point Elixir LangChain at Workers AI models (e.g. Moonshot Kimi K2.6) had no confirmation that the existingChatOpenAImodule would work, and no example to copy.Solution
No new chat model module is needed —
ChatOpenAIalready supports a customendpointandapi_key, which is exactly what Cloudflare's OpenAI-compatible gateway requires. This PR verifies that path end-to-end with live tests against@cf/moonshotai/kimi-k2.6(both non-streaming and streaming-with-tool-calls), and documents the configuration in the README so users can adopt it without guesswork.The streaming test is the load-bearing one: it exercises SSE frame decoding, incremental
tool_call.argumentsaccumulation, and delta merge, and asserts that the chain assembled from streamed deltas matches the finalMessageproduced byLLMChain.run/1. That confirms Cloudflare's stream format is compatible with the OpenAI delta handling already in the library.Changes
README.md— Added Cloudflare Workers AI to the supported providers list and a full configuration example (endpoint URL, env vars, streaming/tool-calling notes).env.example— AddedCLOUDFLARE_ACCOUNT_IDandCLOUDFLARE_API_TOKENtest/chat_models/chat_open_ai_test.exs— Two live tests taggedlive_call: true, live_cloudflare: true:Testing
Run the new live tests against Cloudflare with credentials in env:
Both tests pass against
@cf/moonshotai/kimi-k2.6. Defaultmix testis unaffected since the tests are gated behind thelive_cloudflaretag.