feat(providers): add Cohere provider#581
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds Cohere as a built-in provider with chat, streaming, embeddings, audio transcription, model discovery, Responses translation, and native passthrough. It updates registration, configuration, Helm support, documentation, dashboard links, and stream-error handling. ChangesCohere provider
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant Gateway
participant CohereProvider
participant CohereAPI
Client->>Gateway: Send chat, embeddings, audio, or Responses request
Gateway->>CohereProvider: Route request to cohere
CohereProvider->>CohereAPI: Translate and send API request
CohereAPI-->>CohereProvider: Return response or stream
CohereProvider-->>Gateway: Convert response or failure event
Gateway-->>Client: Return compatible response
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@config/config.example.yaml`:
- Line 16: Update the ENABLED_PASSTHROUGH_PROVIDERS entry in .env.template to
include cohere, matching config/config.example.yaml and ensuring
environment-only deployments expose consistent passthrough behavior.
In `@docs/providers/cohere.mdx`:
- Around line 35-36: Update both curl examples in the Cohere provider
documentation to use the documented default port 8080 instead of 8090, keeping
the remaining command paths and options unchanged.
In `@internal/providers/cohere/chat.go`:
- Around line 411-422: Update normalizeFinishReason so Cohere’s ERROR and
TIMEOUT outcomes are propagated as provider/gateway errors instead of being
mapped to "stop"; if the existing contract requires a finish reason rather than
an error, map TIMEOUT to "length" while retaining "stop" only for genuine
content-stopped reasons such as COMPLETE and STOP_SEQUENCE.
- Around line 293-313: The response_format handling in copyChatExtraFields must
normalize OpenAI’s json_schema shape to Cohere’s schema shape before forwarding.
Add a provider-local helper such as cohereResponseFormat and use it when
assigning out.ResponseFormat, mapping the nested json_schema schema into
response_format.schema while preserving supported fields and existing behavior
for already-Cohere-formatted values.
In `@internal/providers/cohere/stream.go`:
- Around line 106-163: Update the event switch in the stream consumer to handle
Cohere citation-start and citation-end events instead of dropping them through
default. Emit citation-start data in the OpenAI-compatible delta format, and
reuse the non-streaming citation mapping for citation-end when the configured
citation mode requires it, while preserving existing content, tool, and finish
handling.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6c1da995-5da1-4ace-9991-dac13bf5a1db
⛔ Files ignored due to path filters (2)
internal/admin/dashboard/static/dist/assets/index-Dx5V1Ah5.jsis excluded by!**/dist/**internal/admin/dashboard/static/dist/index.htmlis excluded by!**/dist/**
📒 Files selected for processing (26)
.env.templateREADME.mdconfig/config.example.yamlconfig/config_test.godocs/about/roadmap.mdxdocs/advanced/configuration.mdxdocs/docs.jsondocs/providers/cohere.mdxdocs/providers/overview.mdxdocs/snippets/provider-credentials-note.mdxhelm/Chart.yamlhelm/README.mdhelm/templates/NOTES.txthelm/values.schema.jsonhelm/values.yamlinternal/providers/cohere/chat.gointernal/providers/cohere/cohere.gointernal/providers/cohere/cohere_test.gointernal/providers/cohere/embeddings.gointernal/providers/cohere/stream.gointernal/providers/cohere/types.gointernal/providers/config_test.gorun/providers.gorun/providers_test.goweb/dashboard/src/pages/overview/providersLogic.jsweb/dashboard/tests/overview-providers.test.js
Confidence Score: 4/5The PR is not yet safe to merge because Cohere stream read and decoding failures can truncate a Responses stream without emitting response.failed. Scanner and JSON event-decoding failures are propagated as raw pipe errors, bypassing the structured error chunk that the Responses converter requires to produce a terminal failed event. internal/providers/cohere/stream.go
 T-Rex LogsWhat T-Rex did
Sequence DiagramsequenceDiagram
participant C as Client
participant R as Responses converter
participant A as Cohere stream adapter
participant P as Cohere
C->>R: Streaming Responses request
R->>A: StreamResponsesViaChat
A->>P: "POST /v2/chat (stream=true)"
P-->>A: SSE events
A-->>R: OpenAI chat chunks
R-->>C: Responses SSE events
alt Cohere terminal provider error
A-->>R: provider_error chunk
R-->>C: response.failed
else Scanner or JSON decoding failure
A--xR: Raw pipe error
R--xC: Truncated stream without response.failed
end
Reviews (2): Last reviewed commit: "fix(providers): address follow-up review..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.env.template:
- Around line 51-53: Align the `.env.template` passthrough provider
documentation with the `defaultEnabledPassthroughProviders` runtime value by
removing `kilo` from the documented default, or add it to the runtime default if
that is the intended behavior; ensure both defaults match.
In `@internal/providers/responses_converter.go`:
- Around line 359-365: Update the upstream error decoding in
internal/providers/responses_converter.go lines 359-365 to fall back to a JSON
string when object unmarshalling fails, and represent the object’s code with
json.RawMessage before extracting its string value. Add table-driven coverage in
internal/providers/responses_converter_test.go lines 326-364 for object and
scalar string error payloads, including a malformed non-error member that
exercises processChunkTolerant.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: fc9bef80-1e78-4577-9c6d-77908b17b18a
📒 Files selected for processing (8)
.env.templatedocs/providers/cohere.mdxinternal/providers/cohere/chat.gointernal/providers/cohere/cohere_test.gointernal/providers/cohere/stream.gointernal/providers/cohere/types.gointernal/providers/responses_converter.gointernal/providers/responses_converter_test.go
| if err := scanner.Err(); err != nil { | ||
| _ = out.CloseWithError(err) | ||
| return |
There was a problem hiding this comment.
When a Cohere stream encounters a read failure or malformed JSON event after streaming begins, this closes the pipe with a raw error instead of emitting the provider-error chunk consumed by the Responses converter. The client consequently receives a truncated Responses SSE stream without a terminal response.failed event.
Artifacts
- Evidence file captured while the check ran.
Repro: verbose failing test output showing truncated Responses SSE and raw stream errors
- The full command output behind this check.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
internal/providers/cohere/cohere_test.go (1)
184-226: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse table-driven cases for repeated provider mappings.
The
ERROR/TIMEOUTand invalid-request scenarios should be expressed as table-driven tests so each mapping is enumerated consistently and remains easy to extend.Also applies to: 514-536
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/providers/cohere/cohere_test.go` around lines 184 - 226, Extend the table-driven test in TestChatCompletionReturnsCohereGenerationFailures to include the invalid-request finish reason scenario currently covered separately. Add its expected GatewayError status and message to the shared test cases, preserving the existing ERROR and TIMEOUT mappings and assertions.Source: Coding guidelines
internal/providers/cohere/chat.go (2)
298-383: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winReject unsupported
response_formatwhen tools are requested.Cohere does not allow
response_formattogether withtoolsordocuments. Since_convertToCohereRequestbuilds and forwards both fields, add validation there to return anInvalidRequestErrorbefore calling the upstream API, and split the Coheretest fixture so the successful translation test does not include this invalid combination.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/providers/cohere/chat.go` around lines 298 - 383, Update _convertToCohereRequest to return an InvalidRequestError before the upstream call when response_format is present together with tools or documents; keep valid response_format translations unchanged. In internal/providers/cohere/cohere_test.go lines 82-104, split the fixture so the successful translation case no longer includes tools or documents, and add coverage for rejecting those combinations.
101-109: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winReject unsupported Cohere message roles instead of treating them as user messages.
toCohereMessageconverts unknown roles touser, which can change request semantics and be serialized into an invalid Cohere request. Return an invalid-request error; conversion errors are already wrapped bytoCohereChatRequest.Proposed fix
case "system", "user", "assistant", "tool": default: - role = "user" + return chatMessage{}, fmt.Errorf("unsupported message role %q", message.Role)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/providers/cohere/chat.go` around lines 101 - 109, Update toCohereMessage so unsupported normalized roles return an invalid-request error instead of being reassigned to "user". Preserve the existing developer-to-system mapping and accepted system, user, assistant, and tool roles; rely on toCohereChatRequest’s existing error wrapping.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/core/errors_test.go`:
- Around line 344-352: Extend the table-driven Cohere cases in the
ParseProviderError tests to assert the normalized message exactly as "invalid
tool generation", rather than only checking that it is non-empty. Add cases
covering nested, scalar, and null error payloads, verifying each response’s
normalized message and preserving the existing provider-specific type, status,
and code mappings.
---
Outside diff comments:
In `@internal/providers/cohere/chat.go`:
- Around line 298-383: Update _convertToCohereRequest to return an
InvalidRequestError before the upstream call when response_format is present
together with tools or documents; keep valid response_format translations
unchanged. In internal/providers/cohere/cohere_test.go lines 82-104, split the
fixture so the successful translation case no longer includes tools or
documents, and add coverage for rejecting those combinations.
- Around line 101-109: Update toCohereMessage so unsupported normalized roles
return an invalid-request error instead of being reassigned to "user". Preserve
the existing developer-to-system mapping and accepted system, user, assistant,
and tool roles; rely on toCohereChatRequest’s existing error wrapping.
In `@internal/providers/cohere/cohere_test.go`:
- Around line 184-226: Extend the table-driven test in
TestChatCompletionReturnsCohereGenerationFailures to include the invalid-request
finish reason scenario currently covered separately. Add its expected
GatewayError status and message to the shared test cases, preserving the
existing ERROR and TIMEOUT mappings and assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 74822cf6-98b4-43f3-ba05-afcc2e231c59
📒 Files selected for processing (6)
internal/core/errors.gointernal/core/errors_test.gointernal/providers/cohere/chat.gointernal/providers/cohere/cohere_test.gointernal/providers/cohere/stream.gointernal/providers/cohere/types.go
Summary
Validation
Summary by CodeRabbit
/responses, streaming, embeddings, model discovery, audio transcription, and Cohere-native passthrough.response.failedinstead of completing.