Skip to content

feat: add Together AI as a supported model provider#3376

Merged
ronan-thibaut-glitch merged 3 commits into
mainfrom
feat/together-provider
Jul 1, 2026
Merged

feat: add Together AI as a supported model provider#3376
ronan-thibaut-glitch merged 3 commits into
mainfrom
feat/together-provider

Conversation

@Sayt-0

@Sayt-0 Sayt-0 commented Jul 1, 2026

Copy link
Copy Markdown
Member

Summary

Adds first-class support for Together AI as an OpenAI-compatible model provider via the alias map, following the existing fireworks/cerebras pattern.

Closes #3348

Definition of Done

Item Status Where
Alias entry together (openai, https://api.together.xyz/v1, TOGETHER_API_KEY) done pkg/model/provider/aliases.go
Unit test asserts the alias resolves and IsCatalogProvider("together") is true done TestCatalogAliases (together row), pkg/model/provider/aliases_test.go
Provider appears in the model catalog via CatalogProviders() yes (has a BaseURL) aliases.go
task test and task lint pass yes (see Validation)
Example YAML config using a Together AI model done examples/together.yaml

Beyond the stated DoD, this also wires auto-detection, a default model, the JSON schema, and the provider docs, mirroring how Fireworks/Cerebras/DeepSeek were added.

Design decisions

Decision Rationale
Added to openModelHostProviders (system-message coalescing) Together fronts open-weight models (Llama, Qwen, DeepSeek, Kimi, GLM) whose chat templates may reject more than one leading system message. Consecutive system messages are coalesced, per #3344 / #3145 / #2327. First-party APIs with a fixed lineup (mistral, xai, deepseek) are deliberately excluded.
Listed in modelsDevAbsentProviders (example test) models.dev catalogs Together under the provider id togetherai, not together, so the example-validation lookup is skipped — the same treatment as fireworks. No custom catalog-name mapping was added, keeping Together consistent with its sibling providers.
Auto-detection priority after cerebras, before amazon-bedrock Keeps the open-model-host ordering consistent; precedence assertions updated accordingly.
Default model meta-llama/Llama-3.3-70B-Instruct-Turbo Well-known general-purpose Together open model, present in the models.dev togetherai catalog.
Removed pkg/model/provider/openai/repro_issue3344_test.go (scratch repro file) Its reusable TestShouldMergeConsecutiveMessages_Gating and an equivalent openai+base_url (vLLM) merge regression were moved into the properly-named system_message_merge_test.go, so no coverage is lost.

Tests

Test Coverage
TestCatalogAliases (together) Alias resolution and catalog membership.
TestOpenAIAliasProvider_EndToEndRequest (together) Real HTTP request through the full stack against a local OpenAI-compatible server: asserts POST, Bearer TOGETHER_API_KEY, /chat/completions route, verbatim model, stream reassembly, and that consecutive system messages are coalesced into one.
TestOpenAIAliasProvider_LiveAPI (together) Live API call, skipped unless TOGETHER_API_KEY is set, so the default run stays hermetic.
TestShouldMergeConsecutiveMessages_Gating (together) Asserts coalescing is enabled for together.
auto_test.go cases Detection, default model, precedence chain, integration with DefaultModels.
TestParseExamples Validates examples/together.yaml.

Validation

Gate Result
task build passes
task lint (golangci-lint + custom cops) 0 issues / no offenses
go test ./... touched packages pass; remaining failures are pre-existing SSRF/network tests unrelated to this change
CI on this PR all checks green

@Sayt-0 Sayt-0 requested a review from a team as a code owner July 1, 2026 13:04
Add together as a built-in OpenAI-compatible alias provider, following the
fireworks/cerebras pattern. Registers the alias (base URL
https://api.together.xyz/v1, token TOGETHER_API_KEY), auto-detection in
cloudProviders, default model meta-llama/Llama-3.3-70B-Instruct-Turbo, schema
and docs updates, an example config, and tests.

Together AI fronts open-weight models (Llama, Qwen, DeepSeek, Kimi) whose chat
templates may reject more than one leading system message, so it is added to
openModelHostProviders to coalesce consecutive system messages (#2327/#3145/#3344).
It is validated end-to-end by the shared openAIAliasProviders table test.

models.dev catalogs Together AI under the "togetherai" id rather than
"together", so it is exempted in modelsDevAbsentProviders like fireworks.

Also removes the openai/repro_issue3344_test.go scratch file, moving its
reusable shouldMergeConsecutiveMessages gating test and preserving the
openai+base_url (vLLM) merge regression in system_message_merge_test.go.

Closes #3348
@Sayt-0 Sayt-0 force-pushed the feat/together-provider branch from 1f50f1a to 120529b Compare July 1, 2026 13:14
@aheritier aheritier added area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/models LLM model integrations and model providers area/providers/openai For features/issues/fixes related to the usage of OpenAI models kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Jul 1, 2026
feat: add Hugging Face Inference Providers as a supported model provider
@ronan-thibaut-glitch ronan-thibaut-glitch merged commit 6a6c3ac into main Jul 1, 2026
11 checks passed
@Sayt-0 Sayt-0 deleted the feat/together-provider branch July 1, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/models LLM model integrations and model providers area/providers/openai For features/issues/fixes related to the usage of OpenAI models kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Together AI as a supported model provider

4 participants