fix(secrets): resolve llm provider and secret mismatches (#5032)

[raza-khan0108]

Summary

Together AI was mapped inconsistently across the codebase. The frontend (@agenta/entities) and SDK (daytona.py) looked for TOGETHERAI_API_KEY, while standard LLM providers in the backend expected TOGETHER_API_KEY. Furthermore, MINIMAX_API_KEY was missing from the test suite mappings.

As a result, users configuring Together AI or MiniMax encountered secret resolution failures during evaluation and playground execution because the system looked for different environment variable names across layers.

This PR aligns Together AI on TOGETHER_API_KEY across the frontend packages, FastAPI backend, and SDK runners. To prevent breaking existing deployments that saved secrets under the legacy name, fallback aliases (TOGETHERAI_API_KEY) are added in both the frontend (STANDARD_PROVIDER_ENV_ALIASES) and backend secrets core (_LEGACY_SYSTEM_ENV_NAMES). It also adds MINIMAX_API_KEY to the test mappings.

Before:

# Backend evaluation model
togetherai = "TOGETHERAI_API_KEY"

# SDK Daytona runner
"together_ai": "TOGETHERAI_API_KEY",

After:

# Backend evaluation model
togetherai = "TOGETHER_API_KEY"

# SDK Daytona runner
"together_ai": "TOGETHER_API_KEY",

# Fallback in secrets utils
_LEGACY_SYSTEM_ENV_NAMES = {
    LMProvidersEnum.togetherai.value: ("TOGETHERAI_API_KEY",),
}

Testing

Verified locally

• Ran backend secrets unit tests via uv run --project api pytest api/oss/tests/pytest/unit/secrets/test_utils.py (all 3 unit tests passed).
• Ran frontend unit tests via pnpm --filter="@agenta/entities" test (675 tests passed) and pnpm --filter="@agenta/shared" test (189 tests passed).
• Verified code formatting and linting via ruff format, ruff check, and pnpm lint-fix on modified packages with zero errors or warnings.

Added or updated tests

• Updated test_utils.py in API unit tests to assert TOGETHER_API_KEY resolution.
• Added test_get_system_llm_providers_secrets_reads_legacy_togetherai_env in test_utils.py to verify system secret fallback when only TOGETHERAI_API_KEY is set in the environment.
• Updated conftest.py in both API and SDK legacy test suites to use TOGETHER_API_KEY and added MINIMAX_API_KEY to API_KEYS_MAPPING.

QA follow-up

• Validate configuring a Together AI provider key in the UI vault and running a playground prompt using a Together AI model.
• Verify that an existing deployment with TOGETHERAI_API_KEY set in the system environment still resolves correctly when executing evaluations without needing manual secret re-entry.

Demo

Checklist

• I have included a video or screen recording for UI changes, or marked Demo as N/A
• Relevant tests pass locally
• Relevant linting and formatting pass locally
• I have signed the CLA, or I will sign it when the bot prompts me

Contributor Resources

• Contributing guide
• Creating your first PR
• Development mode
• Testing
• Formatting and linting
• Slack support

[vercel]

@raza-khan0108 is attempting to deploy a commit to the agenta projects Team on Vercel.

A member of the Team first needs to authorize it.

[CLAassistant]

All committers have signed the CLA.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

✅ Thanks @raza-khan0108! This PR now meets the contribution requirements and has been reopened. A maintainer will review it soon.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: de68e562-2223-4c34-ba83-bf7e8a46dbc1

📥 Commits

Reviewing files that changed from the base of the PR and between b6fafc3 and 4464c90.

📒 Files selected for processing (9)

• api/oss/src/core/secrets/utils.py
• api/oss/src/models/api/evaluation_model.py
• api/oss/src/utils/env.py
• api/oss/tests/legacy/conftest.py
• api/oss/tests/pytest/unit/secrets/test_utils.py
• sdks/python/agenta/sdk/engines/running/runners/daytona.py
• sdks/python/oss/tests/legacy/new_tests/conftest.py
• web/packages/agenta-entities/src/secret/core/transforms.ts
• web/packages/agenta-shared/src/utils/llmProviders.ts

Disabled knowledge base sources:

• Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

---

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes
  • Standardized Together AI secret handling to use TOGETHER_API_KEY across the app.
  • Added compatibility with the older TOGETHERAI_API_KEY name so existing setups continue to work.
  • Updated provider mapping so secrets and environment variables resolve consistently in both web and Python flows.
  • Expanded test coverage for Together AI secret resolution and added support for an additional provider key in test setup.

Walkthrough

Renames the TogetherAI provider environment variable from TOGETHERAI_API_KEY to TOGETHER_API_KEY across backend enums, secret resolution utilities, SDK runner mappings, and frontend provider lists, while preserving legacy TOGETHERAI_API_KEY as a fallback/alias. Corresponding test fixtures and unit tests are updated, and a MINIMAX_API_KEY test mapping is added.

Changes

TogetherAI Environment Variable Key Correction

Layer / File(s)	Summary
Backend enum and secret resolution api/oss/src/models/api/evaluation_model.py, api/oss/src/core/secrets/utils.py, api/oss/src/utils/env.py	LMProvidersEnum.togetherai constant changed to TOGETHER_API_KEY; secret utils add togetherai to legacy env/slug lookups; LLMConfig reads TOGETHER_API_KEY first with fallback to legacy TOGETHERAI_API_KEY.
Backend tests and legacy fixtures api/oss/tests/pytest/unit/secrets/test_utils.py, api/oss/tests/legacy/conftest.py	Unit tests updated/added to assert normalization of legacy provider slug and legacy env var into TOGETHER_API_KEY; legacy conftest mapping updated to read TOGETHER_API_KEY.
Python SDK runner and fixtures sdks/python/agenta/sdk/engines/running/runners/daytona.py, sdks/python/oss/tests/legacy/new_tests/conftest.py	Daytona runner's together_ai env var mapping corrected to TOGETHER_API_KEY; SDK test fixture mapping updated and a new MINIMAX_API_KEY entry added.
Frontend provider mappings web/packages/agenta-entities/src/secret/core/transforms.ts, web/packages/agenta-shared/src/utils/llmProviders.ts	Forward kind→env mapping for TogetherAI changed to TOGETHER_API_KEY; legacy TOGETHERAI_API_KEY added as reverse-mapping alias; llmAvailableProviders entry updated to the new name.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

Not applicable.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4464c90835

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Continue injecting LiteLLM's TogetherAI key

For Daytona evaluators that call LiteLLM with a together_ai/... model, this map is the only place vault provider keys are copied into the sandbox environment before create(...). LiteLLM still documents TOGETHERAI_API_KEY for Together AI (https://docs.litellm.ai/docs/providers/togetherai), so after this change the vault key is only available as TOGETHER_API_KEY and those evaluators fail authentication; inject the legacy name as well during the transition.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Canonicalize legacy Together names before matching

When a browser still has legacy llmAvailableProvidersToken data with provider.name === "TOGETHERAI_API_KEY" and the project already has a Together vault secret, this new canonical display name makes standardSecrets contain TOGETHER_API_KEY while createStandardSecretAtom looks up by the raw legacy name. The lookup misses, so the migration/submit path creates a second together_ai secret instead of updating the existing one, and later reads can pick the wrong duplicate key; normalize the legacy name before matching.

Useful? React with 👍 / 👎.