Skip to content

feat(providers): vendors that need more than an API key cannot be configured #254

Description

@0xKT

Problem

The onboarding picker's "Another supported vendor (type to search)" step offers the
117 vendors LiteLLM routes to. Every one of them that Raven carries no spec for is
treated as needing a single API key: the wizard prompts for one, stores it, and
reports success. For at least nine of those vendors that produces a provider that
cannot make a call.

Measured against litellm 1.85.0:

Authenticate by device flow, and take no API key at all

Vendor Evidence
chatgpt ships llms/chatgpt/authenticator.py
gigachat ships llms/gigachat/authenticator.py

Raven registers provider login for four families only (openai_codex,
github_copilot, minimax_global, minimax_cn), so a user who configures either
of these can never obtain a token: the key they typed is never read, and the first
call starts a browser device flow.

Need more than one credential, or a credential that is not a key

Vendor validate_environment reports
bedrock AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
sagemaker AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
vertex_ai VERTEXAI_PROJECT, VERTEXAI_LOCATION
azure AZURE_API_BASE, AZURE_API_KEY, AZURE_API_VERSION
cloudflare CLOUDFLARE_API_BASE, CLOUDFLARE_API_KEY
datarobot DATAROBOT_API_TOKEN

This is not a LiteLLM shortcoming. AWS signs requests with an access key, a secret
and a region; GCP uses a service account plus a project and a location; Azure gives
every tenant its own resource URL. Those are the vendors' own auth models, and
LiteLLM reflects them faithfully. The wrong part is Raven's assumption that a
vendor with no spec of ours needs exactly one API key.

A further 85 of the 117 could not be classified by validate_environment at all.
That is a limit of the probe, not evidence that they are fine.

Bedrock and Vertex AI are the ones likely to be missed: neither has a spec, so the
vendor step is the only way to reach them, and it cannot collect their credentials.

Scope

Not a regression of any one change: opening the picker up to every supported vendor
made an existing assumption reachable. Azure OpenAI had the same shape while being
in the curated list, and is fixed separately.

Possible directions

  1. Collect what the vendor actually needs. litellm.validate_environment(model=...)
    returns the missing variables per provider, correctly for bedrock, vertex_ai and
    azure among others. The wizard would prompt for those fields rather than assuming
    one key. ProviderConfig currently holds api_key, api_base, extra_headers
    and models, so it would need somewhere to keep the rest.
  2. Offer only what can be configured. Filter the vendor list to the shapes the
    wizard can actually collect, and give the others a spec (with a login flow where
    the vendor uses one) when someone wants them. Smaller, and it stops promising
    something that does not work.

Worth noting for direction 1: validate_environment itself resolves credentials,
so calling it for github_copilot starts a GitHub device login and blocks for
minutes. Route any such call through _may_prompt in
raven/token_wise/pricing.py, which answers "would handing this model to LiteLLM
start an interactive login" by inspecting the installed package.

Reproduce

  1. raven onboard, pick "Another supported vendor", type bedrock.
  2. The wizard asks for one API key and reports the provider configured.
  3. Nothing supplies AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY, so no call can be
    signed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions