Skip to content

Allow custom headers for BYOK #3399

@ZzetT

Description

@ZzetT

Describe the feature or problem you'd like to solve

Some LLM servers require specific headers to be sent, allow to set custom http(s) headers
Examples could be:

  • X-Tenant-ID / X-Organization-ID: Identifies which company or sub-organization is making the request to ensure their specific data boundaries and rate limits are respected.
  • X-User-ID: Tracks usage down to the individual employee or end-user, often used to prevent a single user from exhausting the company's API quota.
  • X-Fallback-Models: Instructs the gateway on what to do if the primary model is down
  • X-Cache-Bypass: Forces the server to ignore the cache and generate a fresh response from the LLM. Useful for testing or if the LLM is expected to return real-time data.
  • anthropic-beta: Anthropic frequently uses this to enable experimental features

Proposed solution

Add COPILOT_EXTRA_HEADERS (or COPILOT_CUSTOM_HEADERS) to set one ore multiple headers in the format "Name: Value" (newline for multiple headers)

Filter out headers Authorization or x-api-key (or do not allow them)

Example prompts or workflows

export COPILOT_EXTRA_HEADERS="X-Tenant-ID: 1234\nX-Proxy-Bypass: true"

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:configurationConfig files, instruction files, settings, and environment variablesarea:modelsModel selection, availability, switching, rate limits, and model-specific behavior
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions