[Bug]: Issue with Environment Variable Substitution in config.yaml for Azure Model Configuration

[kavin851018]

What happened?

I'm encountering an issue when using environment variable substitution in the config.yaml file as per the official documentation. When I reference environment variables (e.g., os.environ/AZURE_API_KEY) in the model_list configuration, LiteLLM fails to recognize the model and returns a litellm.BadRequestError. However, if I hardcode the values directly into the config.yaml, the request works as expected.

Steps to Reproduce

Docker Compose Configuration:
Below is my docker-compose.yml setup:

version: "3.11"
services:
  litellm:
    build:
      context: .
      args:
        target: runtime
    image: ghcr.io/berriai/litellm:main-stable
    volumes:
      - ./config.yaml:/app/config.yaml 
    command:
      - "--config=/app/config.yaml"
      - "--detailed_debug"
    ports:
      - "4000:4000"
    environment:
      DATABASE_URL: "postgresql://llmproxy:[REDACTED_PASSWORD]@db:5432/litellm"
      STORE_MODEL_IN_DB: "True"
    env_file:
      - .env

  db:
    image: postgres
    restart: always
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: llmproxy
      POSTGRES_PASSWORD: [REDACTED_PASSWORD]
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"]
      interval: 1s
      timeout: 5s
      retries: 10`

Environment Variables:
My .env file contains the following relevant variables:

AZURE_API_BASE="https://[REDACTED_ENDPOINT].openai.azure.com/"
AZURE_API_VERSION="2025-01-01-preview"
AZURE_API_KEY="[REDACTED_KEY]"
LITELLM_MASTER_KEY="sk-1234"
DATABASE_URL="postgresql://llmproxy:[REDACTED_PASSWORD]@db:5432/litellm"
STORE_MODEL_IN_DB="True"

Config File ( Failing Case ):
In my config.yaml, I use environment variable substitution as shown below:

model_list:
  - model_name: mymodel
    litellm_params:
      model: azure/gpt-4o
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
      api_version: "os.environ/AZURE_API_VERSION"

Request:
I send the following curl request:

`curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
  "model": "mymodel",
  "messages": [
    {"role": "user", "content": "Hi there!"}
  ]
}'

Error Response:
I receive the following error:

{
  "error": {
    "message": "litellm.BadRequestError: You passed in model=mymodel. There is no 'model_name' with this string \nReceived Model Group=mymodel\nAvailable Model Group Fallbacks=None",
    "type": null,
    "param": null,
    "code": "400"
  }
}

Working Case:
If I modify the config.yaml to use hardcoded values instead, like this:

model_list:
  - model_name: mymodel
    litellm_params:
      model: azure/gpt-4o
      api_base: "https://[REDACTED_ENDPOINT].openai.azure.com/"
      api_key: "[REDACTED_KEY]"
      api_version: "2025-01-01-preview"```

The same curl request works perfectly and returns a valid response.

Expected Behavior
When using os.environ/VARIABLE_NAME syntax in config.yaml as documented, LiteLLM should correctly substitute the environment variables from the .env file and recognize the model mymodel without throwing a BadRequestError.

Actual Behavior
LiteLLM fails to resolve the model when environment variables are referenced in config.yaml, resulting in the error: litellm.BadRequestError: You passed in model=mymodel. There is no 'model_name' with this string.

Environment
LiteLLM Version: main-stable (from ghcr.io/berriai/litellm:main-stable)
Docker Compose Version: 3.11
Operating System: [Your OS, e.g., Ubuntu 22.04]
Deployment Method: Docker via docker-compose
Additional Notes
The environment variables are correctly loaded, as confirmed by the fact that DATABASE_URL and LITELLM_MASTER_KEY work fine.
The issue seems specific to how LiteLLM parses environment variable references in config.yaml for the model_list.
Could this be a bug in how LiteLLM handles environment variable substitution in the config file? Any assistance or workaround would be greatly appreciated!

Relevant log output

shell

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

litellm:main-stable

Twitter / LinkedIn details

No response

[Roboman341]

Same here, can't make litellm consuming variables from config.yaml

[stronk7]

Some months ago I had some headaches with this till I ensured that the .env file was in the same directory than the docker-compose.yml one (the location has to be relative to that directory).

So it was not LiteLLM's fault but wrong docker / docker compose configuration, in my case.

A good way to discard problems is to connect to the running container and check if the env variables are there or not:

docker exec -it `<<your_litellm_container>>` env

If the env variables aren't there, then it's impossible for LiteLLM to get them.

Ciao :-)

PS: Also, from memory, Docker's --env-file support has some limitations, like not allowing comments, ... and maybe not quotes either, I remember finding some issues requesting to make it to support all the things that dotenv supports, but I think that it has not been done yet. So... try to keep you env file as simple as possible, simply a raw list of name=value pairs.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.