configs.md

Configs

Configs streamline your Gateway management, enabling you to programmatically control various aspects like fallbacks, load balancing, retries, caching, and more.

A configuration is a JSON object that can be used to define routing rules for all the requests coming to your gateway. You can configure multiple configs and use them in your requests.

Creating Configs

Navigate to the ‘Configs’ page in the Portkey app and click 'Create' to start writing a new config.

Using Configs

Configs are supported across all integrations.

• Through the config parameter of the Portkey SDK client(Directly or via frameworks)
• Through the config headers in the OpenAI SDK
• Via the REST API through the x-portkey-config header

Applying Gateway Configs

Gateway configs allow you to unlock the gateway superpowers of Portkey. You can create a config in the UI and attach it's config id in the OpenAI client.

{% tabs %} {% tab title="NodeJS" %}

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    config: "pc-***" // Supports a string config id or a config object
});

{% endtab %}

{% tab title="Python" %}

const portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    config="pc-***" # Supports a string config id or a config object
)

{% endtab %}

{% tab title="OpenAI NodeJS" %}

const openai = new OpenAI({
  apiKey: 'OPENAI_API_KEY', // defaults to process.env["OPENAI_API_KEY"],
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    provider: "openai",
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    config: "CONFIG_ID" // Fetched from the UI
  })
});

{% endtab %}

{% tab title="OpenAI Python" %}

client = OpenAI(
    api_key="OPENAI_API_KEY", # defaults to os.environ.get("OPENAI_API_KEY")
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="openai",
        api_key="PORTKEY_API_KEY", # defaults to os.environ.get("PORTKEY_API_KEY")
        config="CONFIG_ID"
    )
)

{% endtab %}

{% tab title="REST API" %}

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: openai" \ 
  -H "x-portkey-config: $CONFIG_ID" \ 
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{
        "role": "user",
        "content": "Hello!"
      }]
  }'

{% endtab %} {% endtabs %}

If you want to attach the configuration to only a few requests instead of modifying the client, you can send it in the request headers for OpenAI or in the config parameter while using the Portkey SDK.

Note: If you have a default configuration set in the client, but also include a configuration in a specific request, the request-specific configuration will take precedence and replace the default config for that particular request.

{% tabs %} {% tab title="NodeJS" %}

portkey.chat.completions.create({
  messages: [{role: "user", content: "Say this is a test"}],
  model: "gpt-3.5-turbo"
}, {config: "pc-***"})

{% endtab %}

{% tab title="Python" %}

portkey.with_options(config="pc-***").chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'gpt-3.5-turbo'
})

{% endtab %}

{% tab title="OpenAI NodeJS" %}

let reqHeaders = createHeaders({config: "CONFIG_ID});
openai.chat.completions.create({
  messages: [{role: "user", content: "Say this is a test"}],
  model: "gpt-3.5-turbo"
}, {headers: reqHeaders})

{% endtab %}

{% tab title="OpenAI Python" %}

reqHeaders = createHeaders(config="CONFIG_ID")
client.with_options(headers=reqHeaders).chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'gpt-3.5-turbo'
})

{% endtab %} {% endtabs %}

{% hint style="info" %} You can also add the config JSON as a string instead of the slug. {% endhint %}

Configs in Logs

Portkey shows your Config usage smartly on the logs page with the Status column and gives you a snapshot of the Gateway activity for every request. Read more about the status column here.

You can also see the ID of the specific Config used for a request separately in the log details, and jump into viewing/editing it directly from the log details page.

Config Object Documentation

The config object is defined here

{% content-ref url="../../api-reference/config-object.md" %} config-object.md {% endcontent-ref %}