Added http_client option to the BaseOpenAIClientConfiguration class

[avinashmihani]

Added http_client option to the BaseOpenAIClientConfiguration class, to support SSL verification option in model_client intiation.

Why are these changes needed?

In v0.2.x the agents to interact with LLM, was expecting LLM_CONFIG, which had an option to skip the SSL verification. Now in v0.4.x, the model client initiation does not have an option for skipping verification.
With this change, v0.4 as well will allow http_client option in model_client initiation

Related issue number

Closes #5795

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 77.25%. Comparing base (8fe627e) to head (1414a6d).
Report is 19 commits behind head on main.

Files with missing lines	Patch %	Lines
...xt/src/autogen_ext/models/openai/_openai_client.py	50.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5811      +/-   ##
==========================================
- Coverage   77.25%   77.25%   -0.01%     
==========================================
  Files         200      200              
  Lines       14256    14262       +6     
==========================================
+ Hits        11014    11018       +4     
- Misses       3242     3244       +2     

Flag	Coverage Δ
unittests	77.25% <66.66%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ekzhu]

@avinashmihani actually I just realized the http_client should already passed to AsyncOpenAI constructor without this change.

Does this change actually resolves #5795 ?

[avinashmihani]

@avinashmihani actually I just realized the http_client should already passed to AsyncOpenAI constructor without this change.

Does this change actually resolves #5795 ?

@ekzhu Yes, this change worked. I tested in my local. With this change, I am able to update SSL verification value in v0.4.x.

[ekzhu]

The problem though is that CreateArguments is a typed dictionary so it can only contain typed annotation. So we can't use the httpx.AsyncClient type. Perhaps we use Any here? @jackgerrits what's your thought?

[jackgerrits]

You can't put anything into that dictionary that is not json serializable so this is not going to work. This is required for the component config feature.

If a python object must be passed it must be done directly via the constructor and so it wont be supported via config. If it is necesasary to support this argument we will need to strip it out of the exported config dictionary in _to_config

[ekzhu]

@jackgerrits @avinashmihani Question: can we not just forward all parameters that matches the constructor argument list of AsyncOpenAI to the underyling client? Is adding this to the config even necessary?

[jackgerrits]

@avinashmihani if you agree to my changes please go ahead and respond to the CLA

[zijianding]

@avinashmihani actually I just realized the http_client should already passed to AsyncOpenAI constructor without this change.
Does this change actually resolves #5795 ?

@ekzhu Yes, this change worked. I tested in my local. With this change, I am able to update SSL verification value in v0.4.x.

hi @avinashmihani and @ekzhu , I follow the #5795 here.
I also run into a similar httpx.Connection Error while using OpenAIChatCompletionClient, I tried the following code: OpenAI(api_key="sk--", base_url="http://xxx", http_client=httpx.Client(verify=False))
and without the 'http_client=httpx.Client(verify=False)' the same error reproduced, with 'http_client=httpx.Client(verify=False)' everything works fine.

So I'm following your hint to instantiate an AsyncOpenAI, and pass it to instantiate OpenAIChatCompletion, the codes as here:

async_client = AsyncOpenAI(
api_key="sk-xxxx",
base_url="http://api.xxxx.com/",
http_client=httpx.AsyncClient(verify=False),
timeout=10000
)

model_client = OpenAIChatCompletionClient(
client=async_client,
model="xxxx",
model_info=model_info,
base_url="http://api.xxxx.com/",
api_key="sk-xxx",
response_format={"type":"json_object"}
)

but the same connection error happened, though I thought this should have worked. Could you please give me a clue on how to fix this?

[avinashmihani]

@avinashmihani if you agree to my changes please go ahead and respond to the CLA

I am checking with my employer on this. Please allow some time.

[avinashmihani]

@avinashmihani actually I just realized the http_client should already passed to AsyncOpenAI constructor without this change.
Does this change actually resolves #5795 ?

@ekzhu Yes, this change worked. I tested in my local. With this change, I am able to update SSL verification value in v0.4.x.

hi @avinashmihani and @ekzhu , I follow the #5795 here. I also run into a similar httpx.Connection Error while using OpenAIChatCompletionClient, I tried the following code: OpenAI(api_key="sk--", base_url="http://xxx", http_client=httpx.Client(verify=False)) and without the 'http_client=httpx.Client(verify=False)' the same error reproduced, with 'http_client=httpx.Client(verify=False)' everything works fine.

So I'm following your hint to instantiate an AsyncOpenAI, and pass it to instantiate OpenAIChatCompletion, the codes as here:

async_client = AsyncOpenAI( api_key="sk-xxxx", base_url="http://api.xxxx.com/", http_client=httpx.AsyncClient(verify=False), timeout=10000 )

model_client = OpenAIChatCompletionClient( client=async_client, model="xxxx", model_info=model_info, base_url="http://api.xxxx.com/", api_key="sk-xxx", response_format={"type":"json_object"} )

but the same connection error happened, though I thought this should have worked. Could you please give me a clue on how to fix this?

Hi @zijianding,

below is how I am using it.

import httpx

class MyHttpClient(httpx.AsyncClient):
    def __deepcopy__(self, memo):
        return self

from http_client import MyHttpClient

model_client = OpenAIChatCompletionClient(
                model=model_name,
                api_key=<api_key>,
                base_url=https://xxxx,
                model_capabilities=<capabilities json>,
                http_client=MyHttpClient(verify=False),
            )

I updated my autogen package by adding http_client option to the BaseOpenAIClientConfiguration class. Hope that works for you.

[zijianding]

@avinashmihani actually I just realized the http_client should already passed to AsyncOpenAI constructor without this change.
Does this change actually resolves #5795 ?

@ekzhu Yes, this change worked. I tested in my local. With this change, I am able to update SSL verification value in v0.4.x.

hi @avinashmihani and @ekzhu , I follow the #5795 here. I also run into a similar httpx.Connection Error while using OpenAIChatCompletionClient, I tried the following code: OpenAI(api_key="sk--", base_url="http://xxx", http_client=httpx.Client(verify=False)) and without the 'http_client=httpx.Client(verify=False)' the same error reproduced, with 'http_client=httpx.Client(verify=False)' everything works fine.
So I'm following your hint to instantiate an AsyncOpenAI, and pass it to instantiate OpenAIChatCompletion, the codes as here:
async_client = AsyncOpenAI( api_key="sk-xxxx", base_url="http://api.xxxx.com/", http_client=httpx.AsyncClient(verify=False), timeout=10000 )
model_client = OpenAIChatCompletionClient( client=async_client, model="xxxx", model_info=model_info, base_url="http://api.xxxx.com/", api_key="sk-xxx", response_format={"type":"json_object"} )
but the same connection error happened, though I thought this should have worked. Could you please give me a clue on how to fix this?

Hi @zijianding,

below is how I am using it.

import httpx

class MyHttpClient(httpx.AsyncClient):
    def __deepcopy__(self, memo):
        return self

from http_client import MyHttpClient

model_client = OpenAIChatCompletionClient(
                model=model_name,
                api_key=<api_key>,
                base_url=https://xxxx,
                model_capabilities=<capabilities json>,
                http_client=MyHttpClient(verify=False),
            )

I updated my autogen package by adding http_client option to the BaseOpenAIClientConfiguration class. Hope that works for you.

Thanks @avinashmihani , I read the codes and found a simple way when instantiate the chat complete client by directly passing the http_client parameter. I guess there is no need to expose this specific parameter, also thanks for @ekzhu