Python: Feature: Add extenstion for Azure OpenAI On Your Data

[johnliu55-msft]

Since C# has supported this feature (#2392), is there a plan to support this in Python?

[madsbolaris]

Yes, we would like to support this. Would you be interested in submitting a PR to add this functionality?

[johnliu55-msft]

Sure! I suppose this feature should be implemented as an AI connector just like the dotnet implementation?

[johnliu55-msft]

I noticed that the Python implementation lacks a user-friendly way to support multiple AI connectors/LLM models, which is a crucial dependency of supporting Azure OpenAI on your data. The reason behind this is if we implemented the feature as an AI connector, then this AI connector should only be used by certain semantic functions. For instance, the planner should never use the connector to generate a plan.

On the other hand, the dotnet implementation has ServiceId to configure the connector to use for each semantic function.

Seems like there are some issues tracking this dependency, but seems like they are mostly for dotnet:

• .Net: Multiple LLM support for semantic functions #3041
• .Net: Use Multiple Models in a Single Kernel #1787

I'm happy to implement this ServiceId feature for Python if it's required.

[johnliu55-msft]

@matthewbolanos Seems like the dotnet version has introduced the ModelSettings attribute to the PromptTemplateConfig class to specify the AI connector to use for the given semantic function. Will this ModelSettings abstraction be added to Python's implementation?

[madsbolaris]

Yes, our current plan is to bring ModelSettings into the Python version of Semantic Kernel