.Net: API Manifest Plugins

[zengin]

Motivation and Context

This PR adds a prototype implementation of API Manifest based plugins.
APIManifest spec is located here. API Manifest is a format that allows defining an AI plugin from an existing OpenAPI, by specifying which subset of request paths to use in addition to other requirements such as auth. This PR contains a basic prototype covering a subset of the spec. The rest of the evolving spec will be implemented with follow up PRs and they will be shaped by the feedback we get.

Description

This PR exposes two public kernel extensions ImportPluginFromApiManifestAsync and CreatePluginFromApiManifestAsync which are similar in signature as their OpenAPI counterparts. They take APIManifest file location, read contents, slice referenced OpenAPI document based on what is described in the requests section. They then use underlying SemanticKernel OpenAPI function generation logic to generate functions and return a plugin instance.

These features are marked as experimental and give a compiler warning when used. We used SKEXP0099 as the diagnostic ID to avoid collision. We would like to hear what the right way is to increment and assign these numbers.

We also extended the function name generation logic to not rely on operationId as it is optional in OpenAPI spec. We should probably change the main Functions.OpenAPI implementation as well, but current implementation only proposes an override at the moment, where the name is generated from the path and method outside.

We also fixed a minor serialization bug where the depth was not enough to represent Graph API shape.

Lastly, we add Microsoft Graph samples in the KernelSyntaxExamples project. Same argument holds with the naming of `Example99``, i.e. to avoid collision with examples coming from other PRs.

Running the sample

In order to run the samples, you need an appsettings.Development.json file in KernelSyntaxExamples folder in this shape:

{
  "AzureOpenAI": {
    "ChatModelId": "gpt-4",
    "ServiceId": "gpt-4",
    "ChatDeploymentName": "gpt-4",
    "Endpoint": "<endpoint>",
    "ApiKey": "<endpoint>"
  },
  "MsGraph": {
    "ClientId": "<MSA-client-id>",
    "TenantId": "9188040d-6c67-4c5b-b112-36a304b66dad", // MSA/Consumer/Personal tenant,  https://learn.microsoft.com/azure/active-directory/develop/accounts-overview
    "Scopes": [
      "User.Read",
      "Files.ReadWrite",
      "Tasks.ReadWrite",
      "Mail.Send",
      "Contacts.Read",
      "Calendars.Read"
    ],
    "RedirectUri": "http://localhost"
  }
}

• Create an MSAL app registration in the Azure portal, and add the client id to the appsettings.Development.json file.

• Run the following command and login with your test MSA account. The sample expects at least one email message in inbox, a test.txt file in the root OneDrive folder, and at least one event in the calendar.

dotnet test --filter Example99_ApiManifest --logger "console;verbosity=detailed"

Expected output

Running Example99_ApiManifest... 
 
======== [ApiManifest Plugins] Sample 1 - Create and Execute "show the subject of my first message" Plan ======== 
======== Plugins: MessagesPlugin ======== 
======== Expected Output: latest email message subject is shown ======== 
 
>> MessagesPlugin is created. 
-------------------- 
 
Result: 
<REDACTED: email title> 
 
-------------------- 
 
======== [ApiManifest Plugins] Sample 2 - Create and Execute "get contents of file with id=test.txt" Plan ======== 
======== Plugins: DriveItemPlugin MessagesPlugin ======== 
======== Expected Output: test.txt file is fetched and the content is shown ======== 
 
>> DriveItemPlugin is created. 
>> MessagesPlugin is created. 
-------------------- 
 
Result: 
<REDACTED: contents of test.txt in the root OneDrive folder> 
 
-------------------- 
 
======== [ApiManifest Plugins] Sample 3 - Create and Execute "get contents of file with id=test.txt" Plan ======== 
======== Plugins: MessagesPlugin ======== 
======== Expected Output: test.txt file is not fetched because DriveItemPlugin is not loaded ======== 
 
>> MessagesPlugin is created. 
-------------------- 
 
Result: 
The provided functions do not include a method for retrieving the contents of a file. The goal of getting the contents of a file with the ID 'test.txt' cannot be achieved with the current set of functions. 
 
-------------------- 
 
======== [ApiManifest Plugins] Sample 4 - Create and Execute "tell me how many contacts I have" Plan ======== 
======== Plugins: MessagesPlugin ContactsPlugin ======== 
======== Expected Output: number of contacts is shown ======== 
 
>> MessagesPlugin is created. 
>> ContactsPlugin is created. 
-------------------- 
 
Result: 
You have 0 contacts. 
 
-------------------- 
 
======== [ApiManifest Plugins] Sample 5 - Create and Execute "tell me title of first event in my calendar" Plan ======== 
======== Plugins: MessagesPlugin CalendarPlugin ======== 
======== Expected Output: title of first event from calendar is shown if exists ======== 
 
>> MessagesPlugin is created. 
>> CalendarPlugin is created. 
-------------------- 
 
Result: 
API Manifest Event 
 
-------------------- 
 
== DONE == 

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[SergeyMenshykh]

It should be possible to override the user agent if needed. See the example here -

semantic-kernel/dotnet/src/Functions/Functions.OpenApi/Extensions/OpenApiKernelExtensions.cs

Line 154 in e4874bb

executionParameters?.UserAgent,

[zengin]

The type name OpenApiFunctionExecutionParameters suggests that these are parameters to be used while calling the plugin functions (i.e. executing them). For example, we can't use the authCallback override from execution parameters in the line above because it is intended for the REST API call, not for fetching the OpenAPI document. Wouldn't the parameters be overloaded if used at the time of import? If the intention is to use for both, please let me know.

[SergeyMenshykh]

Motivation and Context

This PR adds a prototype implementation of API Manifest based plugins. APIManifest spec is located here. API Manifest is a format that allows defining an AI plugin from an existing OpenAPI, by specifying which subset of request paths to use in addition to other requirements such as auth. This PR contains a basic prototype covering a subset of the spec. The rest of the evolving spec will be implemented with follow up PRs and they will be shaped by the feedback we get.

@zengin Could you please describe the scenarios in which this API Manifest functionality will be used?

[zengin]

@zengin Could you please describe the scenarios in which this API Manifest functionality will be used?

Sure.

The biggest advantage of using API Manifest over an entire OpenAPI document currently is when an API contains too many operations such as Microsoft Graph (thousands of operations), where you only need a slice of it as a plugin.

Another developer's API may not be as big, but they may want to limit operations of the plugin to read-only endpoints (or any other subset criterion that makes sense).

API Manifest also allows adding more than one OpenAPI reference, which a scenario specific plugin may require (e.g. read GitHub issues and add them as a task into the todo list).

Given that API Manifest is under development and is more flexible in terms of the format, we will be able to add features required to describe a plugin, which may not necessarily be part of the OpenAPI specification. Additional decorations it contains are similar but not limited to OpenAI's plugin manifest or Microsoft Copilot's plugin manifest.

Eventually OpenAPI spec may support all these additional decorations to power AI scenarios (as pointed out here), in that case, plugin manifests may be considered an interim solution, but at least in the short term, they are going to be around to complement what's missing in the OpenAPI spec to guide LLMs on how to consume the APIs.

[markwallace-microsoft]

@zengin We are going to close this PR and move forward with a separate package for this functionality.