.Net: Adds support for Tool calls to the Amazon Bedrock connector #11922
Conversation
…ntent This is to support microsoft#11044.
…1-support-binary-content-in-bedrock
There was a problem hiding this comment.
Note that this PR is on top of #11919. When/if that PR get merged, this PR should be rebased.
|
@RogerBarreto - didn't realize this work was already underway (this is just a PR of code already in use from my own fork). |
|
Hi there, Any ETA for these changes to be merged? Thanks! |
1 similar comment
|
Hi there, Any ETA for these changes to be merged? Thanks! |
|
Looking to leverage the changes , any ETA for merging ? |
markwallace-microsoft
added
.NET
github-actions
bot
changed the title
There was a problem hiding this comment.
This isn't something that you modified in this MR but should this line be leveraging your new convert method?
https://github.com/microsoft/semantic-kernel/pull/11922/files#diff-3e02636a8860c28c1525778de8b3d59a03a7f23b0c2a2fc06ae8c343334fd1e3R198
i.e. Instead of this:
private static ChatMessageContentItemCollection CreateChatMessageContentItemCollection(List<ContentBlock> contentBlocks)
{
var itemCollection = new ChatMessageContentItemCollection();
foreach (var contentBlock in contentBlocks)
{
itemCollection.Add(new TextContent(contentBlock.Text));
}
return itemCollection;
}To this:
private static ChatMessageContentItemCollection CreateChatMessageContentItemCollection(List<ContentBlock> contentBlocks)
{
var itemCollection = new ChatMessageContentItemCollection();
foreach (var contentBlock in contentBlocks)
{
itemCollection.Add(BedrockClientUtilities.ConvertContentBlock(contentBlock));
}
return itemCollection;
}
| yield return content; | ||
| break; | ||
|
|
||
| case ContentBlockStopEvent e: |
There was a problem hiding this comment.
I'm not sure if this is dependent on the model but with Claude 3.7 I am seeing multiple "blocks" coming back from the request stream:
I believe this is the similar to how Claude 3.7 returns the message content in 2 items in the non-streaming request?
| // If we are calling a tool, we need to process the tool call | ||
|
|
||
| // Step 1. Create a function call message | ||
| var inputJson = string.Concat(toolContents.Select(t => t.Arguments ?? string.Empty)); |
There was a problem hiding this comment.
Something doesn't feel right here. It could be because of the model or some change on the AWS API but my function call response doesn't have arguments and has the tool name, id, etc already populated:
This seems like we would just need to convert it to a ChatMessageContent.
There was a problem hiding this comment.
Maybe this would be more inline with what works for my model:
// Step 1. Create a function call message
var inputJson = string.Concat(toolContents.Select(t => t.Arguments ?? string.Empty));
var toolInput = string.IsNullOrEmpty(inputJson) ? new Document()
: JsonSerializer.Deserialize<Document>(inputJson);
var toolContent = toolContents.FirstOrDefault();
var toolUseBlock = new ToolUseBlock()
{
Name = toolName,
ToolUseId = toolId,
Input = toolInput,
};
var functionCallContent = (FunctionCallContent)BedrockClientUtilities.ConvertContentBlock(new ContentBlock
{
ToolUse = toolUseBlock,
});
var toolMessage = new ChatMessageContent(AuthorRole.Tool, [functionCallContent]);
var originalLength = workingHistory.Count;
// Step 2. Process the function call
// ...
|
@glorious-beard Thanks for the amazing contribution. As far as I know, we do have already support for i.e.: var kernel = Kernel.CreateBuilder().AddBedrockChatClient("your-model").Build()
// Same implementation as before... |
Hi, Will it not work if using:
I am currently using: Microsoft.SemanticKernel.Connectors.Amazon v1.57.0-alpha Waiting for response. Thanks! |
Hi @MustafaJamal - you def need to use the M.E.AI connector instead of the old connector (above). Using M.E.AI I can use Vision Images, tool calling, etc, etc. Here is a really basic (and verbose!) example of creating a Kernel with a Chat Completion Service that shows the steps |
Hi @davidames , Thanks for response, I tried it too, but 2 types of exception are appearing now:
Call stack: at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)
Call stack: at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream) Note that I tried with or without MaxTokensToSample in AmazonClaudeExecutionSettings. Second, I am using KernelFunction(using attributes) as plugins and none of them is without parameters and those functions are working fine with Azure OpenAI and OpenAI. Waiting for response. Thanks, |
|
Hi @MustafaJamal - function calling is working well for me using M.E.AI - but there was a lot of fiddling around to get it to work, especially around nuget packages - the versions absolutely matter! My versions are Here is my working spike code - it does manual function calling (as our main solution does too) - our prod code uses [KernelFunction] and that works too! I hope it helps! |
Hi @davidames, Many thanks for response. I found that I was using AmazonClaudeExecutionSettings instead of AnthropicPromptExecutionSettings, so it worked for me now and now no exceptions are appearing. But now no functions are being called although assistant has function information here are details:
When I asked assistant: what functions or tool call are available It answered me right: SKWorkItemSkill_CreateWorkItem Also when I asked that: Create Epic work item for User Management, generate title and description by yourself it answered this(although it waited for 2-3 seconds after sending message) but did not do any function call: ** Thinking: Generating title and description for User Management Epic Creating: Preparing to create new Epic work item Now, I'll use the SKWorkItemSkill_CreateWorkItem function to create this Epic: My Nuget API versions as same as yours. Also I checked StreamingChatMessageContent does receive function call related data, but function doesn't call: Waiting for response. Thanks, |
Hi @MustafaJamal - I really don't know - given this conversation has progressed well beyond the scope of the original PR, it's probably time to raise a new issue or discussion. |
Hello @davidames, Many thanks for your help, and yes you are right this discussion is taking long time so I have posted an issue with all the details: Thanks for valuable help :) Cheers, |
Motivation and Context
Why is this change required?
This change enables Semantic Kernel uses access to the tool call APIs in Amazon Bedrock, allowing models like Claude to use tools.
What problem does it solve?
It fixes the lack of tool calling in Amazon Bedrock and brings it closer to parity with OpenAI connectors.
What scenario does it contribute to?
Tool calls
If it fixes an open issue, please link to the issue here.
Description
This piggy backs on the tool metadata generators in FunctionCallingUtilities and PlanningUtilities to allow annotated kernel functions to be used as tools in Amazon Bedrock.
Contribution Checklist