docs/03_SDK/05_ChatTools.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 03 SDK | 05 Chat Tools\n",
    "\n",
    "## Tool Support\n",
    "\n",
    "Chat tools can work in conjunction with a large language model (LLM) such as GPT-4 by allowing developers to provide a set of function definitions to the model which helps the LLM to complete chat completion requests. \n",
    "\n",
    "### Execution Steps\n",
    "\n",
    "![Overview](../../media/img/03_SDK/05_ChatTools_Overview.png)\n",
    "\n",
    "- **(1) Create ChatCompletion Request:** A default ChatCompletion request is created using a default system and user message. Additionally, `ChatCompletionsFunctionToolDefinition` objects can be provided, which contain function descriptions, parameters and return values. The LLM can then check if any of the provided functions can help fulfill the chat completion request and return the selected functions and parameters needed to call the function to the caller.\n",
    "- **(2) Execute Function:** The caller can then execute the selected functions in its own security context and with credentials he owns.\n",
    "- **(3) Provide Function Call Results:** Results from the function calls are provided to the LLM together with the chat history to fulfill the initial request. The LLM takes the additional input data to fulfill the request.\n",
    "\n",
    "\n",
    "### Further information\n",
    "\n",
    "- [MS Learn - How to use function calling with Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/function-calling)\n",
    "- [OpenAI - Function calling](https://platform.openai.com/docs/guides/function-calling)\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Azure Environment\n",
    "\n",
    "To execute the sample code Azure service specific information like endpoint, api key etc. is needed. ([Details and instructions can be found here](../01_DemoEnvironment/01_Environment.ipynb)). Function calling is currently in preview and is supported by the latest GPT-35-Turbo and GPT-4 models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Create OpenAIClient\n",
    "\n",
    "The OpenAIClient from Azure.AI.OpenAI is a .NET client library that acts as the centralized point for all .NET functionality that want to interact with a deployed Azure OpenAI Large Language Model. It provides methods to access the OpenAI REST APIs for various tasks such as text completion, text embedding, and chat completion, etc.. It also allows developers to specify the model, engine, and options for each request, such as temperature, frequency penalty, presence penalty, and stop sequences. \n",
    "\n",
    "The OpenAIClient can connect to any Azure OpenAI resource or to the non-Azure OpenAI inference endpoint, making it a versatile and powerful tool for .NET development with OpenAI.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div></div><div></div><div><strong>Installed Packages</strong><ul><li><span>Azure.AI.OpenAI, 1.0.0-beta.12</span></li><li><span>DotNetEnv, 2.5.0</span></li></ul></div></div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI Client created...\r\n"
     ]
    }
   ],
   "source": [
    "#r \"nuget: Azure.AI.OpenAI, 1.0.0-beta.12\"\n",
    "#r \"nuget: DotNetEnv, 2.5.0\"\n",
    "\n",
    "using Azure; \n",
    "using Azure.AI.OpenAI;\n",
    "using DotNetEnv;\n",
    "using System.IO;\n",
    "using System.Text.Json; \n",
    "\n",
    "//configuration file is created during environment creation\n",
    "//if you skipped the deployment just remove the code and provide values from your deployment\n",
    "static string _configurationFile = @\"../01_DemoEnvironment/conf/application.env\";\n",
    "Env.Load(_configurationFile);\n",
    "\n",
    "string oAiApiKey = Environment.GetEnvironmentVariable(\"SKIT_AOAI1106_APIKEY\") ?? \"SKIT_AOAI1106_APIKEY not found\";\n",
    "string oAiEndpoint = Environment.GetEnvironmentVariable(\"SKIT_AOAI1106_ENDPOINT\") ?? \"SKIT_AOAI1106_ENDPOINT not found\";\n",
    "string chatCompletionDeploymentName = Environment.GetEnvironmentVariable(\"SKIT_AOAI1106_DEPLOYMENTNAME\") ?? \"SKIT_AOAI1106_DEPLOYMENTNAME not found\";\n",
    "string assetsFolder = Path.Combine(Directory.GetCurrentDirectory(), \"..\", \"..\", \"assets\");\n",
    "\n",
    "AzureKeyCredential azureKeyCredential = new AzureKeyCredential(oAiApiKey);\n",
    "OpenAIClient openAIClient = new OpenAIClient(new Uri(oAiEndpoint), azureKeyCredential);\n",
    "\n",
    "\n",
    "Console.WriteLine($\"OpenAI Client created...\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "```\n",
    "Installed Packages\n",
    "    Azure.AI.OpenAI, 1.0.0-beta.12\n",
    "    DotNetEnv, 2.5.0\n",
    "\n",
    "OpenAI Client created...\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Create Tool Definitions\n",
    "\n",
    "The `ChatCompletionsFunctionToolDefinition` class can be used to describe locally available functions that can help the LLM to retrieve additional information to fulfill a requested. This class provides information such as the function description, parameters, and return values, which are used to explain the function and its behavior to the LLM.\n",
    "\n",
    "In this example two 'tools' are defined:\n",
    "- `toolUserDetails`: can be leveraged to retrieve additional information about a specific user.\n",
    "- `toolOrderDetails`: can be leveraged to retrieve expected delivery dates of orders."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tool: SDK_Order_GetOrderDeliveryDate() defined\n",
      "Tool: SDK_Customer_GetCustomerDetails() defined\n"
     ]
    }
   ],
   "source": [
    "//Define tool(s)\n",
    "ChatCompletionsFunctionToolDefinition toolUserDetails = new ChatCompletionsFunctionToolDefinition() {\n",
    "    Name = \"SKit_Customer_GetCustomerDetails\",\n",
    "    Description = \"Get customer details. Function returns all kind of information of registered customers this include first name and sure name\",\n",
    "    Parameters = BinaryData.FromObjectAsJson(\n",
    "        new {\n",
    "            Type = \"object\",\n",
    "            Properties = new {\n",
    "                Id = new {\n",
    "                    Type = \"string\",\n",
    "                    Description = \"The customer Id (i.e. DK3-008AF)\"\n",
    "                }\n",
    "            },\n",
    "            Required = new[] { \n",
    "                \"Id\" \n",
    "            }\n",
    "        },\n",
    "        new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }\n",
    "    )   \n",
    "}; \n",
    "\n",
    "ChatCompletionsFunctionToolDefinition toolOrderDetails = new ChatCompletionsFunctionToolDefinition() {\n",
    "    Name = \"SKit_Order_GetOrderDeliveryDate\",\n",
    "    Description = \"Provides the expected delivery data of an order.\",\n",
    "    Parameters = BinaryData.FromObjectAsJson(\n",
    "        new {\n",
    "            Type = \"object\",\n",
    "            Properties = new {\n",
    "                OrderId = new {\n",
    "                    Type = \"string\",\n",
    "                    Description = \"The ID of the order (i.e. 4711)\"\n",
    "                },\n",
    "                CustomerId = new {\n",
    "                    Type = \"string\",\n",
    "                    Description = \"The ID or name of the person who has ordered goods\"\n",
    "                }\n",
    "            },\n",
    "            Required = new[] { \n",
    "                \"OrderId\" \n",
    "            }\n",
    "        },\n",
    "        new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }\n",
    "    )   \n",
    "}; \n",
    "\n",
    "Console.WriteLine(\"Tool: SDK_Order_GetOrderDeliveryDate() defined\");\n",
    "Console.WriteLine(\"Tool: SDK_Customer_GetCustomerDetails() defined\");\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output: \n",
    "```\n",
    "Tool: SDK_Order_GetOrderDeliveryDate() defined\n",
    "Tool: SDK_Customer_GetCustomerDetails() defined\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Compose ChatCompletionsOptions\n",
    "\n",
    "Each chat would follow similar structure, where _System_, _Agent_ and _User_ messages are added in sequence. Parameters, such as _Temperature_ could be set per call. Additionally to system message and user message a list of `ChatCompletionsFunctionToolDefinition` instances which define tool functions can be provided.\n",
    "\n",
    "In this example the two previously defined tools `toolUserDetails` and `toolOrderDetails` are added to the `ChatCompletionsOptions` instance.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ChatCompletionsOptions created...\r\n"
     ]
    }
   ],
   "source": [
    "//Define System Prompt\n",
    "string systemMessage = @\"You are an AI assistant that helps customers find information about their orders. \n",
    "    There's no need to ask for permission if you need additional tooling to fulfill the information request. \n",
    "    You answer with the first name and sure name of the customer.\n",
    "    You always answer with: 'We expect your order to arrive at: {expected time of arrival}. This information was created for: {first name} {last name}'\";\n",
    "\n",
    "string userMessage = @\"Hey, here's John.\n",
    "    I'm calling on behalf of Contoso and I need information for regarding my order number 4711.\n",
    "    The order was placed on March 1st where customer number is 984-AB489. Can you tell me when it will be delivered?\n",
    "\";\n",
    "\n",
    "//Define chat completion options\n",
    "var chatCompletionsOptions = new ChatCompletionsOptions()\n",
    "{\n",
    "    DeploymentName = chatCompletionDeploymentName,\n",
    "    Messages = { \n",
    "        new ChatRequestSystemMessage(systemMessage),\n",
    "        new ChatRequestUserMessage(userMessage) \n",
    "    },\n",
    "    ToolChoice = ChatCompletionsToolChoice.Auto,\n",
    "    Tools = { \n",
    "        toolUserDetails, \n",
    "        toolOrderDetails\n",
    "    },\n",
    "};\n",
    "\n",
    "\n",
    "Console.WriteLine($\"ChatCompletionsOptions created...\");\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "```\n",
    "ChatCompletionsOptions created...\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Call ChatCompletion API\n",
    "\n",
    "The first interaction with the LLM is to call the `GetChatCompletionAsync()` method to get the response from the LLM. Based on the provided system and user message we expect that LLM to return with a request to call additional locally available functions. The LLM provides the information that functions should be called by setting `chatChoice.FinishReson == CompletionsFinishReason.ToolCalls`. Note that function name and function call parameters are provided. The code cell is just showing the functions which should be called but doesn't call them yet. This will be done in one of the following cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LLM call finish reason: tool_calls\n",
      "Function call: SKit_Customer_GetCustomerDetails with arguments: {\"id\": \"984-AB489\"}\n",
      "Function call: SKit_Order_GetOrderDeliveryDate with arguments: {\"orderId\": \"4711\", \"customerId\": \"984-AB489\"}\n"
     ]
    }
   ],
   "source": [
    "//LLM Call\n",
    "OpenAIClient openAIClient = new OpenAIClient(new Uri(oAiEndpoint), new AzureKeyCredential(oAiApiKey));\n",
    "Response<ChatCompletions> response = await openAIClient.GetChatCompletionsAsync(chatCompletionsOptions);\n",
    "\n",
    "//Tool Function Call\n",
    "Console.WriteLine($\"LLM call finish reason: {response.Value.Choices[0].FinishReason}\");\n",
    "\n",
    "List<ChatRequestToolMessage> chatRequestToolMessages = new List<ChatRequestToolMessage>();\n",
    "ChatChoice chatChoice = response.Value.Choices[0]; \n",
    "\n",
    "//List functions which should be called\n",
    "if (chatChoice.FinishReason == CompletionsFinishReason.ToolCalls) {\n",
    "    foreach (ChatCompletionsToolCall toolCall in chatChoice.Message.ToolCalls)\n",
    "    {\n",
    "        if (toolCall is ChatCompletionsFunctionToolCall functionToolCall)\n",
    "        {\n",
    "            if (functionToolCall != null)\n",
    "            {\n",
    "                Console.WriteLine($\"Function call: {functionToolCall.Name} with arguments: {functionToolCall.Arguments}\");\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "```\n",
    "LLM call finish reason: tool_calls\n",
    "Function call: SKit_Customer_GetCustomerDetails with arguments: {\"id\": \"984-AB489\"}\n",
    "Function call: SKit_Order_GetOrderDeliveryDate with arguments: {\"orderId\": \"4711\", \"customerId\": \"984-AB489\"}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Define Helper Class\n",
    "\n",
    "In the previous step the LLM response provided information which function need to be called. This cell defines a helper function to call functions from DLLs using reflection. The library which is dynamically loaded is build using this source:\n",
    "\n",
    "```csharp\n",
    "namespace SKit; \n",
    "\n",
    "using System.Text.Json;\n",
    "\n",
    "public class Order\n",
    "{\n",
    "    public DateTime GetOrderDeliveryDate(string orderId, string? customerId = null){\n",
    "        return DateTime.Now.AddDays(5);\n",
    "    }\n",
    "}\n",
    "\n",
    "public class Customer\n",
    "{\n",
    "    public string GetCustomerDetails(string id)\n",
    "    {\n",
    "        //convert to json string\n",
    "\n",
    "        return JsonSerializer.Serialize(new {\n",
    "            FirstName = \"Jon\",\n",
    "            LastName = \"Doe\",\n",
    "            Company = \"Contoso\",\n",
    "            Title = \"CTO\"\n",
    "        });\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "Function name and parameter retrieved from the LLM are used to locate the specific function in the dll, instantiate the hosting class and finally execute it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Helper to call function via reflection from dll defined...\r\n"
     ]
    }
   ],
   "source": [
    "using System.IO;\n",
    "using System.Reflection;\n",
    "using System.Text.Json;\n",
    "\n",
    "#nullable enable\n",
    "private string CallFunctionViaReflection(string name, string arguments)\n",
    "{\n",
    "    name = name.Replace(\"_\", \".\");\n",
    "    string typeName = name.Substring(0, name.LastIndexOf(\".\"));\n",
    "    string methodName = name.Substring(name.LastIndexOf(\".\") + 1);\n",
    "\n",
    "    string dllFilePath = Path.Combine(assetsFolder, \"docs\", \"03_SDK\", \"FunctionCollection.dll\");\n",
    "    Assembly assembly = Assembly.LoadFrom(dllFilePath);\n",
    "    Type? type = assembly.GetType(typeName);\n",
    "\n",
    "    MethodInfo? method = type?.GetMethod(methodName);\n",
    "    object? instance = null;\n",
    "    if (type!=null) {\n",
    "        instance = Activator.CreateInstance(type);\n",
    "    }\n",
    "\n",
    "    JsonElement jsonElement = JsonDocument.Parse(arguments).RootElement;\n",
    "    int i = 0;\n",
    "    if (method == null) { return \"\"; }\n",
    "    object[] methodArguments = new object[method.GetParameters().Length];\n",
    "    foreach (var parameter in method.GetParameters())\n",
    "    {\n",
    "        if (parameter.Name == null) { continue; }\n",
    "        var parameterValue = jsonElement.GetProperty(parameter.Name).ToString();\n",
    "        methodArguments[i++] = parameterValue;\n",
    "    }\n",
    "\n",
    "    if (method != null && instance != null) {\n",
    "        object? result = method.Invoke(instance, methodArguments);\n",
    "        return Convert.ToString(result) ?? \"\";\n",
    "    }\n",
    "    \n",
    "    return \"\"; \n",
    "}\n",
    "\n",
    "Console.WriteLine($\"Helper to call function via reflection from dll defined...\");\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Steps 6: Call Local Function(s)\n",
    "\n",
    "The previously defined helper class performs function calls suggested by the LLM response. Results from the function calls are added as `ChatRequestToolMessage` to a list of `ChatRequestToolMessage`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Function call: SKit_Customer_GetCustomerDetails with arguments: {\"id\": \"984-AB489\"} returned {\"FirstName\":\"Jon\",\"LastName\":\"Doe\",\"Company\":\"Contoso\",\"Title\":\"CTO\"}\n",
      "Function call: SKit_Order_GetOrderDeliveryDate with arguments: {\"orderId\": \"4711\", \"customerId\": \"984-AB489\"} returned 2/5/2024 2:01:03 PM\n"
     ]
    }
   ],
   "source": [
    "List<ChatRequestToolMessage> chatRequestToolMessages = new List<ChatRequestToolMessage>();\n",
    "if (chatChoice.FinishReason == CompletionsFinishReason.ToolCalls) {\n",
    "    foreach (ChatCompletionsToolCall toolCall in chatChoice.Message.ToolCalls)\n",
    "    {\n",
    "        if (toolCall is ChatCompletionsFunctionToolCall functionToolCall)\n",
    "        {\n",
    "            if (functionToolCall != null)\n",
    "            {\n",
    "                string functionReturnValue = CallFunctionViaReflection(functionToolCall.Name, functionToolCall.Arguments);\n",
    "                Console.WriteLine($\"Function call: {functionToolCall.Name} with arguments: {functionToolCall.Arguments} returned {functionReturnValue}\");\n",
    "                chatRequestToolMessages.Add(\n",
    "                    new ChatRequestToolMessage(\n",
    "                        functionReturnValue, \n",
    "                        toolCall.Id\n",
    "                    )\n",
    "                );\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "```\n",
    "Function call: SKit_Customer_GetCustomerDetails with arguments: {\"id\": \"984-AB489\"} returned {\"FirstName\":\"Jon\",\"LastName\":\"Doe\",\"Company\":\"Contoso\",\"Title\":\"CTO\"}\n",
    "Function call: SKit_Order_GetOrderDeliveryDate with arguments: {\"orderId\": \"4711\", \"customerId\": \"984-AB489\"} returned 2/4/2024 6:23:22 PM\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 7: Finalize LLM Call\n",
    "\n",
    "In the final step we provide the results from the local function calls to a new chat completion request. Note that the LLM incorporates information from both function calls in it's response. \n",
    "\n",
    "It provides 'Doe' even if the initial user message just mentions 'Hey, here's John' as well as the estimated delivery date of the order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We expect your order to arrive at: 2/5/2024 2:01:03 PM. This information was created for: Jon Doe\r\n"
     ]
    }
   ],
   "source": [
    "if (chatChoice.FinishReason == CompletionsFinishReason.ToolCalls)\n",
    "{\n",
    "    // Create conversation history\n",
    "    ChatRequestAssistantMessage chatRequestAssistantMessage = new(chatChoice.Message);\n",
    "    chatCompletionsOptions.Messages.Add(chatRequestAssistantMessage);\n",
    "\n",
    "    // Provide function call results\n",
    "    foreach (var chatRequestToolMessage in chatRequestToolMessages) {\n",
    "        chatCompletionsOptions.Messages.Add(chatRequestToolMessage);\n",
    "    }\n",
    "\n",
    "    response = await openAIClient.GetChatCompletionsAsync(chatCompletionsOptions);\n",
    "    Console.WriteLine(response.Value.Choices[0].Message.Content); \n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "```\n",
    "We expect your order to arrive at: 2/4/2024 6:23:22 PM. This information was created for: Jon Doe\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "- The concept of Embeddings allows transforming information into a numerical representation preserving the semantic context of the information: [Demo Embeddings](../04_Embeddings/01_BasicEmbeddings.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".NET (C#)",
   "language": "C#",
   "name": ".net-csharp"
  },
  "language_info": {
   "name": "polyglot-notebook"
  },
  "polyglot_notebook": {
   "kernelInfo": {
    "defaultKernelName": "csharp",
    "items": [
     {
      "aliases": [],
      "languageName": "csharp",
      "name": "csharp"
     }
    ]
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}