samples/dotnet/extraction/text/document-extraction-gpt.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Document Extraction with Azure AI Document Intelligence and Azure OpenAI GPT-4o (Text Only)\n",
    "\n",
    "This sample demonstrates how to extract structured data from any document using Azure AI Document Intelligence and Azure OpenAI GPT models.\n",
    "\n",
    "![Data Extraction](../../../../images/extraction-document-intelligence-openai.png)\n",
    "\n",
    "This is achieved by the following process:\n",
    "\n",
    "- Analyze a document using Azure AI Document Intelligence's `prebuilt-layout` model to extract the structure as Markdown.\n",
    "- Construct a system prompt that defines the instruction for extracting structured data from documents.\n",
    "- Construct a user prompt that includes specific extraction instruction for the type of document, and the Markdown content of the document.\n",
    "- Use the Azure OpenAI chat completions API with the GPT-4o model to generate a structured output from the content.\n",
    "\n",
    "## Objectives\n",
    "\n",
    "By the end of this sample, you will have learned how to:\n",
    "\n",
    "- Convert a document to Markdown format using Azure AI Document Intelligence.\n",
    "- Use prompt engineering techniques to instruct GPT-4o to extract structured data from a type of document.\n",
    "- Use the [Structured Outputs feature](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs?tabs=python-secure) to extract structured data from a document using Azure OpenAI's GPT-4o model.\n",
    "- Use the analysis result from Azure AI Document Intelligence to determine the confidence of the extracted structured output.\n",
    "- Use the [logprobs](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#request-body:~:text=False-,logprobs,-integer) parameter in an OpenAI request to determine the confidence of the extracted structured output.\n",
    "\n",
    "## Useful Tips\n",
    "\n",
    "- Combine this technique with a [page classification](../../classification/README.md) approach to reduce the number of pages to extract from to only those that match your criteria for extraction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import modules\n",
    "\n",
    "This sample takes advantage of the following .NET dependencies:\n",
    "\n",
    "- **Azure.AI.DocumentIntelligence** to interface with the Azure AI Document Intelligence API for analyzing documents.\n",
    "- **Azure.AI.OpenAI** to interface with the Azure OpenAI chat completions API to generate structured extraction outputs using the GPT-4o model.\n",
    "- **Azure.Identity** to securely authenticate with deployed Azure Services using Microsoft Entra ID credentials.\n",
    "\n",
    "The following local components are also used:\n",
    "\n",
    "- [**Invoice**](../../modules/samples/models/Invoice.csx) to provide the expected structured output JSON schema for invoice documents.\n",
    "- [**OpenAIStructuredOutputsHelpers**](../../modules/samples/helpers/OpenAIStructuredOutputsHelpers.csx) to generate structured outputs from the OpenAI API and provide parsing functions for the response output.\n",
    "- [**AccuracyEvaluator**](../../modules/samples/evaluation/AccuracyEvaluator.csx) to evaluate the output of the extraction process with expected results.\n",
    "- [**DocumentIntelligenceConfidence**](../../modules/samples/confidence/DocumentIntelligenceConfidence.csx) to calculate the confidence of the extraction process based on the analysis result from the Azure AI Document Intelligence API request.\n",
    "- [**OpenAIConfidence**](../../modules/samples/confidence/OpenAIConfidence.csx) to calculate the confidence of the extraction process based on the `logprobs` response from the OpenAI API request using the `Microsoft.ML.Tokenizers` and `Microsoft.ML.Tokenizers.Data.O200kBase` libraries (the latter is required for tokenizers used for the GPT-4o model).\n",
    "- [**DocumentProcessingResult**](../../modules/samples/models/DocumentProcessingResult.csx) to store the results of the extraction process as a file.\n",
    "- [**AppSettings**](../../modules/samples/AppSettings.csx) to access environment variables from the `.env` file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div></div><div></div><div><strong>Installed Packages</strong><ul><li><span>Azure.AI.DocumentIntelligence, 1.0.0</span></li><li><span>Azure.AI.OpenAI, 2.1.0</span></li><li><span>Azure.Identity, 1.13.2</span></li><li><span>DotNetEnv, 3.1.1</span></li><li><span>Microsoft.ML.Tokenizers, 1.0.2</span></li><li><span>Microsoft.ML.Tokenizers.Data.O200kBase, 1.0.2</span></li></ul></div></div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#r \"nuget: Azure.Identity, 1.13.2\"\n",
    "#r \"nuget: Azure.AI.OpenAI, 2.1.0\"\n",
    "#r \"nuget: Azure.AI.DocumentIntelligence, 1.0.0\"\n",
    "#r \"nuget: DotNetEnv, 3.1.1\"\n",
    "#r \"nuget: Microsoft.ML.Tokenizers, 1.0.2\"\n",
    "#r \"nuget: Microsoft.ML.Tokenizers.Data.O200kBase, 1.0.2\"\n",
    "\n",
    "#!import ../../modules/samples/AppSettings.csx\n",
    "#!import ../../modules/samples/helpers/OpenAIStructuredOutputsHelpers.csx\n",
    "#!import ../../modules/samples/helpers/StopwatchContext.csx\n",
    "#!import ../../modules/samples/models/Invoice.csx\n",
    "#!import ../../modules/samples/models/DocumentProcessingResult.csx\n",
    "#!import ../../modules/samples/evaluation/AccuracyEvaluator.csx\n",
    "#!import ../../modules/samples/confidence/ConfidenceHelpers.csx\n",
    "#!import ../../modules/samples/confidence/OpenAIConfidence.csx\n",
    "#!import ../../modules/samples/confidence/DocumentIntelligenceConfidence.csx\n",
    "\n",
    "using System;\n",
    "using System.IO;\n",
    "using System.Text.Json;\n",
    "using Azure;\n",
    "using Azure.Core;\n",
    "using Azure.Identity;\n",
    "using Azure.AI.OpenAI;\n",
    "using Azure.AI.OpenAI.Chat;\n",
    "using Azure.AI.DocumentIntelligence;\n",
    "using OpenAI;\n",
    "using OpenAI.Chat;\n",
    "using DotNetEnv;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure the Azure services\n",
    "\n",
    "To use Azure AI Document Intelligence and Azure OpenAI, their SDKs are used to create a client instance using a deployed endpoint and authentication credentials.\n",
    "\n",
    "For this sample, the credentials of the Azure CLI are used to authenticate with the deployed services."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "string workingDir = Path.GetFullPath(\"../../../../\");\n",
    "AppSettings settings = new AppSettings(new Dictionary<string, string>(Env.Load(Path.Combine(workingDir, \".env\"))));\n",
    "string samplePath = Path.Combine(workingDir, \"samples/dotnet/extraction/text/\");\n",
    "string sampleName = \"document-extraction-gpt\";\n",
    "\n",
    "DefaultAzureCredential credential = new DefaultAzureCredential(\n",
    "    new DefaultAzureCredentialOptions { \n",
    "        ExcludeWorkloadIdentityCredential = true,\n",
    "        ExcludeAzureDeveloperCliCredential = true,\n",
    "        ExcludeEnvironmentCredential = true,\n",
    "        ExcludeManagedIdentityCredential = true,\n",
    "        ExcludeAzurePowerShellCredential = true,\n",
    "        ExcludeSharedTokenCacheCredential = true,\n",
    "        ExcludeInteractiveBrowserCredential = true\n",
    "    }\n",
    ");\n",
    "\n",
    "AzureOpenAIClient openaiClient = new AzureOpenAIClient(\n",
    "    new Uri(settings.OpenAIEndpoint),\n",
    "    credential\n",
    ");\n",
    "\n",
    "var documentIntelligenceClient = new DocumentIntelligenceClient(\n",
    "    new Uri(settings.AIServicesEndpoint),\n",
    "    credential\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Establish the expected output\n",
    "\n",
    "To compare the accuracy of the extraction process, the expected output of the extraction process has been defined in the following code block based on the details of an [Invoice](../../../assets/invoices/invoice_1.pdf).\n",
    "\n",
    "> **Note**: More invoice examples can be found in the [assets folder](../../../assets/invoices). These examples include the PDF file and an associated JSON metadata file that provides the expected structured output. You can add your own scenarios by following the same structure.\n",
    "\n",
    "The expected output has been defined by a human evaluating the document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "string path = Path.Combine(workingDir, \"samples/assets/invoices/\");\n",
    "string metadataFName = \"invoice_1.json\"; // Change this to the file you want to evaluate\n",
    "string metadataFPath = Path.Combine(path, metadataFName);\n",
    "\n",
    "Dictionary<string, JsonElement> data = JsonSerializer.Deserialize<Dictionary<string, JsonElement>>(File.ReadAllText(metadataFPath));\n",
    "\n",
    "Invoice expected = data[\"expected\"].Deserialize<Invoice>(\n",
    "    new JsonSerializerOptions(JsonSerializerDefaults.Web)\n",
    "    {\n",
    "        PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower\n",
    "    }\n",
    ");\n",
    "string pdfFName = data[\"fname\"].GetString();\n",
    "string pdfFPath = Path.Combine(path, pdfFName);\n",
    "\n",
    "AccuracyEvaluator<Invoice> invoiceEvaluator = new AccuracyEvaluator<Invoice>(matchKeys: new List<string>() { \"ProductCode\", \"Description\" });"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract data from the document\n",
    "\n",
    "The following code block executes the data extraction process using Azure AI Document Intelligence and Azure OpenAI's GPT-4o model.\n",
    "\n",
    "It performs the following steps:\n",
    "\n",
    "1. Get the document bytes from the provided file path. _Note: In this example, we are processing a local document, however, you can use any document storage location of your choice, such as Azure Blob Storage._\n",
    "2. Use Azure AI Document Intelligence to analyze the structure of the document and convert it to Markdown format using the pre-built layout model.\n",
    "3. Using Azure OpenAI's GPT-4o model and its [Structured Outputs feature](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs?tabs=python-secure), extract a structured data transfer object (DTO) from the content of the Markdown."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "AnalyzeResult result;\n",
    "\n",
    "StopwatchContext diSw;\n",
    "\n",
    "using (diSw = new StopwatchContext())\n",
    "{\n",
    "    var pollerResult = await documentIntelligenceClient.AnalyzeDocumentAsync(\n",
    "        WaitUntil.Completed,\n",
    "        options: new AnalyzeDocumentOptions(\n",
    "            modelId: \"prebuilt-layout\",\n",
    "            bytesSource: BinaryData.FromBytes(File.ReadAllBytes(pdfFPath)))\n",
    "        {\n",
    "            OutputContentFormat = DocumentContentFormat.Markdown\n",
    "        }\n",
    "    );\n",
    "\n",
    "    result = pollerResult.Value;\n",
    "}\n",
    "\n",
    "string markdown = result.Content;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "string systemPrompt = \"You are an AI assistant that extracts data from documents.\";"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "List<ChatMessageContentPart> userContent = new List<ChatMessageContentPart>();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "StringBuilder userTextPromptBuilder = new StringBuilder();\n",
    "userTextPromptBuilder.AppendLine(\"Extract the data from this invoice.\");\n",
    "userTextPromptBuilder.AppendLine(\"- If a value is not present, provide null.\");\n",
    "userTextPromptBuilder.AppendLine(\"- Dates should be in the format YYYY-MM-DD.\");\n",
    "\n",
    "string userTextPrompt = userTextPromptBuilder.ToString();\n",
    "\n",
    "userContent.Add(ChatMessageContentPart.CreateTextPart(userTextPrompt));\n",
    "userContent.Add(ChatMessageContentPart.CreateTextPart(markdown));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "ParsedChatCompletion<Invoice> completion;\n",
    "\n",
    "StopwatchContext oaiSw;\n",
    "\n",
    "using (oaiSw = new StopwatchContext())\n",
    "{\n",
    "    completion = await openaiClient\n",
    "        .GetChatClient(settings.GPT4OModelDeploymentName)\n",
    "        .CompleteChatAsync(\n",
    "            [\n",
    "                new SystemChatMessage(systemPrompt),\n",
    "                new UserChatMessage(userContent)\n",
    "            ],\n",
    "            new ChatCompletionOptions\n",
    "            {   \n",
    "                ResponseFormat = CreateJsonSchemaFormat<Invoice>(\"invoice\", jsonSchemaIsStrict: true),\n",
    "                MaxOutputTokenCount = 4096,\n",
    "                Temperature = 0.1f,\n",
    "                TopP = 0.1f,\n",
    "                IncludeLogProbabilities = true\n",
    "            }\n",
    "        );\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Understanding the Structured Outputs JSON schema\n",
    "\n",
    "Using [`OpenAIJsonSchema.For`](../../modules/samples/helpers/OpenAIStructuredOutputsHelpers.csx), the [Invoice](../../modules/samples/models/Invoice.csx) data model is converted to a valid OpenAI supported JSON schema when applied to the `ResponseFormat` parameter of the OpenAI chat completions request.\n",
    "\n",
    "The JSON schema is used to instruct the GPT-4o model to generate a strict output that adheres to the structure defined. The approach makes it easier for developers to manage the data structure in code, with helpful descriptions that will be included in the final JSON schema.\n",
    "\n",
    "Demonstrated below, you can see how the Invoice data model is understood by the OpenAI request:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{\n",
       "  \"type\": \"object\",\n",
       "  \"properties\": {\n",
       "    \"CustomerName\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Name of the customer being invoiced, e.g. Company A\"\n",
       "    },\n",
       "    \"CustomerAddress\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/InvoiceAddress\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Full address of the customer, e.g. 123 Main St., City, Country\"\n",
       "    },\n",
       "    \"CustomerTaxId\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Government tax ID of the customer, e.g. 123456789\"\n",
       "    },\n",
       "    \"ShippingAddress\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/InvoiceAddress\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Full address of the shipping location for the customer (null if the same as customer address), e.g. 123 Main St., City, Country\"\n",
       "    },\n",
       "    \"PurchaseOrder\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Purchase order reference number, e.g. PO-1234\"\n",
       "    },\n",
       "    \"InvoiceId\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Reference ID for the invoice (often invoice number), e.g. INV-1234\"\n",
       "    },\n",
       "    \"InvoiceDate\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Date the invoice was issued or delivered, e.g., 2021-01-01\"\n",
       "    },\n",
       "    \"PayableBy\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Date when the invoice should be paid, e.g., 2021-01-15\"\n",
       "    },\n",
       "    \"VendorName\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Name of the vendor issuing the invoice, e.g. Company B\"\n",
       "    },\n",
       "    \"VendorAddress\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/InvoiceAddress\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Full address of the vendor, e.g. 123 Main St., City, Country\"\n",
       "    },\n",
       "    \"VendorTaxId\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Government tax ID of the vendor, e.g. 123456789\"\n",
       "    },\n",
       "    \"RemittanceAddress\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/InvoiceAddress\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Full address for remitting payment to the vendor (null if the same as vendor address), e.g. 123 Main St., City, Country\"\n",
       "    },\n",
       "    \"Subtotal\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"type\": \"number\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Subtotal amount before discounts and taxes, e.g. 100.00\"\n",
       "    },\n",
       "    \"TotalDiscount\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"type\": \"number\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Total discount amount applied to the invoice, e.g. 10.00\"\n",
       "    },\n",
       "    \"TotalTax\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"type\": \"number\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Total tax amount applied to the invoice, e.g. 18.00\"\n",
       "    },\n",
       "    \"InvoiceTotal\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"type\": \"number\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Total amount due on the invoice, e.g. 108.00\"\n",
       "    },\n",
       "    \"PaymentTerms\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"Payment terms for the invoice, e.g. Net 30\"\n",
       "    },\n",
       "    \"Items\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"type\": \"array\",\n",
       "          \"items\": {\n",
       "            \"anyOf\": [\n",
       "              {\n",
       "                \"$ref\": \"#/$defs/InvoiceItem\"\n",
       "              },\n",
       "              {\n",
       "                \"type\": \"null\"\n",
       "              }\n",
       "            ]\n",
       "          }\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"List of line items on the invoice\"\n",
       "    },\n",
       "    \"TotalItemQuantity\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"type\": \"number\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Total quantity of line items on the invoice\"\n",
       "    },\n",
       "    \"ItemsCustomerSignature\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/InvoiceSignature\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Signature of the customer for the line items on the invoice\"\n",
       "    },\n",
       "    \"ItemsVendorSignature\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/InvoiceSignature\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Signature of the vendor for the line items on the invoice\"\n",
       "    },\n",
       "    \"Returns\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"type\": \"array\",\n",
       "          \"items\": {\n",
       "            \"anyOf\": [\n",
       "              {\n",
       "                \"$ref\": \"#/$defs/InvoiceItem\"\n",
       "              },\n",
       "              {\n",
       "                \"type\": \"null\"\n",
       "              }\n",
       "            ]\n",
       "          }\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"List of line items being returned\"\n",
       "    },\n",
       "    \"TotalReturnQuantity\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"type\": \"number\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Total quantity of line items being returned\"\n",
       "    },\n",
       "    \"ReturnsCustomerSignature\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/InvoiceSignature\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Signature of the customer for the line items being returned\"\n",
       "    },\n",
       "    \"ReturnsVendorSignature\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/InvoiceSignature\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"Signature of the vendor for the line items being returned\"\n",
       "    }\n",
       "  },\n",
       "  \"required\": [\n",
       "    \"CustomerName\",\n",
       "    \"CustomerAddress\",\n",
       "    \"CustomerTaxId\",\n",
       "    \"ShippingAddress\",\n",
       "    \"PurchaseOrder\",\n",
       "    \"InvoiceId\",\n",
       "    \"InvoiceDate\",\n",
       "    \"PayableBy\",\n",
       "    \"VendorName\",\n",
       "    \"VendorAddress\",\n",
       "    \"VendorTaxId\",\n",
       "    \"RemittanceAddress\",\n",
       "    \"Subtotal\",\n",
       "    \"TotalDiscount\",\n",
       "    \"TotalTax\",\n",
       "    \"InvoiceTotal\",\n",
       "    \"PaymentTerms\",\n",
       "    \"Items\",\n",
       "    \"TotalItemQuantity\",\n",
       "    \"ItemsCustomerSignature\",\n",
       "    \"ItemsVendorSignature\",\n",
       "    \"Returns\",\n",
       "    \"TotalReturnQuantity\",\n",
       "    \"ReturnsCustomerSignature\",\n",
       "    \"ReturnsVendorSignature\"\n",
       "  ],\n",
       "  \"additionalProperties\": false,\n",
       "  \"$defs\": {\n",
       "    \"InvoiceAddress\": {\n",
       "      \"type\": \"object\",\n",
       "      \"properties\": {\n",
       "        \"Street\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"Street address, e.g. 123 Main St.\"\n",
       "        },\n",
       "        \"City\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"City, e.g. New York.\"\n",
       "        },\n",
       "        \"State\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"State, e.g. NY.\"\n",
       "        },\n",
       "        \"PostalCode\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"Postal code, e.g. 10001.\"\n",
       "        },\n",
       "        \"Country\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"Country, e.g. USA.\"\n",
       "        }\n",
       "      },\n",
       "      \"required\": [\n",
       "        \"Street\",\n",
       "        \"City\",\n",
       "        \"State\",\n",
       "        \"PostalCode\",\n",
       "        \"Country\"\n",
       "      ],\n",
       "      \"additionalProperties\": false\n",
       "    },\n",
       "    \"InvoiceItem\": {\n",
       "      \"type\": \"object\",\n",
       "      \"properties\": {\n",
       "        \"ProductCode\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"Product code, product number, or SKU associated with the line item, e.g. SKU-123\"\n",
       "        },\n",
       "        \"ProductDescription\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"Description of the line item, e.g. Product A\"\n",
       "        },\n",
       "        \"Quantity\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"integer\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"Quantity of the line item\"\n",
       "        },\n",
       "        \"Tax\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"number\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"Tax amount applied to the line item, e.g. 6.99\"\n",
       "        },\n",
       "        \"TaxRate\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"Tax rate applied to the line item, e.g. 18%\"\n",
       "        },\n",
       "        \"UnitPrice\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"number\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"Net or gross price of one unit of the line item, e.g. 9.99\"\n",
       "        },\n",
       "        \"Total\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"number\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"Total charges associated with the line item, e.g. 19.98\"\n",
       "        },\n",
       "        \"Reason\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"Reason for returning the line item, e.g. Damaged\"\n",
       "        }\n",
       "      },\n",
       "      \"required\": [\n",
       "        \"ProductCode\",\n",
       "        \"ProductDescription\",\n",
       "        \"Quantity\",\n",
       "        \"Tax\",\n",
       "        \"TaxRate\",\n",
       "        \"UnitPrice\",\n",
       "        \"Total\",\n",
       "        \"Reason\"\n",
       "      ],\n",
       "      \"additionalProperties\": false\n",
       "    },\n",
       "    \"InvoiceSignature\": {\n",
       "      \"type\": \"object\",\n",
       "      \"properties\": {\n",
       "        \"Signatory\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"Name of the person who signed the invoice.\"\n",
       "        },\n",
       "        \"IsSigned\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"boolean\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"Indicates whether the invoice is by the signatory.\"\n",
       "        }\n",
       "      },\n",
       "      \"required\": [\n",
       "        \"Signatory\",\n",
       "        \"IsSigned\"\n",
       "      ],\n",
       "      \"additionalProperties\": false\n",
       "    }\n",
       "  }\n",
       "}"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(typeof(Invoice).ModelJsonSchema());"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize the outputs\n",
    "\n",
    "To provide context for the execution of the code, the following code blocks visualize the outputs of the data extraction process.\n",
    "\n",
    "This includes:\n",
    "\n",
    "- The Markdown representation of the document structure as determined by Azure AI Document Intelligence.\n",
    "- The accuracy of the structured data extraction comparing the expected output with the output generated by Azure OpenAI's GPT-4o model.\n",
    "- The confidence score of the structured data extraction by comparing against the Azure AI Document Intelligence analysis.\n",
    "- The execution time of the end-to-end process.\n",
    "- The total number of tokens consumed by the GPT-4o model.\n",
    "\n",
    "### Understanding Accuracy vs Confidence\n",
    "\n",
    "When using AI to extract structured data, both confidence and accuracy are essential for different but complementary reasons.\n",
    "\n",
    "- **Accuracy** measures how close the AI model's output is to a ground truth or expected output. It reflects how well the model's predictions align with reality.\n",
    "  - Accuracy ensures consistency in the extraction process, which is crucial for downstream tasks using the data.\n",
    "- **Confidence** represents the AI model's internal assessment of how certain it is about its predictions.\n",
    "  - Confidence indicates that the model is certain about its predictions, which can be a useful indicator for human reviewers to step in for manual verification.\n",
    "\n",
    "High accuracy and high confidence are ideal, but in practice, there is often a trade-off between the two. While accuracy cannot always be self-assessed, confidence scores can and should be used to prioritize manual verification of low-confidence predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "NEXGEN\n",
       "\n",
       "Innovation drives progress\n",
       "\n",
       "\n",
       "# INVOICE\n",
       "\n",
       "\n",
       "<table>\n",
       "<tr>\n",
       "<th>Customer:</th>\n",
       "<th colspan=\"2\">Sharp Consulting</th>\n",
       "<th colspan=\"2\">Address:</th>\n",
       "<th colspan=\"3\">73 Regal Way, Leeds, LS1 5AB, UK</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Delv. Date:</td>\n",
       "<td>5/16/2024</td>\n",
       "<td colspan=\"2\">Invoice Number:</td>\n",
       "<td colspan=\"2\">3847193</td>\n",
       "<td>Purchase Order:</td>\n",
       "<td>15931</td>\n",
       "</tr>\n",
       "</table>\n",
       "\n",
       "\n",
       "<table>\n",
       "<tr>\n",
       "<th>Item Code</th>\n",
       "<th>Item Desc.</th>\n",
       "<th>Unit Price</th>\n",
       "<th>Quantity</th>\n",
       "<th>Total</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>MA197</td>\n",
       "<td>STRETCHWRAP ROLL</td>\n",
       "<td>16.62</td>\n",
       "<td>5</td>\n",
       "<td>83.10</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>ST4086</td>\n",
       "<td>BALLPOINT PEN MED.</td>\n",
       "<td>2.49</td>\n",
       "<td>10</td>\n",
       "<td>24.90</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>JF9912413BF</td>\n",
       "<td>BUBBLE FILM ROLL CL.</td>\n",
       "<td>15.46</td>\n",
       "<td>12</td>\n",
       "<td>185.52</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "</table>\n",
       "\n",
       "\n",
       "NOTES\n",
       "\n",
       "\n",
       "<table>\n",
       "<tr>\n",
       "<th>Total Pcs .:</th>\n",
       "<th>27</th>\n",
       "<th>Total Price:</th>\n",
       "<th>293.52</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td colspan=\"4\">Payable on or before 5/24/2024</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Cust. Sig.</td>\n",
       "<td>Sfr</td>\n",
       "<td>Drivr. Sig.</td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Cust. Name:</td>\n",
       "<td>Sarah H</td>\n",
       "<td>Drivr. Name:</td>\n",
       "<td>James T</td>\n",
       "</tr>\n",
       "</table>\n",
       "\n",
       "\n",
       "<!-- PageBreak -->\n",
       "\n",
       "\n",
       "<table>\n",
       "<tr>\n",
       "<td colspan=\"6\">NEXGEN Innovation drives progress</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td colspan=\"5\">RETURNS</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Customer:</td>\n",
       "<td>Sharp Consulting</td>\n",
       "<td colspan=\"2\">Address:</td>\n",
       "<td colspan=\"2\">73 Regal Way, Leeds, LS1 5AB, UK</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Coll. Date:</td>\n",
       "<td>5/16/2024</td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"2\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Item Code</td>\n",
       "<td>Item Desc.</td>\n",
       "<td>Pcs.</td>\n",
       "<td colspan=\"3\">Reason</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>MA145</td>\n",
       "<td>POSTAL TUBE BROWN</td>\n",
       "<td>1</td>\n",
       "<td colspan=\"3\">This item was provided in previous order</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>JF7902</td>\n",
       "<td>MAILBOX 25PK</td>\n",
       "<td>1 7</td>\n",
       "<td colspan=\"3\">as replacement</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"3\">Not required</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"3\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td colspan=\"6\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>NOTES</td>\n",
       "<td></td>\n",
       "<td colspan=\"4\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"4\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td colspan=\"5\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td colspan=\"5\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td colspan=\"2\"></td>\n",
       "<td></td>\n",
       "<td colspan=\"3\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"4\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Total Pcs .:</td>\n",
       "<td>2</td>\n",
       "<td></td>\n",
       "<td colspan=\"3\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td colspan=\"6\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Cust. Sig.</td>\n",
       "<td>8</td>\n",
       "<td></td>\n",
       "<td colspan=\"2\">Drivr. Sig.</td>\n",
       "<td>&amp;</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Cust. Name:</td>\n",
       "<td>Sarah H</td>\n",
       "<td></td>\n",
       "<td colspan=\"2\">Drivr. Name:</td>\n",
       "<td>James T</td>\n",
       "</tr>\n",
       "</table>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "// Displays the output of the Azure AI Document Intelligence pre-built layout analysis in Markdown format.\n",
    "display(markdown);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Gets the parsed Invoice object from the completion response.\n",
    "var invoice = completion.Parsed;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Determines the accuracy of the extracted data against the expected values.\n",
    "var accuracy = invoiceEvaluator.Evaluate(expected, invoice);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Determines the confidence of the extracted data using the OpenAI response.\n",
    "var diConfidence = DocumentIntelligenceConfidence<Invoice>.EvaluateConfidence(invoice, result);\n",
    "var oaiConfidence = OpenAIConfidence<Invoice>.EvaluateConfidence(invoice, completion.Origin);\n",
    "\n",
    "var confidence = ConfidenceHelpers.MergeConfidenceValues(diConfidence, oaiConfidence);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Gets the total execution time of the data extraction process.\n",
    "var totalElapsed = diSw.Elapsed + oaiSw.Elapsed;\n",
    "\n",
    "// Gets the prompt tokens and completion tokens from the completion response.\n",
    "var promptTokens = completion.Usage.InputTokenCount;\n",
    "var completionTokens = completion.Usage.OutputTokenCount;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Save the output of the data extraction result\n",
    "var extractionResult = new DataProcessingResult<Invoice>(\n",
    "    invoice,\n",
    "    accuracy,\n",
    "    confidence,\n",
    "    promptTokens,\n",
    "    completionTokens,\n",
    "    totalElapsed\n",
    ");\n",
    "\n",
    "var extractionResultJson = JsonSerializer.Serialize(extractionResult, new JsonSerializerOptions { WriteIndented = true });\n",
    "var extractionResultFPath = Path.Combine(samplePath, $\"{sampleName}.{pdfFName}.json\");\n",
    "\n",
    "await File.WriteAllTextAsync(extractionResultFPath, extractionResultJson);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<details open=\"open\" class=\"dni-treeview\"><summary><span class=\"dni-code-hint\"><code>{ Accuracy = 83.78%, Confidence = 98.58%, ExecutionTime = 40.43 seconds, DocumentIntelligenceExecutionTime = 8.14 seconds, OpenAIExecutionTime = 32.29 seconds, PromptTokens = 2492, CompletionTokens = 439 }</code></span></summary><div><table><thead><tr></tr></thead><tbody><tr><td>Accuracy</td><td><div class=\"dni-plaintext\"><pre>83.78%</pre></div></td></tr><tr><td>Confidence</td><td><div class=\"dni-plaintext\"><pre>98.58%</pre></div></td></tr><tr><td>ExecutionTime</td><td><div class=\"dni-plaintext\"><pre>40.43 seconds</pre></div></td></tr><tr><td>DocumentIntelligenceExecutionTime</td><td><div class=\"dni-plaintext\"><pre>8.14 seconds</pre></div></td></tr><tr><td>OpenAIExecutionTime</td><td><div class=\"dni-plaintext\"><pre>32.29 seconds</pre></div></td></tr><tr><td>PromptTokens</td><td><div class=\"dni-plaintext\"><pre>2492</pre></div></td></tr><tr><td>CompletionTokens</td><td><div class=\"dni-plaintext\"><pre>439</pre></div></td></tr></tbody></table></div></details><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>CustomerName</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999759063428685</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>Sharp Consulting</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>CustomerAddress</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>Street</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999778334235668</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>73 Regal Way</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>City</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999950978545089</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>Leeds</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>State</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>PostalCode</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999111569183523</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>LS1 5AB</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Country</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9980281084252315</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>UK</pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>CustomerTaxId</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>ShippingAddress</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>PurchaseOrder</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999877668922372</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>15931</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>InvoiceId</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999942833006115</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>3847193</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>InvoiceDate</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9994033316986145</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2024-05-16</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>PayableBy</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999921575162415</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2024-05-24</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>VendorName</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9952432423087636</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>NEXGEN</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>VendorAddress</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>VendorTaxId</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>RemittanceAddress</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Subtotal</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>TotalDiscount</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>TotalTax</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>InvoiceTotal</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9998800310164444</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>293.52</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>PaymentTerms</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Items</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.list-1?view=net-7.0\">System.Collections.Generic.List&lt;System.Object&gt;</a></span></td><td><table><thead><tr><th><i>index</i></th><th>value</th></tr></thead><tbody><tr><td>0</td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>ProductCode</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999931310059668</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>MA197</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>ProductDescription</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9995927046732505</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>STRETCHWRAP ROLL</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Quantity</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999400871469832</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>5</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Tax</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>TaxRate</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>UnitPrice</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.999982482274805</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>16.62</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Total</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9585625221997348</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>83.1</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Reason</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.8807905238327978</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td>1</td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>ProductCode</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.999995157456374</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>ST4086</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>ProductDescription</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9996669872330908</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>BALLPOINT PEN MED.</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Quantity</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999770387510999</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>10</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Tax</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>TaxRate</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>UnitPrice</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999957137375154</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2.49</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Total</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9901247866204952</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>24.9</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Reason</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9706747788631018</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td>2</td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>ProductCode</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999964448503291</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>JF9912413BF</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>ProductDescription</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999755421177651</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>BUBBLE FILM ROLL CL.</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Quantity</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999731051281417</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>12</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Tax</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>TaxRate</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>UnitPrice</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999968263007454</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>15.46</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Total</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999913429644429</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>185.52</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Reason</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre></pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>TotalItemQuantity</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999691715224762</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>27</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>ItemsCustomerSignature</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>Signatory</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9969943515811458</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>Sarah H</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>IsSigned</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>True</pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td colspan=\"3\"><i>(6 more)</i></td></tr></tbody></table><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "// Display the outputs of the extraction process.\n",
    "var output = new\n",
    "{\n",
    "    Accuracy = $\"{float.Parse(accuracy[\"overall\"].ToString()) * 100:0.00}%\",\n",
    "    Confidence = $\"{float.Parse(confidence[\"_overall\"].ToString()) * 100:0.00}%\",\n",
    "    ExecutionTime = $\"{totalElapsed.TotalSeconds:0.00} seconds\",\n",
    "    DocumentIntelligenceExecutionTime = $\"{diSw.Elapsed.TotalSeconds:0.00} seconds\",\n",
    "    OpenAIExecutionTime = $\"{oaiSw.Elapsed.TotalSeconds:0.00} seconds\",\n",
    "    PromptTokens = promptTokens,\n",
    "    CompletionTokens = completionTokens,\n",
    "};\n",
    "\n",
    "display(output);\n",
    "display(confidence);"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".NET (C#)",
   "language": "C#",
   "name": ".net-csharp"
  },
  "language_info": {
   "name": "python"
  },
  "polyglot_notebook": {
   "kernelInfo": {
    "defaultKernelName": "csharp",
    "items": [
     {
      "aliases": [],
      "name": "csharp"
     }
    ]
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}