examples/legal_hallucination_detection.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/logikon-ai/logikon/blob/main/examples/legal_hallucination_detection.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "13hfBALnYtHL"
      },
      "source": [
        "# *Build-Your-Own-Metrics #1*:<br>Fact Checking & Hallucination Detection for Legal Explanations\n",
        "\n",
        "**Logikon `/\\/`** – *Analyzing and Evaluating LLM Reasoning Traces*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TUzKTbJtYtHM"
      },
      "source": [
        "#### Overview\n",
        "\n",
        "This notebook illustrates how to build your own evaluations on top of metrics and artifacts created with **Logikon `/\\/`**. More specifically, we'll use an argument map to factcheck an explanation produced by a Legal AI agent against a given reference corpus, which potentially allows us to detect so-called hallucinations. As this demo serves illustrative purposes, we're keeping the implementation of the Legal AI agent and the fact-checking pipeline as simple as possible.  \n",
        "\n",
        "We're using 🤗's lightweight `zephyr-7b-beta` as **logic expert model** [below](#setup-b) for its speed and small footprint. It tends to produce argumentative analyses that are creative and eloquent, rather than rigorous and exact -- and may hence not be the optimal expert model for your use cases."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "apPN-0GVYtHN"
      },
      "source": [
        "> ⚠️ **Important notice.**\n",
        ">\n",
        "> **Logikon `/\\/`** is in early development stage:\n",
        "> * the package is subject to change at any time;\n",
        "> * results can vary due to changes in methods, pipelines, or underlying models;\n",
        "> * current speed and quality of evaluation results are not representative of future product.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PBHAr94lFgCD"
      },
      "source": [
        "#### Table of contents\n",
        "\n",
        "* **[Part I – Legal AI generates explanations (reasoning)](#part-a)**\n",
        "* **[Part II – Analysing legal reasoning with Logikon `/\\/`](#part-b)**\n",
        "    * [Set-up](#setup-b)\n",
        "    * [Select metrics and artifacts](#config_logikon)\n",
        "    * [Run analysis](#run)\n",
        "    * [Inspect results](#inspect)\n",
        "* **[Part III – Fact-checking the Legal AI with Logikon `/\\/` argument maps](#part-c)**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Part I – Legal AI generates explanations (reasoning)<a name=\"part-a\"></a>"
      ],
      "metadata": {
        "id": "yI_f8NboEA3a"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%capture\n",
        "%pip install huggingface_hub"
      ],
      "metadata": {
        "id": "j_r1Fj2gFLCA"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "%%capture\n",
        "# Optional: read environment variables from uploaded .env file, e.g. HUGGINGFACEHUB_API_TOKEN\n",
        "%pip install python-dotenv\n",
        "%load_ext dotenv\n",
        "%dotenv .env"
      ],
      "metadata": {
        "id": "0uk3_5eZJqPw"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# LLM serving as legal expert\n",
        "import os\n",
        "from huggingface_hub import InferenceClient\n",
        "\n",
        "legal_ai = InferenceClient(\n",
        "    \"mistralai/Mistral-7B-Instruct-v0.1\",\n",
        "    token=os.environ.get(\"HUGGINGFACEHUB_API_TOKEN\"),\n",
        ")\n",
        "legal_gen_kwargs = {\n",
        "    \"max_new_tokens\": 640,\n",
        "    \"temperature\":.4,\n",
        "    \"do_sample\": True,\n",
        "    \"seed\": 42\n",
        "}"
      ],
      "metadata": {
        "id": "RkQgDxn3E5dG"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "prompt= (\n",
        "    '<s>[INST] '\n",
        "    'In the case BOWMAN v. MONSANTO the Supreme Court ruled that \"patent exhaustion '\n",
        "    'does not permit a farmer to reproduce patented seeds through planting and '\n",
        "    'harvesting without the patent holder’s permission.\" Can you lay out the reasons '\n",
        "    'that led the Supreme Court to reject Bowman\\'s defence?'\n",
        "    '[/INST]\\n'\n",
        ")"
      ],
      "metadata": {
        "id": "d6_s0BarEPoC"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "completion = legal_ai.text_generation(prompt, **legal_gen_kwargs)\n",
        "\n",
        "from IPython.display import display, Markdown\n",
        "display(Markdown(completion))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 323
        },
        "id": "rleM-YuxFeAb",
        "outputId": "e2e8f860-bb95-4b65-e0b5-36b3a1b07f77"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "\nThe Supreme Court case of Bowman v. Monsanto involved a farmer named Vernon Bowman who purchased soybean seeds from a grain elevator, which had obtained the seeds from a farmer who had purchased them from the patent holder, Monsanto. Bowman then planted the seeds on his own farm and harvested the resulting crops.\n\nThe court ruled in favor of Monsanto, stating that patent exhaustion does not permit a farmer to reproduce patented seeds through planting and harvesting without the patent holder's permission. The reasons for this decision were as follows:\n\n1. The Supreme Court held that the doctrine of patent exhaustion applies only to the first sale of a patented product, and not to subsequent sales or uses of that product. This means that once a patented product has been sold, the patent holder no longer has the exclusive right to control its use or distribution.\n2. In this case, the seeds that Bowman purchased from the grain elevator had already been sold by the patent holder, and the grain elevator had obtained them from a farmer who had purchased them from the patent holder. This means that the seeds were no longer within the scope of the original sale and were not covered by the doctrine of patent exhaustion.\n3. The court also held that the farmer's actions of planting and harvesting the seeds constituted an infringement of the patent holder's exclusive right to reproduce the patented seeds. This is because the seeds were a key component of the patented soybean crop, and the farmer's actions allowed him to produce and sell crops that were identical to those produced by the patent holder.\n\nOverall, the Supreme Court's decision in Bowman v. Monsanto rejected the farmer's defense of patent exhaustion and upheld the patent holder's exclusive right to control the use and distribution of their patented seeds."
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fMyE0lBQYtHN"
      },
      "source": [
        "## Part II – Analysing legal reasoning with **Logikon&nbsp;`/\\/`** <a name=\"part-b\"></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aCSYMwRi0NyR"
      },
      "source": [
        "### Setting up notebook environment to run scoring script<a name=\"setup-b\"></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "VEz318IJCaw7"
      },
      "outputs": [],
      "source": [
        "# Please make sure that the following files from the `examples` folder\n",
        "# in the `logikon-ai/logikon` GitHub repo are available.\n",
        "!test -f \"requirements.txt\" || echo \"Missing requirements.txt\"\n",
        "!test -f \"setup.sh\" || echo \"Missing setup.sh\"\n",
        "!test -f \"scoring.py\" || echo \"Missing scoring.py\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "uDIpY9XuCwkm"
      },
      "outputs": [],
      "source": [
        "# Configure the environment\n",
        "import os\n",
        "\n",
        "os.environ[\"LGK_FRAMEWORK\"] = \"llamacpp\"\n",
        "os.environ[\"LGK_DEVICE\"] = \"gpu\"\n",
        "os.environ[\"LGK_HUBREPO\"] = \"TheBloke/zephyr-7B-beta-GGUF\"\n",
        "os.environ[\"LGK_MODELWEIGHTS\"] = \"zephyr-7b-beta.Q6_K.gguf\"\n",
        "os.environ[\"LGK_TOKENIZER\"] = \"HuggingFaceH4/zephyr-7b-beta\"\n",
        "os.environ[\"LGK_RELEASE\"] = \"main\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "epMW5Zx6W9nr",
        "outputId": "9eed998c-d9cc-4c46-f94d-5a5ff420fff8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Wed Nov 22 13:50:26 2023       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   49C    P8     9W /  70W |      0MiB / 15360MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "# Check gpu available\n",
        "! [ \"$LGK_DEVICE\" = \"gpu\" ] && nvidia-smi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "tSSiej4pY8-i"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "!sh setup.sh"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "iFteYtUPJvDI",
        "outputId": "e3b22f46-c6e9-4ae8-ae86-fa86455fe0fd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'/content/models/zephyr-7b-beta.Q6_K.gguf'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "# Path to model weights as env variable\n",
        "import os\n",
        "\n",
        "if os.environ.get(\"LGK_MODELWEIGHTS\"):\n",
        "    abspath = os.path.join(\n",
        "        os.path.abspath(\"./models\"),\n",
        "        os.environ.get(\"LGK_MODELWEIGHTS\")\n",
        "    )\n",
        "else:\n",
        "    abspath = \"\"\n",
        "os.environ[\"LGK_ABSPATH_MODELWEIGHTS\"] = abspath\n",
        "os.environ[\"LGK_ABSPATH_MODELWEIGHTS\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "BN9HH97iL9OF"
      },
      "outputs": [],
      "source": [
        "# Check whether model weights file is available\n",
        "!test -f \"$LGK_ABSPATH_MODELWEIGHTS\" || echo \"Cannot find model weights\""
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Post-processing prompt\n",
        "# Inserting hints that point out which central claims to use in logical analysis\n",
        "prompt_with_hint = prompt.replace(\"[INST]\",\"\").replace(\"[/INST]\",\"\").replace(\"<s>\",\"\").strip()\n",
        "prompt_with_hint += \" The central claim is simply: Bowman's defence should be rejected.\""
      ],
      "metadata": {
        "id": "C07p0c96O8Dt"
      },
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "0gGYxzS9aoyI",
        "outputId": "f50a55bc-af46-4e4f-f1ef-5b80ec7601c1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{\"prompt\":\"In the case BOWMAN v. MONSANTO the Supreme Court ruled that \\\"patent exhaustion does not permit a farmer to reproduce patented seeds through planting and harvesting without the patent holder\\u2019s permission.\\\" Can you lay out the reasons that led the Supreme Court to reject Bowman's defence? The central claim is simply: Bowman's defence should be rejected.\",\"completion\":\"\\nThe Supreme Court case of Bowman v. Monsanto involved a farmer named Vernon Bowman who purchased soybean seeds from a grain elevator, which had obtained the seeds from a farmer who had purchased them from the patent holder, Monsanto. Bowman then planted the seeds on his own farm and harvested the resulting crops.\\n\\nThe court ruled in favor of Monsanto, stating that patent exhaustion does not permit a farmer to reproduce patented seeds through planting and harvesting without the patent holder's permission. The reasons for this decision were as follows:\\n\\n1. The Supreme Court held that the doctrine of patent exhaustion applies only to the first sale of a patented product, and not to subsequent sales or uses of that product. This means that once a patented product has been sold, the patent holder no longer has the exclusive right to control its use or distribution.\\n2. In this case, the seeds that Bowman purchased from the grain elevator had already been sold by the patent holder, and the grain elevator had obtained them from a farmer who had purchased them from the patent holder. This means that the seeds were no longer within the scope of the original sale and were not covered by the doctrine of patent exhaustion.\\n3. The court also held that the farmer's actions of planting and harvesting the seeds constituted an infringement of the patent holder's exclusive right to reproduce the patented seeds. This is because the seeds were a key component of the patented soybean crop, and the farmer's actions allowed him to produce and sell crops that were identical to those produced by the patent holder.\\n\\nOverall, the Supreme Court's decision in Bowman v. Monsanto rejected the farmer's defense of patent exhaustion and upheld the patent holder's exclusive right to control the use and distribution of their patented seeds.\"}\n"
          ]
        }
      ],
      "source": [
        "# Save reasoning traces to file\n",
        "import pandas as pd\n",
        "df_traces = pd.DataFrame([{\"prompt\": prompt_with_hint, \"completion\": completion}])\n",
        "df_traces.to_json(\"reasoning_traces.jsonl\", orient=\"records\", lines=True)\n",
        "# Check data file available\n",
        "!head -2 \"reasoning_traces.jsonl\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nSrgZcyMAdwP"
      },
      "source": [
        "### Select metrics and artifacts to calculate, and configure scoring methods <a name=\"config_logikon\"></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sIGXP7YieNQg",
        "outputId": "21d708d4-669f-4267-d243-f9122acb8d60"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "artifacts:\n",
            "- fuzzy_argmap_nx\n",
            "- svg_argmap\n",
            "global_kwargs:\n",
            "  expert_model: /content/models/zephyr-7b-beta.Q6_K.gguf\n",
            "  expert_model_kwargs:\n",
            "    n_ctx: 3072\n",
            "    n_gpu_layers: -1\n",
            "    prompt_template:\n",
            "      ass_end: ''\n",
            "...\n"
          ]
        }
      ],
      "source": [
        "# Write config als yaml file\n",
        "\n",
        "config_dict = {\n",
        "    # all metrics\n",
        "    \"metrics\": [\"argmap_size\", \"n_root_nodes\", \"mean_reason_strength\"],\n",
        "    # artifacts to generate\n",
        "    \"artifacts\": [\"fuzzy_argmap_nx\", \"svg_argmap\"],\n",
        "    \"global_kwargs\": {\n",
        "        \"expert_model\": os.environ[\"LGK_ABSPATH_MODELWEIGHTS\"],\n",
        "        \"expert_model_kwargs\": {\n",
        "            \"n_ctx\": 3072,\n",
        "            # comment out line below if no gpu\n",
        "            \"n_gpu_layers\": -1,\n",
        "            \"tokenizer\": os.environ[\"LGK_TOKENIZER\"],\n",
        "            # custom prompt template, make sure this fits the expert_model\n",
        "            \"prompt_template\": {\n",
        "                \"sys_start\": \"\\n<|system|>\\n\",\n",
        "                \"sys_end\": \"\",\n",
        "                \"user_start\": \"\\n<|user|>\\n\",\n",
        "                \"user_end\": \"\",\n",
        "                \"ass_start\": \"\\n<|assistant|>\\n\",\n",
        "                \"ass_end\": \"\",\n",
        "            },\n",
        "        },\n",
        "        \"generation_kwargs\": {\"max_len\": 3072},\n",
        "        \"llm_framework\": \"llama.cpp\",\n",
        "    },\n",
        "}\n",
        "\n",
        "import yaml\n",
        "with open('config.yml', 'w') as outfile:\n",
        "    yaml.dump(config_dict, outfile, default_flow_style=False)\n",
        "\n",
        "# Check config file available\n",
        "!head -10 \"config.yml\" && echo \"...\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pwx5YGLtA5wW"
      },
      "source": [
        "### Run argumentation analysis <a name=\"run\"></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BL4ADqpSf_WB",
        "outputId": "f2505ec7-f079-40ee-808f-4b7e71650dc8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0it [00:00, ?it/s]/usr/bin/dot\n",
            "The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.\n",
            "\n",
            "0it [00:00, ?it/s]\n",
            "\n",
            "tokenizer_config.json: 100% 1.43k/1.43k [00:00<00:00, 6.09MB/s]\n",
            "\n",
            "tokenizer.model: 100% 493k/493k [00:00<00:00, 38.4MB/s]\n",
            "\n",
            "tokenizer.json:   0% 0.00/1.80M [00:00<?, ?B/s]\u001b[A\n",
            "tokenizer.json: 100% 1.80M/1.80M [00:00<00:00, 1.83MB/s]\n",
            "\n",
            "added_tokens.json: 100% 42.0/42.0 [00:00<00:00, 196kB/s]\n",
            "\n",
            "special_tokens_map.json: 100% 168/168 [00:00<00:00, 890kB/s]\n",
            "[Loading llama.cpp model from llama.cpp:/content/models/zephyr-7b-beta.Q6_K.gguf  with  {'n_ctx': 3072, 'n_gpu_layers': -1} ]\n",
            "2023-11-22 13:55:41.235427: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
            "2023-11-22 13:55:41.235493: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
            "2023-11-22 13:55:41.235532: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
            "2023-11-22 13:55:43.213327: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
            "\n",
            "Reason-root edges:   0% 0/2 [00:00<?, ?it/s]\u001b[A\n",
            "Reason-root edges:  50% 1/2 [00:10<00:10, 10.41s/it]\u001b[A\n",
            "Reason-root edges: 100% 2/2 [00:17<00:00,  8.91s/it]\n",
            "\n",
            "Reason-reason edges:   0% 0/8 [00:00<?, ?it/s]\u001b[A\n",
            "Reason-reason edges:  25% 2/8 [00:14<00:42,  7.05s/it]\u001b[A\n",
            "Reason-reason edges:  38% 3/8 [00:28<00:49,  9.96s/it]\u001b[A\n",
            "Reason-reason edges:  50% 4/8 [00:41<00:45, 11.27s/it]\u001b[A\n",
            "Reason-reason edges:  75% 6/8 [00:53<00:17,  8.64s/it]\u001b[A\n",
            "Reason-reason edges:  88% 7/8 [01:08<00:10, 10.34s/it]\u001b[A\n",
            "Reason-reason edges: 100% 8/8 [01:22<00:00, 10.30s/it]\n",
            "1it [04:18, 258.24s/it]\n"
          ]
        }
      ],
      "source": [
        "# 🫖 This will take some time ... you may monitor `scoring.log`\n",
        "!python scoring.py \\\n",
        "  --config-file config.yml \\\n",
        "  --reasoning-traces-file reasoning_traces.jsonl"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7uaXFSRjA_Wu"
      },
      "source": [
        "### Inspect argumentation analysis <a name=\"inspect\"></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "cttOhhYfpyP6"
      },
      "outputs": [],
      "source": [
        "# Read analysis results from file\n",
        "df_scores = pd.read_json(\"outputs/reasoning_traces-scores.jsonl\", lines=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 174
        },
        "id": "JCC66EW08MAx",
        "outputId": "ddfd3229-8107-43bb-d51c-0717bc54b8f4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "#example                     0\n",
              "metric                        \n",
              "argmap_size           8.000000\n",
              "n_root_nodes          2.000000\n",
              "mean_reason_strength  0.668001"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-58dc2817-3dc7-427a-a5a6-00ff0f7feb30\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>#example</th>\n",
              "      <th>0</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>metric</th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>argmap_size</th>\n",
              "      <td>8.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>n_root_nodes</th>\n",
              "      <td>2.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean_reason_strength</th>\n",
              "      <td>0.668001</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-58dc2817-3dc7-427a-a5a6-00ff0f7feb30')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-58dc2817-3dc7-427a-a5a6-00ff0f7feb30 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-58dc2817-3dc7-427a-a5a6-00ff0f7feb30');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-27420385-0f91-419a-9aeb-d24599cad1fe\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-27420385-0f91-419a-9aeb-d24599cad1fe')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-27420385-0f91-419a-9aeb-d24599cad1fe button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "# Display metric scores\n",
        "df_scores[config_dict.get(\"metrics\",[])].T.rename_axis(\"#example\", axis=\"columns\").rename_axis(\"metric\", axis=\"rows\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "saeFhPUzIN_k"
      },
      "outputs": [],
      "source": [
        "# Utility method for rendering artifacts\n",
        "from pprint import pformat\n",
        "\n",
        "import networkx as nx\n",
        "\n",
        "from IPython.display import SVG, HTML, Markdown\n",
        "\n",
        "from logikon.schemas.pros_cons import ProsConsList\n",
        "from logikon.schemas.argument_mapping import FuzzyArgMap\n",
        "\n",
        "def render_artifact(artifact_id, data):\n",
        "  \"\"\"helper function for rendering artifacts\"\"\"\n",
        "\n",
        "  if \"nx\" in artifact_id:\n",
        "    digraph = nx.node_link_graph(data)\n",
        "    return Markdown(str(digraph))\n",
        "\n",
        "  if \"svg\" in artifact_id:\n",
        "    #style = \"<style>svg{width:100% !important;height:100% !important;</style>\"\n",
        "    style = \"<style>svg{width:100% !important;</style>\"\n",
        "    return HTML(style), SVG(data=data)\n",
        "\n",
        "  if \"html\" in artifact_id:\n",
        "    return HTML(data)\n",
        "\n",
        "  if artifact_id == \"proscons\":\n",
        "    proscons = ProsConsList(**data)\n",
        "    return Markdown(f\"`{pformat(proscons)}`\")\n",
        "\n",
        "  if artifact_id == \"relevance_network\":\n",
        "    relnet = FuzzyArgMap(**data)\n",
        "    return Markdown(f\"`{pformat(relnet)[:800]} ...`\")\n",
        "\n",
        "  text = str(data)\n",
        "  text = text[:200] + \" ...\" if len(text)>200 else text\n",
        "  return Markdown(f\"`{text}`\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cg_f1Tt10NyY"
      },
      "source": [
        "> ⛔️ **Limitation**\n",
        ">\n",
        "> Note that the `zephyr-7b-beta` expert model struggles to correctly relate individual reasons to each other -- a task much better accomplished by larger LLMs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "tFo-h4Ba6zfO",
        "outputId": "f8559ee7-d43c-4f06-d48e-d2871f3031b2"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "## fuzzy_argmap_nx"
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "DiGraph with 8 nodes and 6 edges"
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "## svg_argmap"
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<style>svg{width:100% !important;</style>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.SVG object>"
            ],
            "image/svg+xml": "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"516pt\" height=\"756pt\" viewBox=\"0.00 0.00 515.50 756.00\">\n<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 752)\">\n<title>logikon informal argument map</title>\n<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-752 511.5,-752 511.5,4 -4,4\"/>\n<!-- c69dc58d&#45;3668&#45;4519&#45;ad38&#45;f3e99cb5ac24 -->\n<g id=\"node1\" class=\"node\">\n<title>c69dc58d-3668-4519-ad38-f3e99cb5ac24</title>\n<g id=\"a_node1\"><a xlink:title=\"Farmers should be allowed to&lt;BR/&gt;reproduce patented seeds&lt;BR/&gt;through planting and&lt;BR/&gt;harvesting.\">\n<path fill=\"white\" stroke=\"transparent\" d=\"M124,-659C124,-659 270,-659 270,-659 276,-659 282,-665 282,-671 282,-671 282,-730 282,-730 282,-736 276,-742 270,-742 270,-742 124,-742 124,-742 118,-742 112,-736 112,-730 112,-730 112,-671 112,-671 112,-665 118,-659 124,-659\"/>\n<text text-anchor=\"start\" x=\"127\" y=\"-726.9\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">Allow seed reproduction</text>\n<text text-anchor=\"start\" x=\"118\" y=\"-706.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">Farmers should be allowed to</text>\n<text text-anchor=\"start\" x=\"127.5\" y=\"-693.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">reproduce patented seeds</text>\n<text text-anchor=\"start\" x=\"142\" y=\"-680.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">through planting and</text>\n<text text-anchor=\"start\" x=\"167.5\" y=\"-667.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">harvesting.</text>\n<path fill=\"none\" stroke=\"lightblue\" stroke-width=\"4\" d=\"M124,-659C124,-659 270,-659 270,-659 276,-659 282,-665 282,-671 282,-671 282,-730 282,-730 282,-736 276,-742 270,-742 270,-742 124,-742 124,-742 118,-742 112,-736 112,-730 112,-730 112,-671 112,-671 112,-665 118,-659 124,-659\"/>\n</a>\n</g>\n</g>\n<!-- c22e0fb7&#45;2b20&#45;40e8&#45;91c3&#45;a1047bca94ac -->\n<g id=\"node2\" class=\"node\">\n<title>c22e0fb7-2b20-40e8-91c3-a1047bca94ac</title>\n<g id=\"a_node2\"><a xlink:title=\"Farmers should not be allowed&lt;BR/&gt;to reproduce patented seeds&lt;BR/&gt;through planting and&lt;BR/&gt;harvesting.\">\n<path fill=\"white\" stroke=\"transparent\" d=\"M332.5,-659C332.5,-659 485.5,-659 485.5,-659 491.5,-659 497.5,-665 497.5,-671 497.5,-671 497.5,-730 497.5,-730 497.5,-736 491.5,-742 485.5,-742 485.5,-742 332.5,-742 332.5,-742 326.5,-742 320.5,-736 320.5,-730 320.5,-730 320.5,-671 320.5,-671 320.5,-665 326.5,-659 332.5,-659\"/>\n<text text-anchor=\"start\" x=\"343.5\" y=\"-726.9\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">Ban seed reproduction</text>\n<text text-anchor=\"start\" x=\"327\" y=\"-706.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">Farmers should not be allowed</text>\n<text text-anchor=\"start\" x=\"333.5\" y=\"-693.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">to reproduce patented seeds</text>\n<text text-anchor=\"start\" x=\"354.5\" y=\"-680.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">through planting and</text>\n<text text-anchor=\"start\" x=\"380\" y=\"-667.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">harvesting.</text>\n<path fill=\"none\" stroke=\"lightblue\" stroke-width=\"4\" d=\"M332.5,-659C332.5,-659 485.5,-659 485.5,-659 491.5,-659 497.5,-665 497.5,-671 497.5,-671 497.5,-730 497.5,-730 497.5,-736 491.5,-742 485.5,-742 485.5,-742 332.5,-742 332.5,-742 326.5,-742 320.5,-736 320.5,-730 320.5,-730 320.5,-671 320.5,-671 320.5,-665 326.5,-659 332.5,-659\"/>\n</a>\n</g>\n</g>\n<!-- 07a6de58&#45;11a5&#45;4cf9&#45;9f78&#45;901b2b65f590 -->\n<g id=\"node3\" class=\"node\">\n<title>07a6de58-11a5-4cf9-9f78-901b2b65f590</title>\n<g id=\"a_node3\"><a xlink:title=\"The Supreme Court held that&lt;BR/&gt;patent exhaustion applies only&lt;BR/&gt;to the first sale of a&lt;BR/&gt;patented product.\">\n<path fill=\"lightblue\" stroke=\"transparent\" d=\"M121.5,-528C121.5,-528 272.5,-528 272.5,-528 278.5,-528 284.5,-534 284.5,-540 284.5,-540 284.5,-599 284.5,-599 284.5,-605 278.5,-611 272.5,-611 272.5,-611 121.5,-611 121.5,-611 115.5,-611 109.5,-605 109.5,-599 109.5,-599 109.5,-540 109.5,-540 109.5,-534 115.5,-528 121.5,-528\"/>\n<text text-anchor=\"start\" x=\"146\" y=\"-595.9\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">First sale doctrine</text>\n<text text-anchor=\"start\" x=\"120.5\" y=\"-575.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">The Supreme Court held that</text>\n<text text-anchor=\"start\" x=\"116\" y=\"-562.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">patent exhaustion applies only</text>\n<text text-anchor=\"start\" x=\"147.5\" y=\"-549.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">to the first sale of a</text>\n<text text-anchor=\"start\" x=\"151\" y=\"-536.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">patented product.</text>\n<path fill=\"none\" stroke=\"lightblue\" stroke-width=\"4\" d=\"M121.5,-528C121.5,-528 272.5,-528 272.5,-528 278.5,-528 284.5,-534 284.5,-540 284.5,-540 284.5,-599 284.5,-599 284.5,-605 278.5,-611 272.5,-611 272.5,-611 121.5,-611 121.5,-611 115.5,-611 109.5,-605 109.5,-599 109.5,-599 109.5,-540 109.5,-540 109.5,-534 115.5,-528 121.5,-528\"/>\n</a>\n</g>\n</g>\n<!-- 07a6de58&#45;11a5&#45;4cf9&#45;9f78&#45;901b2b65f590&#45;&gt;c69dc58d&#45;3668&#45;4519&#45;ad38&#45;f3e99cb5ac24 -->\n<g id=\"edge1\" class=\"edge\">\n<title>07a6de58-11a5-4cf9-9f78-901b2b65f590-&gt;c69dc58d-3668-4519-ad38-f3e99cb5ac24</title>\n<path fill=\"none\" stroke=\"#a9a9a9\" stroke-width=\"2\" d=\"M197,-617.07C197,-625.48 197,-634.32 197,-642.97\"/>\n<polygon fill=\"#a9a9a9\" stroke=\"#a9a9a9\" stroke-width=\"2\" points=\"193.5,-643 197,-653 200.5,-643 193.5,-643\"/>\n</g>\n<!-- efccab7f&#45;fcdc&#45;4f2d&#45;909b&#45;dd31fa8d2cc2 -->\n<g id=\"node4\" class=\"node\">\n<title>efccab7f-fcdc-4f2d-909b-dd31fa8d2cc2</title>\n<g id=\"a_node4\"><a xlink:title=\"The Supreme Court held that&lt;BR/&gt;patent exhaustion does not&lt;BR/&gt;apply to subsequent sales or&lt;BR/&gt;uses.\">\n<path fill=\"lightblue\" stroke=\"transparent\" d=\"M22,-240C22,-240 164,-240 164,-240 170,-240 176,-246 176,-252 176,-252 176,-311 176,-311 176,-317 170,-323 164,-323 164,-323 22,-323 22,-323 16,-323 10,-317 10,-311 10,-311 10,-252 10,-252 10,-246 16,-240 22,-240\"/>\n<text text-anchor=\"start\" x=\"18.5\" y=\"-307.9\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">Subsequent sales or uses</text>\n<text text-anchor=\"start\" x=\"16\" y=\"-287.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">The Supreme Court held that</text>\n<text text-anchor=\"start\" x=\"21\" y=\"-274.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">patent exhaustion does not</text>\n<text text-anchor=\"start\" x=\"16.5\" y=\"-261.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">apply to subsequent sales or</text>\n<text text-anchor=\"start\" x=\"78.5\" y=\"-248.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">uses.</text>\n<path fill=\"none\" stroke=\"lightblue\" stroke-width=\"4\" d=\"M22,-240C22,-240 164,-240 164,-240 170,-240 176,-246 176,-252 176,-252 176,-311 176,-311 176,-317 170,-323 164,-323 164,-323 22,-323 22,-323 16,-323 10,-317 10,-311 10,-311 10,-252 10,-252 10,-246 16,-240 22,-240\"/>\n</a>\n</g>\n</g>\n<!-- 519e9306&#45;6d1a&#45;4a21&#45;b666&#45;faae319c63a4 -->\n<g id=\"node5\" class=\"node\">\n<title>519e9306-6d1a-4a21-b666-faae319c63a4</title>\n<g id=\"a_node5\"><a xlink:title=\"The Supreme Court held that&lt;BR/&gt;patent exhaustion does not&lt;BR/&gt;apply to the seeds purchased&lt;BR/&gt;by Bowman, as they had already&lt;BR/&gt;been sold once.\">\n<path fill=\"lightblue\" stroke=\"transparent\" d=\"M116,-384C116,-384 278,-384 278,-384 284,-384 290,-390 290,-396 290,-396 290,-468 290,-468 290,-474 284,-480 278,-480 278,-480 116,-480 116,-480 110,-480 104,-474 104,-468 104,-468 104,-396 104,-396 104,-390 110,-384 116,-384\"/>\n<text text-anchor=\"start\" x=\"111\" y=\"-465.4\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">Seeds purchased by Bowman</text>\n<text text-anchor=\"start\" x=\"120\" y=\"-445.4\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">The Supreme Court held that</text>\n<text text-anchor=\"start\" x=\"125\" y=\"-432.4\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">patent exhaustion does not</text>\n<text text-anchor=\"start\" x=\"118.5\" y=\"-419.4\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">apply to the seeds purchased</text>\n<text text-anchor=\"start\" x=\"110\" y=\"-406.4\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">by Bowman, as they had already</text>\n<text text-anchor=\"start\" x=\"154.5\" y=\"-393.4\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">been sold once.</text>\n<path fill=\"none\" stroke=\"lightblue\" stroke-width=\"4\" d=\"M116,-384C116,-384 278,-384 278,-384 284,-384 290,-390 290,-396 290,-396 290,-468 290,-468 290,-474 284,-480 278,-480 278,-480 116,-480 116,-480 110,-480 104,-474 104,-468 104,-468 104,-396 104,-396 104,-390 110,-384 116,-384\"/>\n</a>\n</g>\n</g>\n<!-- efccab7f&#45;fcdc&#45;4f2d&#45;909b&#45;dd31fa8d2cc2&#45;&gt;519e9306&#45;6d1a&#45;4a21&#45;b666&#45;faae319c63a4 -->\n<g id=\"edge2\" class=\"edge\">\n<title>efccab7f-fcdc-4f2d-909b-dd31fa8d2cc2-&gt;519e9306-6d1a-4a21-b666-faae319c63a4</title>\n<path fill=\"none\" stroke=\"#026502\" stroke-width=\"2\" d=\"M125.62,-329.08C134.6,-341.9 144.48,-356 153.95,-369.52\"/>\n<polygon fill=\"#026502\" stroke=\"#026502\" stroke-width=\"2\" points=\"151.17,-371.67 159.78,-377.85 156.91,-367.65 151.17,-371.67\"/>\n</g>\n<!-- 519e9306&#45;6d1a&#45;4a21&#45;b666&#45;faae319c63a4&#45;&gt;07a6de58&#45;11a5&#45;4cf9&#45;9f78&#45;901b2b65f590 -->\n<g id=\"edge3\" class=\"edge\">\n<title>519e9306-6d1a-4a21-b666-faae319c63a4-&gt;07a6de58-11a5-4cf9-9f78-901b2b65f590</title>\n<path fill=\"none\" stroke=\"#a5a7a5\" stroke-width=\"2\" d=\"M197,-486.16C197,-494.49 197,-503.13 197,-511.54\"/>\n<polygon fill=\"#a5a7a5\" stroke=\"#a5a7a5\" stroke-width=\"2\" points=\"193.5,-511.65 197,-521.65 200.5,-511.65 193.5,-511.65\"/>\n</g>\n<!-- e13ffc42&#45;0733&#45;495b&#45;a78d&#45;34bc8ff01d0e -->\n<g id=\"node6\" class=\"node\">\n<title>e13ffc42-0733-495b-a78d-34bc8ff01d0e</title>\n<g id=\"a_node6\"><a xlink:title=\"The patent holder has an&lt;BR/&gt;exclusive right to reproduce&lt;BR/&gt;the patented seeds.\">\n<path fill=\"lightblue\" stroke=\"transparent\" d=\"M233.5,-98C233.5,-98 368.5,-98 368.5,-98 374.5,-98 380.5,-104 380.5,-110 380.5,-110 380.5,-167 380.5,-167 380.5,-173 374.5,-179 368.5,-179 368.5,-179 233.5,-179 233.5,-179 227.5,-179 221.5,-173 221.5,-167 221.5,-167 221.5,-110 221.5,-110 221.5,-104 227.5,-98 233.5,-98\"/>\n<text text-anchor=\"start\" x=\"251.5\" y=\"-163.9\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">Exclusive right to</text>\n<text text-anchor=\"start\" x=\"271.5\" y=\"-151.9\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">reproduce</text>\n<text text-anchor=\"start\" x=\"235\" y=\"-132.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">The patent holder has an</text>\n<text text-anchor=\"start\" x=\"228\" y=\"-119.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">exclusive right to reproduce</text>\n<text text-anchor=\"start\" x=\"249\" y=\"-106.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">the patented seeds.</text>\n<path fill=\"none\" stroke=\"lightblue\" stroke-width=\"4\" d=\"M233.5,-98C233.5,-98 368.5,-98 368.5,-98 374.5,-98 380.5,-104 380.5,-110 380.5,-110 380.5,-167 380.5,-167 380.5,-173 374.5,-179 368.5,-179 368.5,-179 233.5,-179 233.5,-179 227.5,-179 221.5,-173 221.5,-167 221.5,-167 221.5,-110 221.5,-110 221.5,-104 227.5,-98 233.5,-98\"/>\n</a>\n</g>\n</g>\n<!-- f75e852e&#45;49c2&#45;4282&#45;bcbc&#45;d8a43780c8b4 -->\n<g id=\"node7\" class=\"node\">\n<title>f75e852e-49c2-4282-bcbc-d8a43780c8b4</title>\n<g id=\"a_node7\"><a xlink:title=\"The farmer's actions of&lt;BR/&gt;planting and harvesting the&lt;BR/&gt;seeds constituted an&lt;BR/&gt;infringement of the patent&lt;BR/&gt;holder's exclusive right to&lt;BR/&gt;reproduce the patented seeds.\">\n<path fill=\"lightblue\" stroke=\"transparent\" d=\"M226,-227C226,-227 376,-227 376,-227 382,-227 388,-233 388,-239 388,-239 388,-324 388,-324 388,-330 382,-336 376,-336 376,-336 226,-336 226,-336 220,-336 214,-330 214,-324 214,-324 214,-239 214,-239 214,-233 220,-227 226,-227\"/>\n<text text-anchor=\"start\" x=\"253\" y=\"-320.9\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">Farmer's actions</text>\n<text text-anchor=\"start\" x=\"240.5\" y=\"-300.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">The farmer's actions of</text>\n<text text-anchor=\"start\" x=\"228.5\" y=\"-287.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">planting and harvesting the</text>\n<text text-anchor=\"start\" x=\"246\" y=\"-274.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">seeds constituted an</text>\n<text text-anchor=\"start\" x=\"233.5\" y=\"-261.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">infringement of the patent</text>\n<text text-anchor=\"start\" x=\"233.5\" y=\"-248.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">holder's exclusive right to</text>\n<text text-anchor=\"start\" x=\"220\" y=\"-235.9\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">reproduce the patented seeds.</text>\n<path fill=\"none\" stroke=\"lightblue\" stroke-width=\"4\" d=\"M226,-227C226,-227 376,-227 376,-227 382,-227 388,-233 388,-239 388,-239 388,-324 388,-324 388,-330 382,-336 376,-336 376,-336 226,-336 226,-336 220,-336 214,-330 214,-324 214,-324 214,-239 214,-239 214,-233 220,-227 226,-227\"/>\n</a>\n</g>\n</g>\n<!-- e13ffc42&#45;0733&#45;495b&#45;a78d&#45;34bc8ff01d0e&#45;&gt;f75e852e&#45;49c2&#45;4282&#45;bcbc&#45;d8a43780c8b4 -->\n<g id=\"edge4\" class=\"edge\">\n<title>e13ffc42-0733-495b-a78d-34bc8ff01d0e-&gt;f75e852e-49c2-4282-bcbc-d8a43780c8b4</title>\n<path fill=\"none\" stroke=\"#006400\" stroke-width=\"2\" d=\"M301,-185.28C301,-193.43 301,-202.08 301,-210.72\"/>\n<polygon fill=\"#006400\" stroke=\"#006400\" stroke-width=\"2\" points=\"297.5,-210.83 301,-220.83 304.5,-210.83 297.5,-210.83\"/>\n</g>\n<!-- f75e852e&#45;49c2&#45;4282&#45;bcbc&#45;d8a43780c8b4&#45;&gt;519e9306&#45;6d1a&#45;4a21&#45;b666&#45;faae319c63a4 -->\n<g id=\"edge5\" class=\"edge\">\n<title>f75e852e-49c2-4282-bcbc-d8a43780c8b4-&gt;519e9306-6d1a-4a21-b666-faae319c63a4</title>\n<path fill=\"none\" stroke=\"#ff0000\" stroke-width=\"2\" d=\"M259.3,-342.04C252.98,-351.06 246.45,-360.39 240.11,-369.45\"/>\n<polygon fill=\"#ff0000\" stroke=\"#ff0000\" stroke-width=\"2\" points=\"237.21,-367.48 234.34,-377.68 242.95,-371.5 237.21,-367.48\"/>\n</g>\n<!-- 2947cead&#45;1a91&#45;4b32&#45;9ad3&#45;81a0836d6d7e -->\n<g id=\"node8\" class=\"node\">\n<title>2947cead-1a91-4b32-9ad3-81a0836d6d7e</title>\n<g id=\"a_node8\"><a xlink:title=\"The seeds were patented.\">\n<path fill=\"lightblue\" stroke=\"transparent\" d=\"M237.5,-6C237.5,-6 364.5,-6 364.5,-6 370.5,-6 376.5,-12 376.5,-18 376.5,-18 376.5,-38 376.5,-38 376.5,-44 370.5,-50 364.5,-50 364.5,-50 237.5,-50 237.5,-50 231.5,-50 225.5,-44 225.5,-38 225.5,-38 225.5,-18 225.5,-18 225.5,-12 231.5,-6 237.5,-6\"/>\n<text text-anchor=\"start\" x=\"256\" y=\"-35.4\" font-family=\"Arial, Helvetica, sans-serif\" font-weight=\"bold\" font-size=\"12.00\">Nature of seeds</text>\n<text text-anchor=\"start\" x=\"232\" y=\"-15.4\" font-family=\"Arial, Helvetica, sans-serif\" font-size=\"12.00\">The seeds were patented.</text>\n<path fill=\"none\" stroke=\"lightblue\" stroke-width=\"4\" d=\"M237.5,-6C237.5,-6 364.5,-6 364.5,-6 370.5,-6 376.5,-12 376.5,-18 376.5,-18 376.5,-38 376.5,-38 376.5,-44 370.5,-50 364.5,-50 364.5,-50 237.5,-50 237.5,-50 231.5,-50 225.5,-44 225.5,-38 225.5,-38 225.5,-18 225.5,-18 225.5,-12 231.5,-6 237.5,-6\"/>\n</a>\n</g>\n</g>\n<!-- 2947cead&#45;1a91&#45;4b32&#45;9ad3&#45;81a0836d6d7e&#45;&gt;e13ffc42&#45;0733&#45;495b&#45;a78d&#45;34bc8ff01d0e -->\n<g id=\"edge6\" class=\"edge\">\n<title>2947cead-1a91-4b32-9ad3-81a0836d6d7e-&gt;e13ffc42-0733-495b-a78d-34bc8ff01d0e</title>\n<path fill=\"none\" stroke=\"#006400\" stroke-width=\"2\" d=\"M301,-56.1C301,-63.94 301,-72.81 301,-81.73\"/>\n<polygon fill=\"#006400\" stroke=\"#006400\" stroke-width=\"2\" points=\"297.5,-81.77 301,-91.77 304.5,-81.77 297.5,-81.77\"/>\n</g>\n</g>\n</svg>"
          },
          "metadata": {}
        }
      ],
      "source": [
        "# Render argument map (and other generated artifacts)\n",
        "from IPython.display import display, Markdown\n",
        "\n",
        "example_idx = 0\n",
        "\n",
        "for artifact_id in config_dict.get(\"artifacts\",[]):\n",
        "    display(Markdown(f\"## {artifact_id}\"))\n",
        "    data = df_scores.iloc[example_idx].get(artifact_id)\n",
        "    r = render_artifact(artifact_id, data)\n",
        "    if isinstance(r, tuple):\n",
        "        display(*r)\n",
        "    else:\n",
        "        display(r)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Part III – Fact-checking the Legal AI with **Logikon `/\\/`** argument maps<a name=\"part-c\"></a>"
      ],
      "metadata": {
        "id": "opS64wQZXiqM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# LLM serving as AI fact checker\n",
        "import os\n",
        "from huggingface_hub import InferenceClient\n",
        "\n",
        "factchecker_ai = InferenceClient(\n",
        "    \"mistralai/Mistral-7B-Instruct-v0.1\",\n",
        "    token=os.environ.get(\"HUGGINGFACEHUB_API_TOKEN\"),\n",
        ")\n",
        "factchecker_gen_kwargs = {\n",
        "    \"details\": True,\n",
        "    \"max_new_tokens\": 3,\n",
        "    \"temperature\":.3,\n",
        "    \"do_sample\": True,\n",
        "}"
      ],
      "metadata": {
        "id": "LnY9apIDZRol"
      },
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# our reference corpus is a single text: simply an excerpt from the Supreme Court's decision\n",
        "reference_text = \"\"\"\n",
        "SUPREME COURT OF THE UNITED STATES\n",
        "No. 11-796\n",
        "VERNON HUGH BOWMAN, PETITIONER v.\n",
        "MONSANTO COMPANY ET AL.\n",
        "\n",
        "[...]\n",
        "\n",
        "Respondent Monsanto invented a genetic modification that enables soybean plants to survive exposure to glyphosate, the active ingredient in many herbicides (including Monsanto’s own Roundup). Monsanto markets soybean seed containing this altered genetic material as Roundup Ready seed. Farmers planting that seed can use a glyphosate- based herbicide to kill weeds without damaging their crops. Two patents issued to Monsanto cover various aspects of its Roundup Ready technology, including a seed in- corporating the genetic alteration. See Supp. App. SA1–21 (U. S. Patent Nos. 5,352,605 and RE39,247E); see also 657 F. 3d 1341, 1343–1344 (CA Fed. 2011).\n",
        "\n",
        "Monsanto sells, and allows other companies to sell, Roundup Ready soybean seeds to growers who assent to a special licensing agreement. See App. 27a. That agreement permits a grower to plant the purchased seeds in one (and only one) season. He can then consume the resulting crop or sell it as a commodity, usually to a grain elevator or agricultural processor. See 657 F. 3d, at 1344–1345. But under the agreement, the farmer may not save any of the harvested soybeans for replanting, nor may he supply them to anyone else for that purpose. These restrictions reflect the ease of producing new generations of Roundup Ready seed. Because glyphosate resistance comes from the seed’s genetic material, that trait is passed on from the planted seed to the harvested soybeans: Indeed, a single Roundup Ready seed can grow a plant containing dozens of genetically identical beans, each of which, if replanted, can grow another such plant—and so on and so on. See App. 100a. The agreement’s terms prevent the farmer from co-opting that process to produce his own Roundup Ready seeds, forcing him instead to buy from Monsanto each season.\n",
        "\n",
        "Petitioner Vernon Bowman is a farmer in Indiana who, it is fair to say, appreciates Roundup Ready soybean seed. He purchased Roundup Ready each year, from a company affiliated with Monsanto, for his first crop of the season. In accord with the agreement just described, he used all of that seed for planting, and sold his entire crop to a grain elevator (which typically would resell it to an agricultural processor for human or animal consumption).\n",
        "\n",
        "Bowman, however, devised a less orthodox approach for his second crop of each season. Because he thought such late-season planting “risky,” he did not want to pay the premium price that Monsanto charges for Roundup Ready seed. He therefore went to a grain elevator; purchased “commodity soybeans” intended for human or animal consumption; and planted them in his fields.1 Those soybeans came from prior harvests of other local farmers. And because most of those farmers also used Roundup Ready seed, Bowman could anticipate that many of the purchased soybeans would contain Monsanto’s patented technology. When he applied a glyphosate-based herbicide to his fields, he confirmed that this was so; a significant proportion of the new plants survived the treatment, and produced in their turn a new crop of soybeans with the Roundup Ready trait. Bowman saved seed from that crop to use in his late-season planting the next year—and then the next, and the next, until he had harvested eight crops in that way. Each year, that is, he planted saved seed from the year before (sometimes adding more soybeans bought from the grain elevator), sprayed his fields with glyphosate to kill weeds (and any non-resistant plants), and produced a new crop of glyphosate- resistant—i.e., Roundup Ready—soybeans.\n",
        "\n",
        "After discovering this practice, Monsanto sued Bowman for infringing its patents on Roundup Ready seed. Bowman raised patent exhaustion as a defense, arguing that Monsanto could not control his use of the soybeans because they were the subject of a prior authorized sale (from local farmers to the grain elevator). The District Court rejected that argument, and awarded damages to Monsanto of $84,456. The Federal Circuit affirmed. It reasoned that patent exhaustion did not protect Bowman because he had “created a newly infringing article.” 657 F. 3d, at 1348. The “right to use” a patented article following an authorized sale, the court explained, “does not include the right to construct an essentially new article on the template of the original, for the right to make the article remains with the patentee.” Ibid. (brackets and internal quotation marks omitted). Accordingly, Bowman could not “‘replicate’ Monsanto’s patented technology by planting it in the ground to create newly infringing genetic material, seeds, and plants.” Ibid.\n",
        "\"\"\"\n",
        "from IPython.display import display, Markdown\n",
        "display(Markdown(reference_text))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 655
        },
        "id": "XbkcBnLF3_5G",
        "outputId": "2a7bdb4f-1daa-41f0-b3f0-a56cc4f4a3f1"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "\nSUPREME COURT OF THE UNITED STATES\nNo. 11-796\nVERNON HUGH BOWMAN, PETITIONER v.\nMONSANTO COMPANY ET AL.\n\n[...]\n\nRespondent Monsanto invented a genetic modification that enables soybean plants to survive exposure to glyphosate, the active ingredient in many herbicides (including Monsanto’s own Roundup). Monsanto markets soybean seed containing this altered genetic material as Roundup Ready seed. Farmers planting that seed can use a glyphosate- based herbicide to kill weeds without damaging their crops. Two patents issued to Monsanto cover various aspects of its Roundup Ready technology, including a seed in- corporating the genetic alteration. See Supp. App. SA1–21 (U. S. Patent Nos. 5,352,605 and RE39,247E); see also 657 F. 3d 1341, 1343–1344 (CA Fed. 2011).\n\nMonsanto sells, and allows other companies to sell, Roundup Ready soybean seeds to growers who assent to a special licensing agreement. See App. 27a. That agreement permits a grower to plant the purchased seeds in one (and only one) season. He can then consume the resulting crop or sell it as a commodity, usually to a grain elevator or agricultural processor. See 657 F. 3d, at 1344–1345. But under the agreement, the farmer may not save any of the harvested soybeans for replanting, nor may he supply them to anyone else for that purpose. These restrictions reflect the ease of producing new generations of Roundup Ready seed. Because glyphosate resistance comes from the seed’s genetic material, that trait is passed on from the planted seed to the harvested soybeans: Indeed, a single Roundup Ready seed can grow a plant containing dozens of genetically identical beans, each of which, if replanted, can grow another such plant—and so on and so on. See App. 100a. The agreement’s terms prevent the farmer from co-opting that process to produce his own Roundup Ready seeds, forcing him instead to buy from Monsanto each season.\n\nPetitioner Vernon Bowman is a farmer in Indiana who, it is fair to say, appreciates Roundup Ready soybean seed. He purchased Roundup Ready each year, from a company affiliated with Monsanto, for his first crop of the season. In accord with the agreement just described, he used all of that seed for planting, and sold his entire crop to a grain elevator (which typically would resell it to an agricultural processor for human or animal consumption).\n\nBowman, however, devised a less orthodox approach for his second crop of each season. Because he thought such late-season planting “risky,” he did not want to pay the premium price that Monsanto charges for Roundup Ready seed. He therefore went to a grain elevator; purchased “commodity soybeans” intended for human or animal consumption; and planted them in his fields.1 Those soybeans came from prior harvests of other local farmers. And because most of those farmers also used Roundup Ready seed, Bowman could anticipate that many of the purchased soybeans would contain Monsanto’s patented technology. When he applied a glyphosate-based herbicide to his fields, he confirmed that this was so; a significant proportion of the new plants survived the treatment, and produced in their turn a new crop of soybeans with the Roundup Ready trait. Bowman saved seed from that crop to use in his late-season planting the next year—and then the next, and the next, until he had harvested eight crops in that way. Each year, that is, he planted saved seed from the year before (sometimes adding more soybeans bought from the grain elevator), sprayed his fields with glyphosate to kill weeds (and any non-resistant plants), and produced a new crop of glyphosate- resistant—i.e., Roundup Ready—soybeans.\n\nAfter discovering this practice, Monsanto sued Bowman for infringing its patents on Roundup Ready seed. Bowman raised patent exhaustion as a defense, arguing that Monsanto could not control his use of the soybeans because they were the subject of a prior authorized sale (from local farmers to the grain elevator). The District Court rejected that argument, and awarded damages to Monsanto of $84,456. The Federal Circuit affirmed. It reasoned that patent exhaustion did not protect Bowman because he had “created a newly infringing article.” 657 F. 3d, at 1348. The “right to use” a patented article following an authorized sale, the court explained, “does not include the right to construct an essentially new article on the template of the original, for the right to make the article remains with the patentee.” Ibid. (brackets and internal quotation marks omitted). Accordingly, Bowman could not “‘replicate’ Monsanto’s patented technology by planting it in the ground to create newly infringing genetic material, seeds, and plants.” Ibid.\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "prompt_factchecking= (\n",
        "    '<s>[INST] '\n",
        "    'Assignment: Determine whether a claim is asserted or implied by a reference text.\\n'\n",
        "    'Read the following reference text carefully:\\n'\n",
        "    '<text>\\n{text}\\n</text>'\n",
        "    'You are supposed to judge whether the reference text contains (states or implies) '\n",
        "    'the following claim:\\n'\n",
        "    '<claim>\\n{claim}\\n</claim>'\n",
        "    'Now, re-read the text carefully, and check whether the claim is true according to '\n",
        "    'the text.\\n'\n",
        "    'Does the text assert or imply the claim?\\n'\n",
        "    '(A) The text implies that the claim is true.'\n",
        "    '(B) The text does not imply that the claim is true.'\n",
        "    'Just answer with \"A\" or \"B\", you\\'ll have the opportunity to explain '\n",
        "    'your answer later.'\n",
        "    '[/INST]\\n'\n",
        ")"
      ],
      "metadata": {
        "id": "2kUa3-fzZRol"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# monte-carlo inference to estimate confidence levels\n",
        "def factual_accuracy(claim, N=9):\n",
        "    \"\"\"factchecker_ai's confidence in claim being implied by reference_text\"\"\"\n",
        "    completions = []\n",
        "    for i in range(N):\n",
        "        completion = factchecker_ai.text_generation(\n",
        "            prompt_factchecking.format(\n",
        "                text=reference_text,\n",
        "                claim=claim,\n",
        "            ),\n",
        "            seed=i,\n",
        "            **factchecker_gen_kwargs,\n",
        "        )\n",
        "        completions.append(completion)\n",
        "    token_texts = [\n",
        "        [t.text for t in c.details.tokens]\n",
        "        for c in completions\n",
        "    ]\n",
        "    count_asserted = sum(('A' in tts) for tts in token_texts)\n",
        "    return count_asserted/N"
      ],
      "metadata": {
        "id": "qlcSscyqZRol"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "\n",
        "# get reconstructed argument map\n",
        "argmap = df_scores.iloc[0].get(\"fuzzy_argmap_nx\")\n",
        "\n",
        "# score factual accuracy of each reason claim in argmap\n",
        "hallucination_data = []\n",
        "for node in argmap[\"nodes\"]:\n",
        "  if node.get(\"node_type\") == \"reason\":\n",
        "    text = node.get(\"text\")\n",
        "    factivity_score = factual_accuracy(text) if text else None\n",
        "    hallucination_data.append({\n",
        "        \"label\": node.get(\"label\"),\n",
        "        \"text\": text,\n",
        "        \"factual_accuracy\": factivity_score,\n",
        "    })\n",
        "df_factivity = pd.DataFrame(hallucination_data)"
      ],
      "metadata": {
        "id": "dVHKLG-r-6jf"
      },
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# plot factual accuracy of reason claims\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "cmap = plt.get_cmap(\"plasma_r\")\n",
        "\n",
        "fig, ax = plt.subplots()\n",
        "ax.barh(width=\"factual_accuracy\", y=\"label\", data=df_factivity, color=cmap(df_factivity.factual_accuracy.values))\n",
        "ax.set_xlim(0,1)\n",
        "ax.set_xlabel(\"Factual Accuray\\n (confidence that claim is implied by reference text)\")\n",
        "plt.show()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 466
        },
        "id": "ec4I2Bg-ENc4",
        "outputId": "949728fe-23c0-49dd-bb20-6bd78a36afa6"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuUAAAHBCAYAAADO78VDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABrRklEQVR4nO3deVxO6f8/8Ndd6W4vJZURSYtKpWSJUShTZjSyjHVSyG7sMsZE2RnZxzKYsg3GIIbsZJR9qUEppcRHxliTpe0+vz/8nK9bizJxLK/n43E/Hu5zrnOd9znnvvW6r677JBMEQQAREREREUlGReoCiIiIiIg+dQzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUlMTeoCiD5lCoUCN2/ehK6uLmQymdTlEBERUTkIgoBHjx6hRo0aUFGpnDFuhnIiCd28eRPm5uZSl0FERERv4Pr166hZs2al9MVQTiQhXV1dAM/f1Hp6ehJXQ0REROWRk5MDc3Nz8ed4ZWAoJ5LQiykrenp6DOVEREQfmMqcesovehIRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHEGMqJiIiIiCSmJnUBRAT8oL8FcpmW1GUQEZEEIhRdpS6B3gMcKSciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMop/dWWFgYTExMIJPJEB0dLXU5AIDMzEzIZDIkJCRIXQoRERF9RBjKPxBBQUGQyWSYOXOm0vLo6GjIZLIK9dWyZUuMGDGiEqurfMnJyQgPD8fy5cuRnZ2Ntm3bSl0SERER0VvDUP4B0dDQwKxZs3D//n2pSwEA5Ofnv7W+09PTAQDt27eHqakp5HL5W9sXERERkdQYyj8g3t7eMDU1xYwZM0ptc/fuXXTv3h2fffYZtLS04OjoiA0bNojrg4KCcOTIESxYsAAymQwymQyZmZmIioqCgYGBUl+vjsKHhYWhQYMGWLlyJerUqQMNDQ0AwIMHDxAcHAxjY2Po6emhdevWSExMLPNYLly4gNatW0NTUxNGRkbo378/cnNzxf34+fkBAFRUVEr9TcD9+/fRs2dPGBsbQ1NTE9bW1oiMjBTXX79+HV26dIGBgQEMDQ3Rvn17ZGZmKvWxcuVK2NnZQUNDA/Xq1cOSJUuU1p86dQouLi7Q0NCAm5sbzp8/X6EaiIiIiMqDofwDoqqqiunTp2PRokW4ceNGiW2ePXuGhg0bYteuXbh48SL69++PgIAAnDp1CgCwYMECuLu7o1+/fsjOzkZ2djbMzc3LXUNaWhq2bNmCrVu3ivOqv/nmG9y+fRu7d+/G2bNn4erqCi8vL9y7d6/EPh4/fgwfHx9UrVoVp0+fxubNm3HgwAEMHToUADBmzBgx2L6osSShoaFISkrC7t27kZycjKVLl6JatWoAgIKCAvj4+EBXVxdHjx5FfHw8dHR04OvrK47wr1+/HhMnTsS0adOQnJyM6dOnIzQ0FKtXrwYA5Obmol27drC3t8fZs2cRFhaGMWPGlLsGIiIiovJSk7oAqpgOHTqgQYMGmDRpElatWlVs/WeffaYUHL/77jvs3bsXv//+Oxo3bgx9fX2oq6tDS0sLpqamFd5/fn4+1qxZA2NjYwBAXFwcTp06hdu3b4tTTObMmYPo6Gj88ccf6N+/f7E+fvvtNzx79gxr1qyBtrY2AGDx4sXw8/PDrFmzYGJiIo7al1VjVlYWXFxc4ObmBgCwsLAQ123atAkKhQIrV64UR9ojIyNhYGCA2NhYfPHFF5g0aRIiIiLQsWNHAECdOnWQlJSE5cuXIzAwEL/99hsUCgVWrVoFDQ0NODg44MaNGxg0aFC5aihJXl4e8vLyxOc5OTllticiIqJPA0P5B2jWrFlo3bp1sVFbACgqKsL06dPx+++/43//+x/y8/ORl5cHLS2tStl37dq1xUAOAImJicjNzYWRkZFSu6dPn4rzwl+VnJwMZ2dnMZADQPPmzaFQKJCSkgITE5Ny1TJo0CB06tQJ586dwxdffAF/f380a9ZMrCstLQ26urpK2zx79gzp6el4/Pgx0tPT0bdvX/Tr109cX1hYCH19fbFOJycncZoOALi7u5e7hpLMmDED4eHh5To+IiIi+nQwlH+APDw84OPjg/HjxyMoKEhp3U8//YQFCxZg/vz5cHR0hLa2NkaMGPHaL2WqqKhAEASlZQUFBcXavRykgedTPMzMzBAbG1us7atz1Ctb27Ztce3aNcTExGD//v3w8vLCkCFDMGfOHOTm5qJhw4ZYv359se2MjY3F+esrVqxAkyZNlNarqqpWSg0lGT9+PEaNGiU+z8nJqdD0ISIiIvo4MZR/oGbOnIkGDRrA1tZWaXl8fDzat2+Pb7/9FgCgUCiQmpoKe3t7sY26ujqKioqUtjM2NsajR4/w+PFjMXiX517crq6uuHXrFtTU1F47deMFOzs7REVFKe0rPj4eKioqxY7ndYyNjREYGIjAwEC0aNECY8eOxZw5c+Dq6opNmzahevXq0NPTK7advr4+atSogatXr6Jnz56l1rl27Vo8e/ZMHC0/ceJEuWsoiVwu551kiIiIqBh+0fMD5ejoiJ49e2LhwoVKy62trbF//34cO3YMycnJGDBgAP755x+lNhYWFjh58iQyMzNx584dKBQKNGnSBFpaWvjhhx+Qnp6O3377DVFRUa+tw9vbG+7u7vD398e+ffuQmZmJY8eOYcKECThz5kyJ2/Ts2RMaGhoIDAzExYsXcfjwYXz33XcICAgo99QVAJg4cSK2b9+OtLQ0XLp0CTt37oSdnZ24j2rVqqF9+/Y4evQoMjIyEBsbi2HDholfkg0PD8eMGTOwcOFCpKam4sKFC4iMjMTcuXMBAD169IBMJkO/fv2QlJSEmJiYYmG7rBqIiIiIyouh/AM2efJkKBQKpWU//vgjXF1d4ePjg5YtW8LU1BT+/v5KbcaMGQNVVVXY29vD2NgYWVlZMDQ0xLp16xATEyPeRjEsLOy1NchkMsTExMDDwwO9e/eGjY0NunXrhmvXrpUasLW0tLB3717cu3cPjRo1QufOneHl5YXFixdX6PjV1dUxfvx4ODk5wcPDA6qqqti4caO4j7/++gu1atVCx44dYWdnh759++LZs2fiyHlwcDBWrlyJyMhIODo6wtPTE1FRUahTpw4AQEdHB3/++ScuXLgAFxcXTJgwAbNmzSp3DURERETlJRNenUhMRO9MTk4O9PX1MQS/Qi6rnC/jEhHRhyVC0VXqEqiCXvz8fvjwYYnTZN8ER8qJiIiIiCTGUE5EREREJDGGciIiIiIiiTGUExERERFJjKGciIiIiEhiDOVERERERBJjKCciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJKYmdQFEBEx/2Al6enpSl0FEREQS4Ug5EREREZHEGMqJiIiIiCTGUE5EREREJDGGciIiIiIiiTGUExERERFJjKGciIiIiEhiDOVERERERBJjKCciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxNakLICKgQ9X1UJNpSl0GERFRpdpbGCR1CR8MjpQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHEGMqJiIiIiCTGUE5EREREJDGGciIiIiIiiTGUExERERFJjKGciIiIiEhiDOVERERERBJjKCciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIokxlNMnRSaTITo6WuoyiIiIiJQwlL+ngoKCIJPJij3S0tKkLq3cZDIZMjMzJdl3WFgYGjRoUGx5dnY22rZt++4LIiIiIiqDmtQFUOl8fX0RGRmptMzY2LjC/RQVFUEmk0FF5d18BsvPz4e6uvo72VdFmZqaSl0CERERUTEcKX+PyeVymJqaKj1UVVUxd+5cODo6QltbG+bm5hg8eDByc3PF7aKiomBgYIAdO3bA3t4ecrkcWVlZsLCwwNSpU9GrVy/o6Oigdu3a2LFjB/7991+0b98eOjo6cHJywpkzZ5TqiIuLQ4sWLaCpqQlzc3MMGzYMjx8/FtdbWFhgypQp6NWrF/T09NC/f/9ix3L//n307NkTxsbG0NTUhLW1dbEPHC/bs2cPPv/8cxgYGMDIyAjt2rVDenq6UpsbN26ge/fuMDQ0hLa2Ntzc3HDy5ElERUUhPDwciYmJ4m8YoqKiABSfvnLhwgW0bt0ampqaMDIyQv/+/ZXOZVBQEPz9/TFnzhyYmZnByMgIQ4YMQUFBgdhmyZIlsLa2hoaGBkxMTNC5c+eyLywRERHRKxjKP0AqKipYuHAhLl26hNWrV+PQoUMICQlRavPkyRPMmjULK1euxKVLl1C9enUAwLx589C8eXOcP38eX331FQICAtCrVy98++23OHfuHOrWrYtevXpBEAQAQHp6Onx9fdGpUyf8/fff2LRpE+Li4jB06FCl/c2ZMwfOzs44f/48QkNDi9UcGhqKpKQk7N69G8nJyVi6dCmqVatW6jE+fvwYo0aNwpkzZ3Dw4EGoqKigQ4cOUCgUAIDc3Fx4enrif//7H3bs2IHExESEhIRAoVCga9euGD16NBwcHJCdnY3s7Gx07dq1xH34+PigatWqOH36NDZv3owDBw4UO7bDhw8jPT0dhw8fxurVqxEVFSWG/DNnzmDYsGGYPHkyUlJSsGfPHnh4eLzmChIREREpkwkv0he9V4KCgrBu3TpoaGiIy9q2bYvNmzcXa/vHH39g4MCBuHPnDoDnI+W9e/dGQkICnJ2dxXYWFhZo0aIF1q5dCwC4desWzMzMEBoaismTJwMATpw4AXd3d2RnZ8PU1BTBwcFQVVXF8uXLxX7i4uLg6emJx48fQ0NDAxYWFnBxccG2bdtKPZ6vv/4a1apVw6+//vpG5+POnTswNjbGhQsXUL9+ffzyyy8YM2YMMjMzYWhoWKx9WFgYoqOjkZCQoLRcJpNh27Zt8Pf3x4oVKzBu3Dhcv34d2traAICYmBj4+fnh5s2bMDExQVBQEGJjY5Geng5VVVUAQJcuXaCiooKNGzdi69at6N27N27cuAFdXd3XHkdeXh7y8vLE5zk5OTA3N0drlSVQk2m+0bkhIiJ6X+0tDJK6hLciJycH+vr6ePjwIfT09CqlT46Uv8datWqFhIQE8bFw4UIAwIEDB+Dl5YXPPvsMurq6CAgIwN27d/HkyRNxW3V1dTg5ORXr8+VlJiYmAABHR8diy27fvg0ASExMRFRUFHR0dMSHj48PFAoFMjIyxO3c3NzKPJZBgwZh48aNaNCgAUJCQnDs2LEy21+5cgXdu3eHpaUl9PT0YGFhAQDIysoCACQkJMDFxaXEQF5eycnJcHZ2FgM5ADRv3hwKhQIpKSniMgcHBzGQA4CZmZl4ftq0aYPatWvD0tISAQEBWL9+vdJ1eNWMGTOgr68vPszNzd+4fiIiIvp4MJS/x7S1tWFlZSU+zMzMkJmZiXbt2sHJyQlbtmzB2bNn8fPPPwN4/gXLFzQ1NSGTyYr1WaVKFfHfL9aXtOzlaSIDBgxQ+nCQmJiIK1euoG7dukq1lqVt27a4du0aRo4ciZs3b8LLywtjxowptb2fnx/u3buHFStW4OTJkzh58qTSMWpqvrtR5ZfPD/D8HL04P7q6ujh37hw2bNgAMzMzTJw4Ec7Oznjw4EGJfY0fPx4PHz4UH9evX3/b5RMREdEHgKH8A3P27FkoFApERESgadOmsLGxwc2bN9/a/lxdXZGUlKT04eDFo6J3WDE2NkZgYCDWrVuH+fPn45dffimx3d27d5GSkoIff/wRXl5esLOzw/3795XaODk5ISEhAffu3SuxD3V1dRQVFZVZj52dHRITE5W+tBofHw8VFRXY2tqW+7jU1NTg7e2N2bNn4++//0ZmZiYOHTpUYlu5XA49PT2lBxERERFD+QfGysoKBQUFWLRoEa5evYq1a9di2bJlb21/48aNw7FjxzB06FAkJCTgypUr2L59e7EvQ77OxIkTsX37dqSlpeHSpUvYuXMn7OzsSmxbtWpVGBkZ4ZdffkFaWhoOHTqEUaNGKbXp3r07TE1N4e/vj/j4eFy9ehVbtmzB8ePHATyfP5+RkYGEhATcuXNHaR73Cz179oSGhgYCAwNx8eJFHD58GN999x0CAgLEaTyvs3PnTixcuBAJCQm4du0a1qxZA4VCUaFQT0RERMRQ/oFxdnbG3LlzMWvWLNSvXx/r16/HjBkz3tr+nJyccOTIEaSmpqJFixZwcXHBxIkTUaNGjQr1o66ujvHjx8PJyQkeHh5QVVXFxo0bS2z74kuUZ8+eRf369TFy5Ej89NNPxfrbt28fqlevji+//BKOjo6YOXOmOPe7U6dO8PX1RatWrWBsbIwNGzYU24+Wlhb27t2Le/fuoVGjRujcuTO8vLywePHich+XgYEBtm7ditatW8POzg7Lli3Dhg0b4ODgUIGzQ0RERJ863n2FSEIvvr3Nu68QEdHHiHdfKT+OlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHEGMqJiIiIiCTGUE5EREREJDGGciIiIiIiiTGUExERERFJTE3qAogI2Ha/J/T09KQug4iIiCTCkXIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGJqUhdARMAaq3XQVNGUugwiovdS31u9pS6B6K3jSDkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHEGMqJiIiIiCTGUE5EREREJDGGciIiIiIiiTGUExERERFJjKGciIiIiEhiDOVERERERBJjKCciIiIiktgHGcqjoqJgYGBQaf0FBQXB39+/0vqrCAsLC8yfP7/c7TMzMyGTyZCQkPDWavpYxcbGQiaT4cGDB1KXQkRERKTkrYTyoKAgyGSyYg9fX9+3sbv/bMGCBYiKipJk36dPn0b//v0rtc/yfmgJCwtDgwYNKnXfRERERFRxam+rY19fX0RGRiotk8vlb2t3/4m+vv4732d+fj7U1dVhbGz8zvf9LhQUFKBKlSqV3q8gCCgqKoKa2lt76RIRERG9c29t+opcLoepqanSo2rVqgCeTyNQV1fH0aNHxfazZ89G9erV8c8//wAAHjx4gAEDBsDExAQaGhqoX78+du7cWeK+Spp+MmLECLRs2VJ8/scff8DR0RGampowMjKCt7c3Hj9+XGz7X375BTVq1IBCoVDqr3379ujTp4/4fPv27XB1dYWGhgYsLS0RHh6OwsLCUs/Hi31MmzYNNWrUgK2tLYDi01cuX76Mzz//HBoaGrC3t8eBAwcgk8kQHR2t1N/Vq1fRqlUraGlpwdnZGcePHxfPbe/evfHw4UPxNxRhYWHF6omKikJ4eDgSExPFdi9+W5CVlYX27dtDR0cHenp66NKli3hdSvJiSs2mTZvg6ekJDQ0NrF+/HgCwcuVK2NnZQUNDA/Xq1cOSJUuKbbdx40Y0a9ZMvM5HjhwR27yYcrJ79240bNgQcrkccXFxyMvLw7Bhw1C9enVoaGjg888/x+nTp5XqiomJgY2NDTQ1NdGqVStkZmYqrS/pNwXz58+HhYWF0rJff/0VDg4OkMvlMDMzw9ChQ8V1Dx48QHBwMIyNjaGnp4fWrVsjMTGx1HNFREREVBJJhhtbtmyJESNGICAgAImJibh69SpCQ0OxefNmmJiYQKFQoG3btnj06BHWrVuHunXrIikpCaqqqm+0v+zsbHTv3h2zZ89Ghw4d8OjRIxw9ehSCIBRr+8033+C7777D4cOH4eXlBQC4d+8e9uzZg5iYGADA0aNH0atXLyxcuBAtWrRAenq6OAVl0qRJpdZx8OBB6OnpYf/+/SWuLyoqgr+/P2rVqoWTJ0/i0aNHGD16dIltJ0yYgDlz5sDa2hoTJkxA9+7dkZaWhmbNmmH+/PmYOHEiUlJSAAA6OjrFtu/atSsuXryIPXv24MCBAwCe/8ZAoVCIgfzIkSMoLCzEkCFD0LVrV8TGxpZ6bADw/fffIyIiAi4uLmIwnzhxIhYvXgwXFxecP38e/fr1g7a2NgIDA8Xtxo4di/nz58Pe3h5z586Fn58fMjIyYGRkpNT3nDlzYGlpiapVqyIkJARbtmzB6tWrUbt2bcyePRs+Pj5IS0uDoaEhrl+/jo4dO2LIkCHo378/zpw5U+q5LMvSpUsxatQozJw5E23btsXDhw8RHx8vrv/mm2+gqamJ3bt3Q19fH8uXL4eXlxdSU1NhaGhY4f0RERHRp+mthfKdO3cWC4M//PADfvjhBwDA1KlTsX//fvTv3x8XL15EYGAgvv76awDAgQMHcOrUKSQnJ8PGxgYAYGlp+ca1ZGdno7CwEB07dkTt2rUBAI6OjiW2rVq1Ktq2bYvffvtNDOV//PEHqlWrhlatWgEAwsPD8f3334vB0tLSElOmTEFISEiZoVxbWxsrV66Eurp6iev379+P9PR0xMbGwtTUFAAwbdo0tGnTpljbMWPG4KuvvhLrcXBwQFpaGurVqwd9fX3IZDKxj5JoampCR0cHampqSu3279+PCxcuICMjA+bm5gCANWvWwMHBAadPn0ajRo1K7XPEiBHo2LGj+HzSpEmIiIgQl9WpUwdJSUlYvny5UigfOnQoOnXqBOB5CN6zZw9WrVqFkJAQsc3kyZPF8/D48WMsXboUUVFRaNu2LQBgxYoV2L9/P1atWoWxY8di6dKlqFu3LiIiIgAAtra2uHDhAmbNmlVq/SWZOnUqRo8ejeHDh4vLXpyDuLg4nDp1Crdv3xanZs2ZMwfR0dH4448/SvyuQF5eHvLy8sTnOTk5FaqHiIiIPk5vLZS3atUKS5cuVVr28sihuro61q9fDycnJ9SuXRvz5s0T1yUkJKBmzZpiIP+vnJ2d4eXlBUdHR/j4+OCLL75A586dxek0r+rZsyf69euHJUuWQC6XY/369ejWrRtUVJ7P9klMTER8fDymTZsmblNUVIRnz57hyZMn0NLSKrFfR0fHUgM5AKSkpMDc3FwpJDdu3LjEtk5OTuK/zczMAAC3b99GvXr1Su2/PJKTk2Fubi4GcgCwt7eHgYEBkpOTywzlbm5u4r8fP36M9PR09O3bF/369ROXFxYWFpvD7+7uLv5bTU0Nbm5uSE5OLrXv9PR0FBQUoHnz5uKyKlWqoHHjxuJ2ycnJaNKkSan7KY/bt2/j5s2b4oezVyUmJiI3N1dpRB8Anj59ivT09BK3mTFjBsLDwytUBxEREX383loo19bWhpWVVZltjh07BuD59JB79+5BW1sbwPNR3IpQUVEpNhWloKBA/Leqqir279+PY8eOYd++fVi0aBEmTJiAkydPok6dOsX68/PzgyAI2LVrFxo1aoSjR48qfWjIzc1FeHi40qjwCxoaGqXW+eL4KsPLX6KUyWQAUGwe/Lv28vHl5uYCeD6C/Wo4fpNpSJV57l543evmda/D3NxcmJmZlTitp7S734wfPx6jRo0Sn+fk5Ch9ACIiIqJPk2T3KU9PT8fIkSPF0BYYGCiGSicnJ9y4cQOpqanl6svY2BjZ2dlKy169j7dMJkPz5s0RHh6O8+fPQ11dHdu2bSuxPw0NDXTs2BHr16/Hhg0bYGtrC1dXV3G9q6srUlJSYGVlVezxYjT9Tdja2uL69etKX6p89cuL5aGuro6ioqI3amdnZ4fr16/j+vXr4rKkpCQ8ePAA9vb25a7BxMQENWrUwNWrV4udo1c/CJ04cUL8d2FhIc6ePQs7O7tS+65bty7U1dWV5nYXFBTg9OnTYo12dnY4depUqfsBnr9ubt26pRTMX37d6OrqwsLCAgcPHiyxDldXV9y6dQtqamrFjrFatWolbiOXy6Gnp6f0ICIiInprI+V5eXm4deuW8s7U1FCtWjUUFRXh22+/hY+PD3r37g1fX184OjoiIiICY8eOhaenJzw8PNCpUyfMnTsXVlZWuHz5cqn3Om/dujV++uknrFmzBu7u7li3bh0uXrwIFxcXAMDJkydx8OBBfPHFF6hevTpOnjyJf//9t8zg17NnT7Rr1w6XLl3Ct99+q7Ru4sSJaNeuHWrVqoXOnTtDRUUFiYmJuHjxIqZOnfrG56xNmzaoW7cuAgMDMXv2bDx69Ag//vgjgP8bDS8PCwsL5Obm4uDBg3B2doaWllaJU2osLCyQkZEhThfS1dWFt7c3HB0d0bNnT8yfPx+FhYUYPHgwPD09laaQlEd4eDiGDRsGfX19+Pr6Ii8vD2fOnMH9+/eVRot//vlnWFtbw87ODvPmzcP9+/eV7nTzKm1tbQwaNAhjx46FoaEhatWqhdmzZ+PJkyfo27cvAGDgwIHi6yk4OBhnz54tdi/6li1b4t9//8Xs2bPRuXNn7NmzB7t371YKymFhYRg4cCCqV68ufvk4Pj4e3333Hby9veHu7g5/f3/Mnj0bNjY2uHnzJnbt2oUOHTpU+HwRERHRp+utjZTv2bMHZmZmSo/PP/8cwPMvL167dg3Lly8H8HxO9C+//IIff/xRvJ3cli1b0KhRI3Tv3h329vYICQkpdfTXx8cHoaGhCAkJQaNGjfDo0SP06tVLXK+np4e//voLX375JWxsbPDjjz8iIiJC/JJgSVq3bg1DQ0OkpKSgR48exfa3c+dO7Nu3D40aNULTpk0xb9488Uukb0pVVRXR0dHIzc1Fo0aNEBwcjAkTJgAoe1rMq5o1a4aBAweia9euMDY2xuzZs0ts16lTJ/j6+qJVq1YwNjbGhg0bIJPJsH37dlStWhUeHh7w9vaGpaUlNm3aVOHjCQ4OxsqVKxEZGQlHR0d4enoiKiqq2Ej5zJkzMXPmTDg7OyMuLg47duwodaT55W06deqEgIAAuLq6Ii0tDXv37hW/J1CrVi1s2bIF0dHRcHZ2xrJlyzB9+nSlPuzs7LBkyRL8/PPPcHZ2xqlTpzBmzBilNoGBgZg/fz6WLFkCBwcHtGvXDleuXAHw/INSTEwMPDw80Lt3b9jY2KBbt264du0aTExMKny+iIiI6NMlE0q6LyC9N+Lj4/H5558jLS0NdevWlbqcSpWZmYk6derg/Pnzn+xfFs3JyYG+vj4WGf8MTZWKfZeCiOhT0fdWb6lLIFLy4uf3w4cPK20qKv8s4ntm27Zt0NHRgbW1NdLS0jB8+HA0b978owvkRERERPR/GMrfM48ePcK4ceOQlZWFatWqwdvbW7zXNhERERF9nBjK3zO9evVSmg//MbOwsCjxr6oSERERfWokuyUiERERERE9x1BORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREElOTugAiAnqlfQs9PT2pyyAiIiKJcKSciIiIiEhiDOVERERERBJjKCciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpKYmtQFEBFwpHUEtNU0pC6DiD5grU+Ml7oEIvoPOFJORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHEGMqJiIiIiCT2wYRymUyG6OhoqcuoMAsLC8yfP1/qMop5XV1BQUHw9/d/Z/UQERERfcoqFMr//fdfDBo0CLVq1YJcLoepqSl8fHwQHx//tuqjj4xMJhMfampqqFWrFkaNGoW8vDypSyMiIiKSjFpFGnfq1An5+flYvXo1LC0t8c8//+DgwYO4e/fu26rvvVZQUIAqVapIXcYHJzIyEr6+vigoKEBiYiJ69+4NbW1tTJkyRerSiIiIiCRR7pHyBw8e4OjRo5g1axZatWqF2rVro3Hjxhg/fjy+/vprpXbBwcEwNjaGnp4eWrdujcTERKW+tm/fDldXV2hoaMDS0hLh4eEoLCwU11+5cgUeHh7Q0NCAvb099u/fr7R9fn4+hg4dCjMzM2hoaKB27dqYMWNGqbW/mIoRHh4u1jVw4EDk5+eLbUqaztGgQQOEhYWJz2UyGZYuXYqvv/4a2tramDZtGgDgzz//RKNGjaChoYFq1aqhQ4cOSv08efIEffr0ga6uLmrVqoVffvlFaf24ceNgY2MDLS0tWFpaIjQ0FAUFBeL6xMREtGrVCrq6utDT00PDhg1x5swZcX1cXBxatGgBTU1NmJubY9iwYXj8+LG4/vbt2/Dz84Ompibq1KmD9evXl3quXlXaOVuzZg2MjIyKjXD7+/sjICCgzD4NDAxgamoKc3NztGvXDu3bt8e5c+eU2ixduhR169aFuro6bG1tsXbtWnHdmDFj0K5dO/H5/PnzIZPJsGfPHnGZlZUVVq5cCeD/rv/06dNhYmICAwMDTJ48GYWFhRg7diwMDQ1Rs2ZNREZGKtXwuusSFhaGBg0aYO3atbCwsIC+vj66deuGR48elefUEhEREYnKHcp1dHSgo6OD6OjoMqcafPPNN7h9+zZ2796Ns2fPwtXVFV5eXrh37x4A4OjRo+jVqxeGDx+OpKQkLF++HFFRUWLAVSgU6NixI9TV1XHy5EksW7YM48aNU9rHwoULsWPHDvz+++9ISUnB+vXrYWFhUWb9Bw8eRHJyMmJjY7FhwwZs3boV4eHh5T18UVhYGDp06IALFy6gT58+2LVrFzp06IAvv/wS58+fx8GDB9G4cWOlbSIiIuDm5obz589j8ODBGDRoEFJSUsT1urq6iIqKQlJSEhYsWIAVK1Zg3rx54vqePXuiZs2aOH36NM6ePYvvv/9eHKFPT0+Hr68vOnXqhL///hubNm1CXFwchg4dKm4fFBSE69ev4/Dhw/jjjz+wZMkS3L59+7XHWtY5++abb1BUVIQdO3aI7W/fvo1du3ahT58+5T6fqampOHToEJo0aSIu27ZtG4YPH47Ro0fj4sWLGDBgAHr37o3Dhw8DADw9PREXF4eioiIAwJEjR1CtWjXExsYCAP73v/8hPT0dLVu2FPs8dOgQbt68ib/++gtz587FpEmT0K5dO1StWhUnT57EwIEDMWDAANy4cUPc5nXXBXh+/qOjo7Fz507s3LkTR44cwcyZM8t9/EREREQAIBMEQShv4y1btqBfv354+vQpXF1d4enpiW7dusHJyQnA8xHbr776Crdv34ZcLhe3s7KyQkhICPr37w9vb294eXlh/Pjx4vp169YhJCQEN2/exL59+/DVV1/h2rVrqFGjBgBgz549aNu2LbZt2wZ/f38MGzYMly5dwoEDByCTyV5bd1BQEP78809cv34dWlpaAIBly5Zh7NixePjwIVRUVGBhYYERI0ZgxIgR4nYNGjSAv7+/OFouk8kwYsQIpWDWrFkzWFpaYt26dSXu28LCAi1atBBHegVBgKmpKcLDwzFw4MASt5kzZw42btwojobr6elh0aJFCAwMLNY2ODgYqqqqWL58ubgsLi4Onp6eePz4MbKysmBra4tTp06hUaNGAIDLly/Dzs4O8+bNUzreip6zwYMHIzMzEzExMQCAuXPn4ueff0ZaWlqp10Umk0FDQwOqqqooLCxEXl4e2rVrh61bt4ofNJo3bw4HBwel3yh06dIFjx8/xq5du/DgwQMYGRnh5MmTaNiwIapVq4axY8ciOjoaJ06cwPr16zFu3DgxYAcFBSE2NhZXr16Fisrzz6H16tVD9erV8ddffwEAioqKoK+vj5UrV6Jbt27lui5hYWH46aefcOvWLejq6gIAQkJC8Ndff+HEiRMl9pGXl6f0oTYnJwfm5ubY0XAitNU0StyGiKg8Wp8Y//pGRFQpcnJyoK+vj4cPH0JPT69S+qzQFz07deqEmzdvYseOHfD19UVsbCxcXV0RFRUF4Pk0i9zcXBgZGYkj6zo6OsjIyEB6errYZvLkyUrr+/Xrh+zsbDx58gTJyckwNzcXAzkAuLu7K9URFBSEhIQE2NraYtiwYdi3b99ra3d2dhbD5Ys+c3Nzcf369YqcAri5uSk9T0hIgJeXV5nbvPjQAjwPpaampkoj1Zs2bULz5s1hamoKHR0d/Pjjj8jKyhLXjxo1CsHBwfD29sbMmTPFcwk8P59RUVFK59PHxwcKhQIZGRlITk6GmpoaGjZsKG5Tr149GBgYvPZYX3fO+vXrh3379uF///sfACAqKgpBQUGv/aA0b948JCQkIDExETt37kRqaqrSlJfk5GQ0b95caZvmzZsjOTkZwPPpL87OzoiNjcWFCxegrq6O/v374/z588jNzcWRI0fg6emptL2Dg4MYyAHAxMQEjo6O4nNVVVUYGRlV6LoAzz90vQjkAGBmZlbmbyFmzJgBfX198WFubl7muSIiIqJPQ4VviaihoYE2bdogNDQUx44dQ1BQECZNmgQAyM3NhZmZGRISEpQeKSkpGDt2rNgmPDxcaf2FCxdw5coVaGiUb6TQ1dUVGRkZmDJlCp4+fYouXbqgc+fOFT0UJSoqKnj1lwYvzx9+QVtbW+m5pqbma/t+9cugMpkMCoUCAHD8+HH07NkTX375JXbu3Inz589jwoQJSvPdw8LCcOnSJXz11Vc4dOgQ7O3tsW3bNgDPz+eAAQOUzmdiYiKuXLmCunXrlu/g35CLiwucnZ2xZs0anD17FpcuXUJQUNBrtzM1NYWVlRVsbW3x1VdfITw8HJs2bUJaWlq5992yZUvExsaKAdzQ0BB2dnaIi4srMZSXdA3+63Uprd8XfZRk/PjxePjwofio6IdCIiIi+jhV6O4rJbG3txfvH+7q6opbt25BTU2t1Dnerq6uSElJgZWVVYnr7ezscP36dWRnZ8PMzAwASpwKoKenh65du6Jr167o3LkzfH19ce/ePRgaGpbYb2JiIp4+fSqG6BMnTkBHR0ccqTQ2NkZ2drbYPicnBxkZGa89ficnJxw8eBC9e/d+bduSHDt2DLVr18aECRPEZdeuXSvWzsbGBjY2Nhg5ciS6d++OyMhIdOjQAa6urkhKSir1fNarVw+FhYU4e/asOH0lJSUFDx48eG1trztnwPPpM/Pnz8f//vc/eHt7v9HIr6qqKgDg6dOnAJ6/BuLj45Wm68THx8Pe3l587unpiV9//RVqamrw9fUF8Dyob9iwAampqUrzyd9Eea9LRcnlcqWpXURERERABUL53bt38c0336BPnz5wcnKCrq4uzpw5g9mzZ6N9+/YAAG9vb7i7u8Pf3x+zZ8+GjY0Nbt68KX4Z0s3NDRMnTkS7du1Qq1YtdO7cGSoqKkhMTMTFixcxdepUeHt7w8bGBoGBgfjpp5+Qk5OjFIyA53OXzczM4OLiAhUVFWzevBmmpqZlTsnIz89H37598eOPPyIzMxOTJk3C0KFDxSkNrVu3RlRUFPz8/GBgYICJEyeKYbEskyZNgpeXF+rWrYtu3bqhsLAQMTExxb6cWhpra2tkZWVh48aNaNSoEXbt2iWOggPPg+rYsWPRuXNn1KlTBzdu3MDp06fRqVMnAM/vENK0aVMMHToUwcHB0NbWRlJSEvbv34/FixfD1tYWvr6+GDBgAJYuXQo1NTWMGDGiXCP8rztnANCjRw+MGTMGK1aswJo1a8p1zA8ePMCtW7egUChw5coVTJ48GTY2NrCzswMAjB07Fl26dIGLiwu8vb3x559/YuvWrThw4IDYh4eHBx49eoSdO3eKX6xs2bIlOnfuDDMzM9jY2JSrltK87roQERERVaYK3X2lSZMmmDdvHjw8PFC/fn2EhoaiX79+WLx4MYDnv7qPiYmBh4cHevfuDRsbG3Tr1g3Xrl2DiYkJAMDHxwc7d+7Evn370KhRIzRt2hTz5s1D7dq1nxekooJt27bh6dOnaNy4MYKDg8U7s7ygq6uL2bNnw83NDY0aNRK/bPhyWHyVl5cXrK2t4eHhga5du+Lrr79Wut3h+PHj4enpiXbt2uGrr76Cv79/uaZ/tGzZEps3b8aOHTvQoEEDtG7dGqdOnSrvacXXX3+NkSNHYujQoWjQoAGOHTuG0NBQcb2qqiru3r2LXr16wcbGBl26dEHbtm3Fu6A4OTnhyJEjSE1NRYsWLeDi4oKJEycqzcmPjIxEjRo14OnpiY4dO6J///6oXr36a2t73TkDAH19fXTq1Ak6Ojrl/gugvXv3hpmZGWrWrInu3bvDwcEBu3fvhpra88+I/v7+WLBgAebMmQMHBwcsX74ckZGRSqPfVatWhaOjI4yNjVGvXj0Az4O6QqEoNnXlTbzuuhARERFVpgrdfeVDFRQUhAcPHojTbKhyeXl5wcHBAQsXLpS6lA/Oi29v8+4rRPRf8e4rRO/O27j7yn+eU06frvv37yM2NhaxsbFYsmSJ1OUQERERfbAYyumNubi44P79+5g1axZsbW2lLoeIiIjog/VJhPIX91GnypWZmSl1CUREREQfhQrfp5yIiIiIiCoXQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHEGMqJiIiIiCTGUE5EREREJDGGciIiIiIiiTGUExERERFJTE3qAogI8Dw0Gnp6elKXQURERBLhSDkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHEGMqJiIiIiCSmJnUBRAQozrpCoaMqdRmVTqVRitQlEBERfRA4Uk5EREREJDGGciIiIiIiiTGUExERERFJjKGciIiIiEhiDOVERERERBJjKCciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQksY8+lMfGxkImk+HBgwdSl/LR4DklIiIiqlzvdSj/999/MWjQINSqVQtyuRympqbw8fFBfHy81KW9tywsLDB//nypyyAiIiKiClCTuoCydOrUCfn5+Vi9ejUsLS3xzz//4ODBg7h7967UpdF7Jj8/H+rq6lKXQURERPRG3tuR8gcPHuDo0aOYNWsWWrVqhdq1a6Nx48YYP348vv76awBAZmYmZDIZEhISlLaTyWSIjY1V6i8+Ph5OTk7Q0NBA06ZNcfHiRXHdtWvX4Ofnh6pVq0JbWxsODg6IiYkR11+8eBFt27aFjo4OTExMEBAQgDt37ojrHz9+jF69ekFHRwdmZmaIiIhAy5YtMWLECLGNTCZDdHS0Uk0GBgaIiooSn1+/fh1dunSBgYEBDA0N0b59e2RmZorrg4KC4O/vjzlz5sDMzAxGRkYYMmQICgoKAAAtW7bEtWvXMHLkSMhkMshkshLPrSAICAsLE38DUaNGDQwbNkxcv3btWri5uUFXVxempqbo0aMHbt++XWJfL8TFxaFFixbQ1NSEubk5hg0bhsePH4vrlyxZAmtra2hoaMDExASdO3cus78tW7bAwcEBcrkcFhYWiIiIUFpvYWGBKVOmoFevXtDT00P//v1L7Kek3xw0aNAAYWFh5ToXeXl5GDNmDD777DNoa2ujSZMmSq+t1712iIiIiMrjvQ3lOjo60NHRQXR0NPLy8v5zf2PHjkVERAROnz4NY2Nj+Pn5iWF2yJAhyMvLw19//YULFy5g1qxZ0NHRAfA85Ldu3RouLi44c+YM9uzZg3/++QddunRR6vvIkSPYvn079u3bh9jYWJw7d65C9RUUFMDHxwe6uro4evQo4uPjoaOjA19fX+Tn54vtDh8+jPT0dBw+fBirV69GVFSUGOy3bt2KmjVrYvLkycjOzkZ2dnaJ+9qyZQvmzZuH5cuX48qVK4iOjoajo6NSLVOmTEFiYiKio6ORmZmJoKCgUmtPT0+Hr68vOnXqhL///hubNm1CXFwchg4dCgA4c+YMhg0bhsmTJyMlJQV79uyBh4dHqf2dPXsWXbp0Qbdu3XDhwgWEhYUhNDRU6QMMAMyZMwfOzs44f/48QkNDX3OGS/a6czF06FAcP34cGzduxN9//41vvvkGvr6+uHLlCoCyXztERERE5fXeTl9RU1NDVFQU+vXrh2XLlsHV1RWenp7o1q0bnJycKtzfpEmT0KZNGwDA6tWrUbNmTWzbtg1dunRBVlYWOnXqJIYxS0tLcbvFixfDxcUF06dPF5f9+uuvMDc3R2pqKmrUqIFVq1Zh3bp18PLyUuq/IjZt2gSFQoGVK1eKI9yRkZEwMDBAbGwsvvjiCwBA1apVsXjxYqiqqqJevXr46quvcPDgQfTr1w+GhoZQVVUVR7hLk5WVBVNTU3h7e6NKlSqoVasWGjduLK7v06eP+G9LS0ssXLgQjRo1Qm5ubomBc8aMGejZs6f4mwFra2ssXLgQnp6eWLp0KbKysqCtrY127dpBV1cXtWvXhouLS6n1zZ07F15eXmLQtrGxQVJSEn766SelDwetW7fG6NGjX39yy1DWucjKykJkZCSysrJQo0YNAMCYMWOwZ88eREZGYvr06WW+dkqSl5en9CEzJyfnP9VPREREH4f3dqQceD6n/ObNm9ixYwd8fX0RGxsLV1fXYiOm5eHu7i7+29DQELa2tkhOTgYADBs2DFOnTkXz5s0xadIk/P3332LbxMREHD58WBy519HRQb169QA8HyFOT09Hfn4+mjRpUqz/ikhMTERaWhp0dXXF/RgaGuLZs2dIT08X2zk4OEBVVVV8bmZm9tqpJa/65ptv8PTpU1haWqJfv37Ytm0bCgsLxfVnz56Fn58fatWqBV1dXXh6egJ4HlJLqz0qKkrpHPn4+EChUCAjIwNt2rRB7dq1YWlpiYCAAKxfvx5Pnjwptb7k5GQ0b95caVnz5s1x5coVFBUVicvc3NwqdNwlKetcXLhwAUVFRbCxsVE6tiNHjojXpKzXTklmzJgBfX198WFubv6fj4GIiIg+fO91KAcADQ0NtGnTBqGhoTh27BiCgoIwadIkAICKyvPyBUEQ27+YklIRwcHBuHr1KgICAnDhwgW4ublh0aJFAIDc3Fz4+fkhISFB6XHlypUyp2C8SiaTKdX5aq25ublo2LBhsf2kpqaiR48eYrsqVaoU61ehUFToeM3NzZGSkoIlS5ZAU1MTgwcPhoeHBwoKCvD48WP4+PhAT08P69evx+nTp7Ft2zYAUJpG87Lc3FwMGDBAqe7ExERcuXIFdevWha6uLs6dO4cNGzbAzMwMEydOhLOz83++paK2tvZr26ioqJR53ss6F7m5uVBVVcXZs2eVji05ORkLFiwAUPZrpyTjx4/Hw4cPxcf169ff8OiJiIjoY/Leh/JX2dvbi18gNDY2BgCludMvf+nzZSdOnBD/ff/+faSmpsLOzk5cZm5ujoEDB2Lr1q0YPXo0VqxYAQBwdXXFpUuXYGFhASsrK6WHtrY26tatiypVquDkyZPF+n+ZsbGxUp1XrlxRGi12dXXFlStXUL169WL70dfXL/f5UVdXVxpNLo2mpib8/PywcOFCxMbG4vjx47hw4QIuX76Mu3fvYubMmWjRogXq1av32pF4V1dXJCUlFavbyspKvCOKmpoavL29MXv2bPz999/IzMzEoUOHSuzPzs6u2G0v4+PjYWNjo/RbgvJ49bzn5OQgIyOjXOfCxcUFRUVFuH37drHjenl6UGmvnZLI5XLo6ekpPYiIiIje2znld+/exTfffIM+ffrAyckJurq6OHPmDGbPno327dsDeB6mmjZtipkzZ6JOnTq4ffs2fvzxxxL7mzx5MoyMjGBiYoIJEyagWrVq8Pf3BwCMGDECbdu2hY2NDe7fv4/Dhw+LgX3IkCFYsWIFunfvjpCQEBgaGiItLQ0bN27EypUroaOjg759+2Ls2LEwMjJC9erVMWHCBHEU/4XWrVtj8eLFcHd3R1FREcaNG6c06t2zZ0/89NNPaN++PSZPnoyaNWvi2rVr2Lp1K0JCQso9R93CwgJ//fUXunXrBrlcjmrVqhVrExUVhaKiIjRp0gRaWlpYt24dNDU1Ubt2bSgUCqirq2PRokUYOHAgLl68iClTppS5z3HjxqFp06YYOnQogoODoa2tjaSkJOzfvx+LFy/Gzp07cfXqVXh4eKBq1aqIiYmBQqEodYrP6NGj0ahRI0yZMgVdu3bF8ePHsXjxYixZsqRc5+BlrVu3RlRUFPz8/GBgYICJEycqBfuyzoWRkRF69uyJXr16ISIiAi4uLvj3339x8OBBODk54auvvirztUNERERUXu/tSLmOjg6aNGmCefPmwcPDA/Xr10doaCj69euHxYsXi+1+/fVXFBYWomHDhhgxYgSmTp1aYn8zZ87E8OHD0bBhQ9y6dQt//vmnOIpbVFSEIUOGwM7ODr6+vrCxsREDYI0aNRAfH4+ioiJ88cUXcHR0xIgRI2BgYCAG759++gktWrSAn58fvL298fnnn6Nhw4ZK+4+IiIC5uTlatGiBHj16YMyYMdDS0hLXa2lp4a+//kKtWrXQsWNH2NnZoW/fvnj27FmFRlMnT56MzMxM1K1bV/xNwqsMDAywYsUKNG/eHE5OTjhw4AD+/PNPGBkZwdjYGFFRUdi8eTPs7e0xc+ZMzJkzp8x9Ojk54ciRI0hNTUWLFi3g4uKCiRMnil+ONDAwwNatW9G6dWvY2dlh2bJl2LBhAxwcHErsz9XVFb///js2btyI+vXrY+LEiZg8eXKZd4Apzfjx4+Hp6Yl27drhq6++gr+/P+rWrVuucwE8/7Jtr169MHr0aNja2sLf3x+nT59GrVq1AJT92iEiIiIqL5nw6oRbqhQtW7ZEgwYN+Nc1qUw5OTnQ19fH/UN1oadTsak5HwKVRilSl0BERFTpXvz8fvjwYaVNRX1vR8qJiIiIiD4VDOVERERERBJ7b7/o+aF7+U+xExERERGVhSPlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHEGMqJiIiIiCTGUE5EREREJDGGciIiIiIiialJXQARASoNz0FFT0/qMoiIiEgiHCknIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREEmMoJyIiIiKSGEM5EREREZHE1KQugIiAK73HQKeKutRlEH30bDculroEIqIScaSciIiIiEhiDOVERERERBJjKCciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5e+Zli1bYsSIEVKX8VpRUVEwMDCo9H6DgoLg7+9f6f2Wx9s6JiIiIqLXYSiXQFBQEGQyWbFHWloatm7diilTpvyn/mUyGaKjoyun2A9cRUJ+165dkZqa+nYLIiIiIiqBmtQFfKp8fX0RGRmptMzY2Biqqqplbpefnw91dfW3WdonqaCgAJqamtDU1JS6FCIiIvoEcaRcInK5HKampkoPVVXVYtNXLCwsMGXKFPTq1Qt6enro378/8vPzMXToUJiZmUFDQwO1a9fGjBkzxPYA0KFDB8hkMvH5q8rqAwDmzp0LR0dHaGtrw9zcHIMHD0Zubm6Zx7R9+3a4urpCQ0MDlpaWCA8PR2FhYanti4qKMGrUKBgYGMDIyAghISEQBEGpTV5eHoYNG4bq1atDQ0MDn3/+OU6fPq3U5tKlS2jXrh309PSgq6uLFi1aID09HWFhYVi9ejW2b98u/jYiNjYWmZmZkMlk2LRpEzw9PaGhoYH169cXm74SFhaGBg0aYO3atbCwsIC+vj66deuGR48eiW0UCgVmzJiBOnXqQFNTE87Ozvjjjz/KPE9EREREr2Io/wDMmTMHzs7OOH/+PEJDQ7Fw4ULs2LEDv//+O1JSUrB+/XoxfL8IrJGRkcjOzi4WYF8oqw8AUFFRwcKFC3Hp0iWsXr0ahw4dQkhISKk1Hj16FL169cLw4cORlJSE5cuXIyoqCtOmTSt1m4iICERFReHXX39FXFwc7t27h23btim1CQkJwZYtW7B69WqcO3cOVlZW8PHxwb179wAA//vf/+Dh4QG5XI5Dhw7h7Nmz6NOnDwoLCzFmzBh06dIFvr6+yM7ORnZ2Npo1ayb2/f3332P48OFITk6Gj49PiTWmp6cjOjoaO3fuxM6dO3HkyBHMnDlTXD9jxgysWbMGy5Ytw6VLlzBy5Eh8++23OHLkSKnHTURERPQqTl+RyM6dO6GjoyM+b9u2LTZv3lxi29atW2P06NHi86ysLFhbW+Pzzz+HTCZD7dq1xXXGxsYAAAMDA5iampa6/7L6AFBstH7q1KkYOHAglixZUmJ/4eHh+P777xEYGAgAsLS0xJQpUxASEoJJkyaVuM38+fMxfvx4dOzYEQCwbNky7N27V1z/+PFjLF26FFFRUWjbti0AYMWKFdi/fz9WrVqFsWPH4ueff4a+vj42btyIKlWqAABsbGzEPjQ1NZGXl1fiuRgxYoS479IoFApERUVBV1cXABAQEICDBw9i2rRpyMvLw/Tp03HgwAG4u7uLxx0XF4fly5fD09OzWH95eXnIy8sTn+fk5JS5fyIiIvo0MJRLpFWrVli6dKn4XFtbu9S2bm5uSs+DgoLQpk0b2NrawtfXF+3atcMXX3xRof2/ro8DBw5gxowZuHz5MnJyclBYWIhnz57hyZMn0NLSKtZfYmIi4uPjlUbGi4qKSt3m4cOHyM7ORpMmTcRlampqcHNzE6ewpKeno6CgAM2bNxfbVKlSBY0bN0ZycjIAICEhAS1atBADeUW8el5LYmFhIQZyADAzM8Pt27cBAGlpaXjy5AnatGmjtE1+fj5cXFxK7G/GjBkIDw+vcK1ERET0cWMol4i2tjasrKzK3fZlrq6uyMjIwO7du3HgwAF06dIF3t7eFZrLXFYfmZmZaNeuHQYNGoRp06bB0NAQcXFx6Nu3L/Lz80sM5bm5uQgPDy9x5FlDQ6PcdVXUf/liZlkfhF54NezLZDIoFAoAEOfY79q1C5999plSO7lcXmJ/48ePx6hRo8TnOTk5MDc3r1DdRERE9PFhKP9A6enpoWvXrujatSs6d+4MX19f3Lt3D4aGhqhSpQqKioreuI+zZ89CoVAgIiICKirPv3bw+++/l9mXq6srUlJSyv1BQ19fH2ZmZjh58iQ8PDwAAIWFhTh79ixcXV0BAHXr1oW6ujri4+PF6TUFBQU4ffq0OL3GyckJq1evRkFBQYmj5erq6uU6F2/C3t4ecrkcWVlZJU5VKYlcLi81sBMREdGni6H8AzR37lyYmZnBxcUFKioq2Lx5M0xNTcU7h1hYWODgwYNo3rw55HI5qlatWqE+rKysUFBQgEWLFsHPzw/x8fFYtmxZmTVNnDgR7dq1Q61atdC5c2eoqKggMTERFy9exNSpU0vcZvjw4Zg5cyasra1Rr149zJ07Fw8ePBDXa2trY9CgQRg7diwMDQ1Rq1YtzJ49G0+ePEHfvn0BAEOHDsWiRYvQrVs3jB8/Hvr6+jhx4gQaN24MW1tbWFhYYO/evUhJSYGRkRH09fXf7KSXQFdXF2PGjMHIkSOhUCjw+eef4+HDh4iPj4eenp44v56IiIjodXj3lQ+Qrq4uZs+eDTc3NzRq1AiZmZmIiYkRR7UjIiKwf/9+mJublzq3uaw+nJ2dMXfuXMyaNQv169fH+vXrlW6XWBIfHx/s3LkT+/btQ6NGjdC0aVPMmzev2BdIXzZ69GgEBAQgMDAQ7u7u0NXVRYcOHZTazJw5E506dUJAQABcXV2RlpaGvXv3ih80jIyMcOjQIeTm5sLT0xMNGzbEihUrxFHzfv36wdbWFm5ubjA2NkZ8fHy5z3N5TJkyBaGhoZgxYwbs7Ozg6+uLXbt2oU6dOpW6HyIiIvq4yYRXbwxNRO9MTk4O9PX1caZjP+hU4R+FInrbbDculroEIvoIvPj5/fDhQ+jp6VVKnxwpJyIiIiKSGEM5EREREZHEGMqJiIiIiCTGUE5EREREJDGGciIiIiIiiTGUExERERFJjKGciIiIiEhiDOVERERERBJjKCciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpKYmtQFEBFgHTkHenp6UpdBREREEuFIORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIokxlBMRERERSYyhnIiIiIhIYgzlREREREQSYygnIiIiIpIYQzkRERERkcQYyomIiIiIJMZQTkREREQkMYZyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5UREREREElOTugCiT5kgCACAnJwciSshIiKi8nrxc/vFz/HKwFBOJKG7d+8CAMzNzSWuhIiIiCrq7t270NfXr5S+GMqJJGRoaAgAyMrKqrQ3Nb2ZnJwcmJub4/r169DT05O6nE8ar8X7g9fi/cLr8f54+PAhatWqJf4crwwM5UQSUlF5/rUOfX19/gf7ntDT0+O1eE/wWrw/eC3eL7we748XP8crpa9K64mIiIiIiN4IQzkRERERkcQYyokkJJfLMWnSJMjlcqlL+eTxWrw/eC3eH7wW7xdej/fH27gWMqEy7+VCREREREQVxpFyIiIiIiKJMZQTEREREUmMoZyIiIiISGIM5URv2c8//wwLCwtoaGigSZMmOHXqVJntN2/ejHr16kFDQwOOjo6IiYl5R5V+/CpyLVasWIEWLVqgatWqqFq1Kry9vV977aj8Kvq+eGHjxo2QyWTw9/d/uwV+Qip6LR48eIAhQ4bAzMwMcrkcNjY2/H+qklT0WsyfPx+2trbQ1NSEubk5Ro4ciWfPnr2jaj9ef/31F/z8/FCjRg3IZDJER0e/dpvY2Fi4urpCLpfDysoKUVFRFd+xQERvzcaNGwV1dXXh119/FS5duiT069dPMDAwEP75558S28fHxwuqqqrC7NmzhaSkJOHHH38UqlSpIly4cOEdV/7xqei16NGjh/Dzzz8L58+fF5KTk4WgoCBBX19fuHHjxjuu/ONT0WvxQkZGhvDZZ58JLVq0ENq3b/9uiv3IVfRa5OXlCW5ubsKXX34pxMXFCRkZGUJsbKyQkJDwjiv/+FT0Wqxfv16Qy+XC+vXrhYyMDGHv3r2CmZmZMHLkyHdc+ccnJiZGmDBhgrB161YBgLBt27Yy21+9elXQ0tISRo0aJSQlJQmLFi0SVFVVhT179lRovwzlRG9R48aNhSFDhojPi4qKhBo1aggzZswosX2XLl2Er776SmlZkyZNhAEDBrzVOj8FFb0WryosLBR0dXWF1atXv60SPxlvci0KCwuFZs2aCStXrhQCAwMZyitJRa/F0qVLBUtLSyE/P/9dlfjJqOi1GDJkiNC6dWulZaNGjRKaN2/+Vuv81JQnlIeEhAgODg5Ky7p27Sr4+PhUaF+cvkL0luTn5+Ps2bPw9vYWl6moqMDb2xvHjx8vcZvjx48rtQcAHx+fUttT+bzJtXjVkydPUFBQAENDw7dV5ifhTa/F5MmTUb16dfTt2/ddlPlJeJNrsWPHDri7u2PIkCEwMTFB/fr1MX36dBQVFb2rsj9Kb3ItmjVrhrNnz4pTXK5evYqYmBh8+eWX76Rm+j+V9bNbrTKLIqL/c+fOHRQVFcHExERpuYmJCS5fvlziNrdu3Sqx/a1bt95anZ+CN7kWrxo3bhxq1KhR7D9eqpg3uRZxcXFYtWoVEhIS3kGFn443uRZXr17FoUOH0LNnT8TExCAtLQ2DBw9GQUEBJk2a9C7K/ii9ybXo0aMH7ty5g88//xyCIKCwsBADBw7EDz/88C5KppeU9rM7JycHT58+haamZrn64Ug5EdFrzJw5Exs3bsS2bdugoaEhdTmflEePHiEgIAArVqxAtWrVpC7nk6dQKFC9enX88ssvaNiwIbp27YoJEyZg2bJlUpf2yYmNjcX06dOxZMkSnDt3Dlu3bsWuXbswZcoUqUujN8SRcqK3pFq1alBVVcU///yjtPyff/6BqalpiduYmppWqD2Vz5tcixfmzJmDmTNn4sCBA3BycnqbZX4SKnot0tPTkZmZCT8/P3GZQqEAAKipqSElJQV169Z9u0V/pN7kfWFmZoYqVapAVVVVXGZnZ4dbt24hPz8f6urqb7Xmj9WbXIvQ0FAEBAQgODgYAODo6IjHjx+jf//+mDBhAlRUOO76rpT2s1tPT6/co+QAR8qJ3hp1dXU0bNgQBw8eFJcpFAocPHgQ7u7uJW7j7u6u1B4A9u/fX2p7Kp83uRYAMHv2bEyZMgV79uyBm5vbuyj1o1fRa1GvXj1cuHABCQkJ4uPrr79Gq1atkJCQAHNz83dZ/kflTd4XzZs3R1pamvjBCABSU1NhZmbGQP4fvMm1ePLkSbHg/eLD0vPvJ9K7Umk/uyv2HVQiqoiNGzcKcrlciIqKEpKSkoT+/fsLBgYGwq1btwRBEISAgADh+++/F9vHx8cLampqwpw5c4Tk5GRh0qRJvCViJanotZg5c6agrq4u/PHHH0J2drb4ePTokVSH8NGo6LV4Fe++Unkqei2ysrIEXV1dYejQoUJKSoqwc+dOoXr16sLUqVOlOoSPRkWvxaRJkwRdXV1hw4YNwtWrV4V9+/YJdevWFbp06SLVIXw0Hj16JJw/f144f/68AECYO3eucP78eeHatWuCIAjC999/LwQEBIjtX9wScezYsUJycrLw888/85aIRO+jRYsWCbVq1RLU1dWFxo0bCydOnBDXeXp6CoGBgUrtf//9d8HGxkZQV1cXHBwchF27dr3jij9eFbkWtWvXFgAUe0yaNOndF/4Rquj74mUM5ZWrotfi2LFjQpMmTQS5XC5YWloK06ZNEwoLC99x1R+nilyLgoICISwsTKhbt66goaEhmJubC4MHDxbu37//7gv/yBw+fLjE//9fnP/AwEDB09Oz2DYNGjQQ1NXVBUtLSyEyMrLC+5UJAn/HQUREREQkJc4pJyIiIiKSGEM5EREREZHEGMqJiIiIiCTGUE5EREREJDGGciIiIiIiiTGUExERERFJjKGciIiIiEhiDOVERERERBJjKCciInpDFhYWmD9/vtRlENFHgKGciIgkFxQUBJlMVuyRlpb2n/tu2bIlRowY8d+L/A9u3LgBdXV11K9fX9I6iOj9xVBORETvBV9fX2RnZys96tSpI3VZlSIqKgpdunRBTk4OTp48KWktRUVFUCgUktZARMUxlBMR0XtBLpfD1NRU6aGqqoq5c+fC0dER2traMDc3x+DBg5Gbm6u0bXx8PFq2bAktLS1UrVoVPj4+uH//PoKCgnDkyBEsWLBAHH3PzMxEVFQUDAwMlPqIjo6GTCYTn6enp6N9+/YwMTGBjo4OGjVqhAMHDlT4uARBQGRkJAICAtCjRw+sWrWqWJvS6gcAhUKB2bNnw8rKCnK5HLVq1cK0adMAALGxsZDJZHjw4IHYV0JCgnicAMRj3bFjB+zt7SGXy5GVlYXTp0+jTZs2qFatGvT19eHp6Ylz586J/fTp0wft2rVTqrOgoADVq1cv8RiI6L9hKCcioveaiooKFi5ciEuXLmH16tU4dOgQQkJCxPUJCQnw8vKCvb09jh8/jri4OPj5+aGoqAgLFiyAu7s7+vXrJ46+m5ubl2u/ubm5+PLLL3Hw4EGcP38evr6+8PPzQ1ZWVoXqP3z4MJ48eQJvb298++232LhxIx4/flyu+gFg/PjxmDlzJkJDQ5GUlITffvsNJiYmFarhyZMnmDVrFlauXIlLly6hevXqePToEQIDAxEXF4cTJ07A2toaX375JR49egQACA4Oxp49e5CdnS32s3PnTjx58gRdu3at0P6JqBwEIiIiiQUGBgqqqqqCtra2+OjcuXOJbTdv3iwYGRmJz7t37y40b9681L49PT2F4cOHKy2LjIwU9PX1lZZt27ZNeN2PRQcHB2HRokXi89q1awvz5s0rc5sePXoII0aMEJ87OzsLkZGR5ao/JydHkMvlwooVK0pcf/jwYQGAcP/+fXHZ+fPnBQBCRkaGIAjPjxWAkJCQUGadRUVFgq6urvDnn3+Ky+zt7YVZs2aJz/38/ISgoKAy+yGiN8ORciIiei+0atUKCQkJ4mPhwoUAgAMHDsDLywufffYZdHV1ERAQgLt37+LJkycA/m+kubLl5uZizJgxsLOzg4GBAXR0dJCcnFyhkfIHDx5g69at+Pbbb8Vl3377rdL0j7LqT05ORl5e3n8+PnV1dTg5OSkt++eff9CvXz9YW1tDX18fenp6yM3NVTq+4OBgREZGiu13796NPn36/KdaiKhkalIXQEREBADa2tqwsrJSWpaZmYl27dph0KBBmDZtGgwNDREXF4e+ffsiPz8fWlpa0NTUrPC+VFRUIAiC0rKCggKl52PGjMH+/fsxZ84cWFlZQVNTE507d0Z+fn659/Pbb7/h2bNnaNKkibhMEAQoFAqkpqbCxsamzPpfd2wqKipin6Udx4t+Xp4vDwCBgYG4e/cuFixYgNq1a0Mul8Pd3V3p+Hr16oXvv/8ex48fx7Fjx1CnTh20aNGi7IMmojfCkXIiInpvnT17FgqFAhEREWjatClsbGxw8+ZNpTZOTk44ePBgqX2oq6uL87NfMDY2xqNHj4rN7X5ZfHw8goKC0KFDBzg6OsLU1FT88mR5rVq1CqNHj1b6DUBiYiJatGiBX3/99bX1W1tbQ1NTs9T1xsbGAKA07/vV4yhNfHw8hg0bhi+//BIODg6Qy+W4c+eOUhsjIyP4+/sjMjISUVFR6N27d7n6JqKKYygnIqL3lpWVFQoKCrBo0SJcvXoVa9euxbJly5TajB8/HqdPn8bgwYPx999/4/Lly1i6dKkYMC0sLHDy5ElkZmbizp07UCgUaNKkCbS0tPDDDz8gPT0dv/32G6KiopT6tba2xtatW8Ug3aNHjwrdSjAhIQHnzp1DcHAw6tevr/To3r07Vq9ejcLCwjLr19DQwLhx4xASEoI1a9YgPT0dJ06cEKe/WFlZwdzcHGFhYbhy5Qp27dqFiIiIctVnbW2NtWvXIjk5GSdPnkTPnj1LHJkPDg7G6tWrkZycjMDAwHIfPxFVDEM5ERG9t5ydnTF37lzMmjUL9evXx/r16zFjxgylNjY2Nti3bx8SExPRuHFjuLu7Y/v27VBTez5Dc8yYMVBVVYW9vT2MjY2RlZUFQ0NDrFu3DjExMXB0dMSGDRsQFham1O/cuXNRtWpVNGvWDH5+fvDx8YGrq2u5a1+1ahXs7e1Rr169Yus6dOiA27dvIyYm5rX1h4aGYvTo0Zg4cSLs7OzQtWtX3L59GwBQpUoVbNiwAZcvX4aTkxNmzZqFqVOnlru++/fvw9XVFQEBARg2bBiqV69erJ23tzfMzMzg4+ODGjVqlPv4iahiZMKrk+qIiIiI/r/c3Fx89tlniIyMRMeOHaUuh+ijxS96EhERUTEKhQJ37txBREQEDAwM8PXXX0tdEtFHjaGciIiIisnKykKdOnVQs2ZNREVFidNpiOjt4PQVIiIiIiKJ8YueREREREQSYygnog9CQEAApk+f/k73eevWLbRp0wba2towMDAAAMhkMkRHR5e6TWZmJmQyWbnvFf2+ed3xvStvch6DgoLg7+9f6bVYWFhg/vz5b7RtWFgYGjRoUKn1lCQ2NhYymQwPHjwAAERFRYmv2f/ida+Hli1bYsSIEf95P5UlPj4ejo6OqFKlylt5LXxK8vPzYWFhgTNnzkhdyieDoZyI3nuJiYmIiYnBsGHD3ul+582bh+zsbCQkJCA1NRXA8z/S0rZt23dax9vwNsPi2wrHr7NgwYJi9xqvDKdPn0b//v3faNsxY8aU+YeN3pauXbuKr9lPyahRo9CgQQNkZGS8ldeC1N7W+7akftXV1TFmzBiMGzeu0vdHJWMoJ6L33qJFi/DNN99AR0fnne43PT0dDRs2hLW1tXj/ZlNTU8jl8ndaB5WPvr5+pYwOv8rY2BhaWlpvtK2Ojg6MjIwquaLX09TULPGe4x8iQRBQWFhYrrbp6elo3bo1atas+cavhfz8/Dfa7mPUs2dPxMXF4dKlS1KX8klgKCei91pRURH++OMP+Pn5KS3Py8vDuHHjYG5uDrlcDisrK/GvHALAkSNH0LhxY8jlcpiZmeH7779X+sHesmVLDBs2DCEhITA0NISpqanSH4+xsLDAli1bsGbNGshkMgQFBQEo/uv8U6dOwcXFBRoaGnBzc8P58+eLHcPFixfRtm1b6OjowMTEBAEBAUp/zvx1tQDAgwcPMGDAAJiYmEBDQwP169fHzp07xfVxcXFo0aIFNDU1YW5ujmHDhin9CfmXRUVFITw8HImJiZDJZJDJZEqjinfu3EGHDh2gpaUFa2tr7NixQ+l69O3bF3Xq1IGmpiZsbW2xYMECcX1YWBhWr16N7du3i33HxsaWWIdCocDs2bNhZWUFuVyOWrVqYdq0aSW2fd1+geIj9C1btsR3332HESNGoGrVqjAxMcGKFSvw+PFj9O7dG7q6urCyssLu3btL3OcLL09fEQQBYWFhqFWrFuRyOWrUqFHmb3BeHYF8UeP06dNhYmICAwMDTJ48GYWFhRg7diwMDQ1Rs2ZNREZGitu8mMqzceNGNGvWTLz+R44cKXW/JU1f2b59O1xdXaGhoQFLS0uEh4crvSeuXLkCDw8PaGhowN7eHvv37y/zvLxQWFiIoUOHQl9fH9WqVUNoaChe3ENi8uTJqF+/frFtGjRogNDQ0BL7ezEVZ/fu3WjYsCHkcjni4uKgUCgwY8YM8TXg7OyMP/74Q+kc3b17F3369FF6TZfn/Td06FCMGDEC1apVg4+PT7m3e1/etw8ePEBwcDCMjY2hp6eH1q1bIzExEQDw77//wtTUVGn637Fjx6Curo6DBw+W2W/VqlXRvHlzbNy4scSaqJIJRETvsXPnzgkAhFu3bikt79Kli2Bubi5s3bpVSE9PFw4cOCBs3LhREARBuHHjhqClpSUMHjxYSE5OFrZt2yZUq1ZNmDRpkri9p6enoKenJ4SFhQmpqanC6tWrBZlMJuzbt08QBEG4ffu24OvrK3Tp0kXIzs4WHjx4IAiCIAAQtm3bJgiCIDx69EgwNjYWevToIVy8eFH4888/BUtLSwGAcP78eUEQBOH+/fuCsbGxMH78eCE5OVk4d+6c0KZNG6FVq1blrqWoqEho2rSp4ODgIOzbt09IT08X/vzzTyEmJkYQBEFIS0sTtLW1hXnz5gmpqalCfHy84OLiIgQFBZV4Tp88eSKMHj1acHBwELKzs4Xs7GzhyZMn4vHVrFlT+O2334QrV64Iw4YNE3R0dIS7d+8KgiAI+fn5wsSJE4XTp08LV69eFdatWydoaWkJmzZtEs9Jly5dBF9fX7HvvLy8EusICQkRqlatKkRFRQlpaWnC0aNHhRUrVgiCIAgZGRlK5/F1+xUEQQgMDBTat2+vdF51dXWFKVOmCKmpqcKUKVMEVVVVoW3btsIvv/wipKamCoMGDRKMjIyEx48fl1ijIAhC7dq1hXnz5gmCIAibN28W9PT0hJiYGOHatWvCyZMnhV9++aXUbSdNmiQ4Ozsr1airqysMGTJEuHz5srBq1SoBgODj4yNMmzZNrLNKlSrC9evXlc5FzZo1hT/++ENISkoSgoODBV1dXeHOnTuCIAjC4cOHBQDC/fv3BUEQhMjISEFfX1/c719//SXo6ekJUVFRQnp6urBv3z7BwsJCCAsLEwTh+Wusfv36gpeXl5CQkCAcOXJEcHFxUXq9l8TT01PQ0dERhg8fLly+fFm8Li/OyfXr1wUVFRXh1KlT4jbnzp0TZDKZkJ6eXmKfL47FyclJ2Ldvn5CWlibcvXtXmDp1qlCvXj1hz549Qnp6uhAZGSnI5XIhNjZWKCwsFLKzswU9PT1h/vz54mu6vO8/HR0dYezYscLly5eFy5cvf3DvW29vb8HPz084ffq0kJqaKowePVowMjIS37e7du0SqlSpIpw+fVrIyckRLC0thZEjR762X0EQhHHjxgmenp6lvgao8jCUE9F7bdu2bYKqqqqgUCjEZSkpKQIAYf/+/SVu88MPPwi2trZK2/z888+Cjo6OUFRUJAjC8x+on3/+udJ2jRo1EsaNGyc+b9++vRAYGKjU5uWQsnz5csHIyEh4+vSpuH7p0qVKYXLKlCnCF198odTH9evXBQBCSkpKuWrZu3evoKKiIrZ/Vd++fYX+/fsrLTt69KigoqKiVNvLXg2LLx/fjz/+KD7Pzc0VAAi7d+8usR9BEIQhQ4YInTp1Ep+/Go5LkpOTI8jlcjGEv+rVUP4m+331vBYWFgra2tpCQECAuCw7O1sAIBw/frzU/bwcyiMiIgQbGxshPz+/zON7oaRQXrt2bfF1KAiCYGtrK7Ro0aJYnRs2bBAE4f/OxcyZM8U2BQUFQs2aNYVZs2YJgvD6UO7l5SVMnz5dqba1a9cKZmZmgiA8f42pqakJ//vf/8T1u3fvLlcot7OzU3qvjRs3TrCzsxOft23bVhg0aJD4/LvvvhNatmxZap8vjiU6Olpc9uzZM0FLS0s4duyYUtu+ffsK3bt3F5/r6+sLkZGR4vPyvv9cXFyU2nxI79ujR48Kenp6wrNnz5SW161bV1i+fLn4fPDgwYKNjY3Qo0cPwdHRUal9af8fCIIgLFiwQLCwsChxHVUu/iUAInqvPX36FHK5HDKZTFyWkJAAVVVVeHp6lrhNcnIy3N3dlbZp3rw5cnNzcePGDdSqVQsA4OTkpLSdmZkZbt++Xe7akpOT4eTkBA0NDXGZu7u7UpvExEQcPny4xPnw6enpsLGxeW0tCQkJqFmzptj2VYmJifj777+xfv16cZkgCFAoFMjIyICdnV25j+nVWrS1taGnp6d0Xn7++Wf8+uuvyMrKwtOnT5Gfn1/hL58lJycjLy8PXl5e5d7mTfb78rGoqqrCyMgIjo6O4jITExMAKPd1/+abbzB//nxYWlrC19cXX375Jfz8/Cr0h3UcHBygovJ/s0dNTEyUpni8qPPVml5+bampqcHNzQ3Jycnl2mdiYiLi4+OVpgcVFRXh2bNnePLkCZKTk2Fubo4aNWqUuL+yNG3aVOm95u7ujoiICBQVFUFVVRX9+vVDnz59MHfuXKioqOC3337DvHnzXtuvm5ub+O+0tDQ8efIEbdq0UWqTn58PFxeXMo+7PO+/hg0bvtF278P7NjExEbm5ucW+u/D06VOkp6eLz+fMmYP69etj8+bNOHv2bLm/G6OpqYknT56Uqy39NwzlRPReq1atGp48eYL8/Hyoq6sDeP5DojJUqVJF6blMJoNCoaiUvl/Izc2Fn58fZs2aVWydmZlZuWp53fHm5uZiwIABJc5tfvEBpCLKqmXjxo0YM2YMIiIi4O7uDl1dXfz00084efJkhfZR0Wv4pvst6VheXvYiTJb3upubmyMlJQUHDhzA/v37MXjwYPz00084cuRIsX29aU0vllXmazE3Nxfh4eHo2LFjsXUvf6h8G/z8/CCXy7Ft2zaoq6ujoKAAnTt3fu122tra4r9zc3MBALt27cJnn32m1K6scFne99/L+6rIdu/D+zY3NxdmZmYlfnfj5e8VpKen4+bNm1AoFMjMzFT6cFqWe/fuwdjYuNz10JtjKCei99qLkdCkpCTx346OjlAoFDhy5Ai8vb2LbWNnZ4ctW7ZAEAQxdMXHx0NXVxc1a9astNrs7Oywdu1aPHv2TAw2J06cUGrj6uqKLVu2wMLC4o3/TLmTkxNu3LiB1NTUEkfdXF1dkZSUBCsrq3L3qa6ujqKiogrXEh8fj2bNmmHw4MHispdH48rbt7W1NTQ1NXHw4EEEBwdXyn7fFU1NTfj5+cHPzw9DhgxBvXr1cOHCBbi6ur7V/Z44cQIeHh4Ann+58uzZsxg6dGi5tnV1dUVKSkqprxE7Oztcv34d2dnZYuh89bVcmlc/GJ04cQLW1tZQVVUF8HxUPzAwEJGRkVBXV0e3bt0q/KHM3t4ecrkcWVlZpf6GrCRv+v77kN63rq6uuHXrFtTU1GBhYVHidvn5+fj222/RtWtX2NraIjg4GBcuXBDv0FPWe/bixYtl/jaCKg/vvkJE7zVjY2O4uroiLi5OXGZhYYHAwED06dMH0dHRyMjIQGxsLH7//XcAwODBg3H9+nV89913uHz5MrZv345JkyZh1KhRStMG/qsePXpAJpOhX79+SEpKQkxMDObMmaPUZsiQIbh37x66d++O06dPIz09HXv37kXv3r3LHYo9PT3h4eGBTp06Yf/+/cjIyMDu3buxZ88eAMC4ceNw7NgxDB06FAkJCbhy5Qq2b99eZmCzsLBARkYGEhIScOfOHeTl5ZWrFmtra5w5cwZ79+5FamoqQkNDcfr06WJ9//3330hJScGdO3dQUFBQrB8NDQ2MGzcOISEhWLNmDdLT03HixAmlO+hUdL/vQlRUFFatWoWLFy/i6tWrWLduHTQ1NVG7du23vu+ff/4Z27Ztw+XLlzFkyBDcv38fffr0Kde2EydOxJo1axAeHo5Lly4hOTkZGzduxI8//ggA8Pb2ho2NDQIDA5GYmIijR49iwoQJ5eo7KysLo0aNQkpKCjZs2IBFixZh+PDhSm2Cg4Nx6NAh7Nmzp9w1v0xXVxdjxozByJEjsXr1aqSnp+PcuXNYtGgRVq9eXep2b/r++5Det97e3nB3d4e/vz/27duHzMxMHDt2DBMmTBD/8M+ECRPw8OFDLFy4EOPGjYONjY3SdSjr/4OjR4/iiy++KNcx03/DUE5E773g4GCleZcAsHTpUnTu3BmDBw9GvXr10K9fP/FWYp999hliYmJw6tQpODs7Y+DAgejbt68YQCqLjo4O/vzzT1y4cAEuLi6YMGFCsV9316hRA/Hx8SgqKsIXX3wBR0dHjBgxAgYGBhX6gLBlyxY0atQI3bt3h729PUJCQsRw4OTkhCNHjiA1NRUtWrSAi4sLJk6cqDQ/+FWdOnWCr68vWrVqBWNjY2zYsKFcdQwYMAAdO3ZE165d0aRJE9y9e1dp9BoA+vXrB1tbW7i5ucHY2Bjx8fEl9hUaGorRo0dj4sSJsLOzQ9euXUud212e/b4LBgYGWLFiBZo3bw4nJyccOHAAf/755zu5F/nMmTMxc+ZMODs7Iy4uDjt27EC1atXKta2Pjw927tyJffv2oVGjRmjatCnmzZsnfphQUVHBtm3b8PTpUzRu3BjBwcGl3p7yVb169RK3GzJkCIYPH17sjy1ZW1ujWbNmqFevHpo0aVKxA///pkyZgtDQUMyYMQN2dnbw9fXFrl27UKdOnVK3edP334f0vpXJZIiJiYGHhwd69+4NGxsbdOvWDdeuXYOJiQliY2Mxf/58rF27Fnp6elBRUcHatWtx9OhRLF26tNR+AeD48eN4+PBhuaYb0X8nE4T/fzNRIqL31NOnT2Fra4tNmzaV+8tnRB+LzMxM1KlTB+fPn39rf4X1bRMEAdbW1hg8eDBGjRoldTlUTl27doWzszN++OEHqUv5JHBOORG99zQ1NbFmzRqlP9xBRB+Gf//9Fxs3bsStW7fQu3dvqcuhcsrPz4ejoyNGjhwpdSmfDIZyIvogtGzZUuoSiOgNVK9eHdWqVcMvv/yCqlWrSl0OlZO6unqlT/mjsnH6ChERERGRxPhFTyIiIiIiiTGUExERERFJjKGciIiIiEhiDOVERERERBJjKCciIiIikhhDORERERGRxBjKiYiIiIgkxlBORERERCQxhnIiIiIiIon9P5TrJcpLgny0AAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9LZfAmj2YtHS"
      },
      "source": [
        "****\n",
        "\n",
        "(c) 2023 The Logikon AI Team | License: Apache 2.0"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.12"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}