evaluate-annotations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluating annotations of understatements\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "polars is already installed.\n",
      "openai is already installed.\n",
      "instructor is already installed.\n",
      "pydantic is already installed.\n",
      "statsmodels is already installed.\n",
      "os is already installed.\n"
     ]
    }
   ],
   "source": [
    "# Install necessary libraries\n",
    "import sys\n",
    "libraries = [\"polars\", \"openai\", \"instructor\", \"pydantic\", \"statsmodels\", \"os\"]\n",
    "\n",
    "for library in libraries:\n",
    "    try:\n",
    "        __import__(library)\n",
    "        print(f\"{library} is already installed.\")\n",
    "    except ImportError:\n",
    "        print(f\"{library} not found, installing...\")\n",
    "        !{sys.executable} -m pip install {library}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "from typing_extensions import Literal\n",
    "from openai import OpenAI\n",
    "import polars as pl\n",
    "import instructor\n",
    "import statsmodels\n",
    "import os\n",
    "import dotenv\n",
    "import json\n",
    "from statsmodels.stats.inter_rater import fleiss_kappa, aggregate_raters\n",
    "import pandas as pd # statsmodels expects a pandas DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set your key and a model from https://platform.openai.com/docs/models/continuous-model-upgrades\n",
    "chosen_model = \"gpt-4-turbo\"\n",
    "dotenv.load_dotenv('.env')\n",
    "key = os.environ.get('OPENAI_API_KEY')\n",
    "client = OpenAI(api_key=key)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Annotating understatements with LLMs\n",
    "\n",
    "### Step 1: Import and preprocess data\n",
    "\n",
    "The sentences for the annotation task are in a Google Sheet and exported as TSV. \n",
    "\n",
    "To annotate the sentences:\n",
    "1. Open the TSV files\n",
    "2. Extract the sentences from the TSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Open the TSV file.\n",
    "with open('datasets/annotation-task.tsv', 'r') as file:\n",
    "    text = file.read()\n",
    "\n",
    "# Extract the sentences from the TSV file.\n",
    "rows = text.split('\\n')\n",
    "data = [row.split('\\t') for row in rows]\n",
    "unlabeled_dataset = data[1:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Test the understatement annotation task\n",
    "\n",
    "Let's test a simple prompt for the understatement annotation task.\n",
    "\n",
    "In the prompt, we present a phrase in context and ask for four labels:\n",
    "\n",
    "1. Whether the sentence is an understatement (Yes/No)\n",
    "2. *It's confidence in the label (1-3)\n",
    "3. If it thinks the phrase is an understatement, whether its litoses or meiosis (Litotes/Meiosis)\n",
    "4. If it thinks the phrase is an understatement, the pragmatic function of the understatement (Mocking/Humorous/Tempering)\n",
    "\n",
    "The phrase in context is surrounded with a pair of square brackets.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Read the following text and answer four questions: \n",
      "\n",
      "---START OF TEXT---\n",
      "“Do you know, my friends,” said Michel Ardan, “that if one of us had succumbed to the shock consequent on departure, we should have had a great deal of trouble to bury him? What am I saying? to etherize him, as here ether takes the place of earth. You see the accusing body would have followed us into space like a remorse.”  “[That would have been sad],” said Nicholl.\n",
      "---END OF TEXT---\n",
      "\n",
      "Question 1: Is the phrase in square brackets an understatement?\n",
      "Question 2: On a scale from 1 to 3, how confident are you in your answer to question 1?\n",
      "Question 3: If you think the phrase is an understatement, is it type litoses or meiosis?\n",
      "Question 4: If you think the phrase is an understatement, is the pragmatic function of the understatement mocking, humorous, or tempering?\n"
     ]
    }
   ],
   "source": [
    "first_example = unlabeled_dataset[60][1]\n",
    "prompt = f\"\"\"Read the following text and answer four questions: \n",
    "\n",
    "---START OF TEXT---\n",
    "{first_example}\n",
    "---END OF TEXT---\n",
    "\n",
    "Question 1: Is the phrase in square brackets an understatement?\n",
    "Question 2: On a scale from 1 to 3, how confident are you in your answer to question 1?\n",
    "Question 3: If you think the phrase is an understatement, is it type litoses or meiosis?\n",
    "Question 4: If you think the phrase is an understatement, is the pragmatic function of the understatement mocking, humorous, or tempering?\"\"\"\n",
    "\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we create a helper function to pass the prompt to the LLM:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here's how we interact with the model\n",
    "def annotate(chosen_prompt):\n",
    "    resp = client.chat.completions.create(\n",
    "        model=chosen_model, \n",
    "        messages=[\n",
    "            {\"role\": \"user\", \n",
    "            \"content\": chosen_prompt},\n",
    "        ],\n",
    "        temperature=1,\n",
    "        )\n",
    "    annotation = resp.choices[0].message.content\n",
    "    return annotation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test the prompt with our first sentence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer 1: Yes, the phrase in square brackets, \"That would have been sad,\" is an understatement.\n",
      "\n",
      "Answer 2: I am 3 (very confident) in my answer to question 1. The expression \"That would have been sad\" minimizes the emotional impact and the unusual, eerie scenario described, making it a clear understatement.\n",
      "\n",
      "Answer 3: The understatement is an example of meiosis. Meiosis is a type of understatement that deliberately understates or de-emphasizes the importance of something. In this case, describing the potential presence of an \"accusing body\" haunting them in space as merely \"sad\" significantly downplays the scenario.\n",
      "\n",
      "Answer 4: The pragmatic function of the understatement in this context is likely humorous. The use of such a casual remark in the context of a bizarre and potentially horrifying situation can add a touch of irony and levity, suggesting that the speaker is using humor to deal with the concept of death and its consequences in a detached, unusual environment like space.\n"
     ]
    }
   ],
   "source": [
    "print(annotate(prompt))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Done. While the model has replied to all four questions, we have two issues:\n",
    "\n",
    "1. The model only annotated one sentence. We need to annotate **all** the sentences.\n",
    "2. The model's answers are unstructured. We need to extract the answers and save them in a structured format.\n",
    "\n",
    "Let's solve these issues one by one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Annotate many texts (understatements)\n",
    "\n",
    "If we iterate over all the sentences, we can annotate them one by one and save the annotations in a list called `annotations_list`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " --- ANNOTATING TEXT 1 ---\n",
      "Question 1: Yes, the phrase in square brackets is an understatement.\n",
      "\n",
      "Question 2: I am 3 (very confident) in my answer to question 1.\n",
      "\n",
      "Question 3: The phrase in square brackets is a type of meiosis. This is because meiosis involves understating a situation or an element to subtly highlight its opposite, usually extreme, characteristic. In this context, Mrs. Allen's description of merely \"[dressing] up a little\" distinctly underplays the extravagance and grandeur of her highly elaborate and noticeable gown.\n",
      "\n",
      "Question 4: The pragmatic function of the understatement is humorous. By significantly downplaying the effort and lavishness of her appearance, Mrs. Allen highlights the contrast between her words and her actual appearance, eliciting amusement from the audience and possibly from those hearing or reading about the scene. This light-hearted self-deprecation intentionally adds a layer of wit to her grand entrance.\n",
      "\n",
      "\n",
      " --- ANNOTATING TEXT 2 ---\n",
      "Answer 1: No, the phrase in square brackets is not an understatement. It is actually a form of litotes, which is a type of understatement but serves a different function in this context.\n",
      "\n",
      "Answer 2: I am confident at level 3 in my answer to question 1 that the phrase is not merely an understatement but specifically a litotes used for effect.\n",
      "\n",
      "Answer 3: The phrase \"not unremarkable\" is a litotes. Litotes is a figure of speech involving double negatives or, in this case, an understated negative to affirm a positive, typically for emphasis or irony.\n",
      "\n",
      "Answer 4: The pragmatic function of the litotes \"not unremarkable\" in this context is likely mocking. Lord Thompson uses this phrase to ironically emphasize that the carriage, while being described in a seemingly positive light, is actually ordinary and lacks the grandeur one might expect. The irony implies a mocking tone towards the overstatement of the carriage's remarkability.\n",
      "\n",
      "\n",
      " --- ANNOTATING TEXT 3 ---\n",
      "Question 1: Yes, the phrase in square brackets [\"took a tiny interest\"] is an understatement.\n",
      "\n",
      "Question 2: I am 3 out of 3 confident about the answer to question 1, confirming it as an understatement.\n",
      "\n",
      "Question 3: The type of understatement is meiosis. This is distinguished by the diminution of the importance or significance of something, which in this context refers to Emma Woodhouse's concern for Mr. Knightley.\n",
      "\n",
      "Question 4: The pragmatic function of the understatement is tempering. The usage of \"tiny\" downplays the extent of Emma's concern, possibly suggesting her modesty or restraint in expressing her emotions outwardly. This tempering helps balance out what might be perceived as an overreaction, given Mr. Knightley's generally robust health.\n",
      "\n",
      "\n",
      " --- ANNOTATING TEXT 4 ---\n",
      "Question 1: Yes, the phrase in square brackets is an understatement.\n",
      "\n",
      "Question 2: I am 3 out of 3 on the confidence scale that the phrase in square brackets is an understatement.\n",
      "\n",
      "Question 3: The type of understatement used in the phrase in square brackets is meiosis. Meiosis is a form of understatement that intentionally understates the scale or importance of a subject. In this context, the phrase \"Sometimes he's not quite the same, but we've done pretty good work\" significantly downplays potential negative outcomes or transformations of the patients, which can be substantial.\n",
      "\n",
      "Question 4: The pragmatic function of the understatement in this context seems to be tempering. By using this understatement, Dr. Rothberg is likely trying to soften the harsh realities of the treatment and its possible effects, aiming to make the situation seem more manageable or less daunting to the patient. This use of understatement helps in cushioning the impact of the fact that patients might end up significantly changed.\n",
      "\n",
      "\n",
      " --- ANNOTATING TEXT 5 ---\n",
      "Question 1: Yes, the phrase in square brackets appears to be an understatement.\n",
      "\n",
      "Question 2: I would rate my confidence as 3.\n",
      "\n",
      "Question 3: The type of understatement is meiosis, as it minimizes the intensity of the relic being cursed, possibly suggesting it has even more severe or numerous curses than being merely \"twice as cursed\".\n",
      "\n",
      "Question 4: The pragmatic function of the understatement in this context is humorous. Trace's comment, being half-joking, indicates an attempt to lighten the mood or add levity to the situation despite the serious implications of their discovery.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "annotations_list = []\n",
    "\n",
    "for i, row in enumerate(unlabeled_dataset[:5], 1):\n",
    "    \n",
    "    prompt = f\"\"\"Read the following text and answer four questions: \n",
    "\n",
    "---START OF TEXT---\n",
    "{row[1]}\n",
    "---END OF TEXT---\n",
    "\n",
    "Question 1: Is the phrase in square brackets an understatement?\n",
    "Question 2: On a scale from 1 to 3, how confident are you in your answer to question 1?\n",
    "Question 3: If you think the phrase in square brackets is an understatement, is it type litoses or meiosis?\n",
    "Question 4: If you think the phrase in square brackets is an understatement, is the pragmatic function of the understatement mocking, humorous, or tempering?\"\"\"\n",
    "    \n",
    "    print(f\" --- ANNOTATING TEXT {i} ---\")\n",
    "    annotation = annotate(prompt)\n",
    "    print(annotation)\n",
    "    print(\"\\n\")\n",
    "\n",
    "    # store the annotations in the dictionary\n",
    "    annotations_list.append(annotation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Annotating understatements with structured output\n",
    "\n",
    "Now that we can run the prompt for all samples in the dataset, we need to extract the answers and save them in a structured format.\n",
    "\n",
    "To do this, we first need to amend the prompt to elicit structured responses. We do this by commanding the model to respond with a JSON object only:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = f\"\"\"Read the following text and answer four questions: \n",
    "\n",
    "---START OF TEXT---\n",
    "{first_example}\n",
    "---END OF TEXT---\n",
    "\n",
    "Question 1: Is the phrase in square brackets an understatement?\n",
    "Question 2: On a scale from 1 to 3, how confident are you in your answer to question 1?\n",
    "Question 3: If you think the phrase in square brackets is an understatement, is it type litoses or meiosis?\n",
    "Question 4: If you think the phrase in square brackets is an understatement, is the pragmatic function of the understatement mocking, humorous, or tempering?\n",
    "\n",
    "Respond with a JSON object only. Do not return any other information.\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To ensure the LLMs responses adhere to a structured format, we need to define a data model.\n",
    "\n",
    "All responses are then matched to the data model, ensuring uniform structure. \n",
    "\n",
    "We use `pydantic` to define a data model, named `Understatement` and with four attributes:\n",
    "\n",
    "1. `is_understatement`: Whether the sentence is an understatement (Yes/No)\n",
    "2. `confidence`: The model's confidence in the label (1-3)\n",
    "3. `understatement_type`: If the phrase is an understatement, whether it's litotes or meiosis (Litotes/Meiosis)\n",
    "4. `pragmatic_function`: If the phrase is an understatement, the pragmatic function of the understatement (Mocking/Humorous/Tempering)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Understatement(BaseModel):\n",
    "    \"\"\"\n",
    "    An understatement with its annotations.\n",
    "    \"\"\"\n",
    "    is_understatement: bool\n",
    "    confidence: Literal[1, 2, 3]\n",
    "    understatement_type: Literal[\"litotes\", \"meiosis\", None]\n",
    "    pragmatic_function: Literal[\"mocking\", \"humorous\", \"tempering\", None]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we initialize a client for `instructor` to extract the answers from the LLM's output and validate them against the `Understatement` model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = instructor.patch(OpenAI(api_key=key), mode=instructor.Mode.MD_JSON)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's update our helper function to use the `instructor` client, and pass the `Understatement` model to validate the responses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def annotate_structured(chosen_prompt):\n",
    "    resp = client.chat.completions.create(\n",
    "        model=chosen_model, \n",
    "        messages=[\n",
    "            {\"role\": \"user\", \n",
    "            \"content\": chosen_prompt},\n",
    "        ],\n",
    "        temperature=1,\n",
    "        response_model=Understatement,\n",
    "        )\n",
    "    return resp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's use the updated helper function to get a structured `annotation`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "is_understatement=True confidence=2 understatement_type='meiosis' pragmatic_function='humorous'\n"
     ]
    }
   ],
   "source": [
    "annotation = annotate_structured(prompt)\n",
    "print(annotation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result is an `Understatement` object, with four attributes, corresponding to the four labels assigned per sample. \n",
    "\n",
    "These labels are chosen from closed sets of labels, of boolean (`is_understatement`), categorical (`understatement_type` and `pragmatic_function`), and ordinal (`confidence`) type."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5: Annotate many understatements with structured responses\n",
    "\n",
    "Let's combine the steps above to annotate all the sentences in the dataset and save the annotations in a structured format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " --- ANNOTATING TEXT  ---\n",
      "is_understatement=True confidence=3 understatement_type='meiosis' pragmatic_function='humorous'\n",
      " --- ANNOTATING TEXT  ---\n",
      "is_understatement=True confidence=3 understatement_type='litotes' pragmatic_function='mocking'\n",
      " --- ANNOTATING TEXT  ---\n",
      "is_understatement=True confidence=3 understatement_type='meiosis' pragmatic_function='tempering'\n",
      " --- ANNOTATING TEXT  ---\n",
      "is_understatement=True confidence=3 understatement_type='meiosis' pragmatic_function='tempering'\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[48], line 20\u001b[0m\n\u001b[1;32m      6\u001b[0m     prompt \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\"\"\u001b[39m\u001b[38;5;124mRead the following text and answer four questions: \u001b[39m\n\u001b[1;32m      7\u001b[0m \n\u001b[1;32m      8\u001b[0m \u001b[38;5;124m---START OF TEXT---\u001b[39m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     16\u001b[0m \n\u001b[1;32m     17\u001b[0m \u001b[38;5;124mRespond with a JSON object only.\u001b[39m\u001b[38;5;124m\"\"\"\u001b[39m\n\u001b[1;32m     19\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m---> 20\u001b[0m         annotation \u001b[38;5;241m=\u001b[39m \u001b[43mannotate_structured\u001b[49m\u001b[43m(\u001b[49m\u001b[43mprompt\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     21\u001b[0m     \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m     22\u001b[0m         \u001b[38;5;66;03m# If the model fails to return a valid JSON object try again\u001b[39;00m\n\u001b[1;32m     23\u001b[0m         \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mError: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n",
      "Cell \u001b[0;32mIn[46], line 2\u001b[0m, in \u001b[0;36mannotate_structured\u001b[0;34m(chosen_prompt)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mannotate_structured\u001b[39m(chosen_prompt):\n\u001b[0;32m----> 2\u001b[0m     resp \u001b[38;5;241m=\u001b[39m \u001b[43mclient\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mchat\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompletions\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m      3\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mchosen_model\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\n\u001b[1;32m      4\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmessages\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m[\u001b[49m\n\u001b[1;32m      5\u001b[0m \u001b[43m            \u001b[49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mrole\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43muser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\n\u001b[1;32m      6\u001b[0m \u001b[43m            \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcontent\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mchosen_prompt\u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m      7\u001b[0m \u001b[43m        \u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m      8\u001b[0m \u001b[43m        \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m1\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m      9\u001b[0m \u001b[43m        \u001b[49m\u001b[43mresponse_model\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mUnderstatement\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     10\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     11\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m resp\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/instructor/patch.py:392\u001b[0m, in \u001b[0;36mwrap_chatcompletion.<locals>.new_chatcompletion_sync\u001b[0;34m(response_model, validation_context, max_retries, *args, **kwargs)\u001b[0m\n\u001b[1;32m    381\u001b[0m \u001b[38;5;129m@wraps\u001b[39m(func)\n\u001b[1;32m    382\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mnew_chatcompletion_sync\u001b[39m(\n\u001b[1;32m    383\u001b[0m     response_model\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    387\u001b[0m     \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[1;32m    388\u001b[0m ):\n\u001b[1;32m    389\u001b[0m     response_model, new_kwargs \u001b[38;5;241m=\u001b[39m handle_response_model(\n\u001b[1;32m    390\u001b[0m         response_model\u001b[38;5;241m=\u001b[39mresponse_model, kwargs\u001b[38;5;241m=\u001b[39mkwargs, mode\u001b[38;5;241m=\u001b[39mmode\n\u001b[1;32m    391\u001b[0m     )  \u001b[38;5;66;03m# type: ignore\u001b[39;00m\n\u001b[0;32m--> 392\u001b[0m     response \u001b[38;5;241m=\u001b[39m \u001b[43mretry_sync\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    393\u001b[0m \u001b[43m        \u001b[49m\u001b[43mfunc\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfunc\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    394\u001b[0m \u001b[43m        \u001b[49m\u001b[43mresponse_model\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mresponse_model\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    395\u001b[0m \u001b[43m        \u001b[49m\u001b[43mvalidation_context\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mvalidation_context\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    396\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmax_retries\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmax_retries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    397\u001b[0m \u001b[43m        \u001b[49m\u001b[43margs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    398\u001b[0m \u001b[43m        \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mnew_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    399\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmode\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmode\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    400\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m  \u001b[38;5;66;03m# type: ignore\u001b[39;00m\n\u001b[1;32m    401\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m response\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/instructor/patch.py:300\u001b[0m, in \u001b[0;36mretry_sync\u001b[0;34m(func, response_model, validation_context, args, kwargs, max_retries, strict, mode)\u001b[0m\n\u001b[1;32m    297\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m retries \u001b[38;5;241m<\u001b[39m\u001b[38;5;241m=\u001b[39m max_retries:\n\u001b[1;32m    298\u001b[0m     \u001b[38;5;66;03m# Excepts ValidationError, and JSONDecodeError\u001b[39;00m\n\u001b[1;32m    299\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 300\u001b[0m         response \u001b[38;5;241m=\u001b[39m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    301\u001b[0m         stream \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstream\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mFalse\u001b[39;00m)\n\u001b[1;32m    302\u001b[0m         \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(response, ChatCompletion) \u001b[38;5;129;01mand\u001b[39;00m response\u001b[38;5;241m.\u001b[39musage \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/openai/_utils/_utils.py:275\u001b[0m, in \u001b[0;36mrequired_args.<locals>.inner.<locals>.wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    273\u001b[0m             msg \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mMissing required argument: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mquote(missing[\u001b[38;5;241m0\u001b[39m])\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    274\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(msg)\n\u001b[0;32m--> 275\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/openai/resources/chat/completions.py:667\u001b[0m, in \u001b[0;36mCompletions.create\u001b[0;34m(self, messages, model, frequency_penalty, function_call, functions, logit_bias, logprobs, max_tokens, n, presence_penalty, response_format, seed, stop, stream, temperature, tool_choice, tools, top_logprobs, top_p, user, extra_headers, extra_query, extra_body, timeout)\u001b[0m\n\u001b[1;32m    615\u001b[0m \u001b[38;5;129m@required_args\u001b[39m([\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmessages\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel\u001b[39m\u001b[38;5;124m\"\u001b[39m], [\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmessages\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstream\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n\u001b[1;32m    616\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcreate\u001b[39m(\n\u001b[1;32m    617\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    665\u001b[0m     timeout: \u001b[38;5;28mfloat\u001b[39m \u001b[38;5;241m|\u001b[39m httpx\u001b[38;5;241m.\u001b[39mTimeout \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m|\u001b[39m NotGiven \u001b[38;5;241m=\u001b[39m NOT_GIVEN,\n\u001b[1;32m    666\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ChatCompletion \u001b[38;5;241m|\u001b[39m Stream[ChatCompletionChunk]:\n\u001b[0;32m--> 667\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_post\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    668\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/chat/completions\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m    669\u001b[0m \u001b[43m        \u001b[49m\u001b[43mbody\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmaybe_transform\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    670\u001b[0m \u001b[43m            \u001b[49m\u001b[43m{\u001b[49m\n\u001b[1;32m    671\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmessages\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    672\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmodel\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    673\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfrequency_penalty\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfrequency_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    674\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfunction_call\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunction_call\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    675\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfunctions\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunctions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    676\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlogit_bias\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogit_bias\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    677\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlogprobs\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    678\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmax_tokens\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    679\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mn\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    680\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mpresence_penalty\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mpresence_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    681\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mresponse_format\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mresponse_format\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    682\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mseed\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mseed\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    683\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstop\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstop\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    684\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstream\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    685\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtemperature\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    686\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtool_choice\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtool_choice\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    687\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtools\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtools\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    688\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtop_logprobs\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_logprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    689\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtop_p\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_p\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    690\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43muser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43muser\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    691\u001b[0m \u001b[43m            \u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    692\u001b[0m \u001b[43m            \u001b[49m\u001b[43mcompletion_create_params\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCompletionCreateParams\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    693\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    694\u001b[0m \u001b[43m        \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmake_request_options\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    695\u001b[0m \u001b[43m            \u001b[49m\u001b[43mextra_headers\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_headers\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_query\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_query\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_body\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_body\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\n\u001b[1;32m    696\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    697\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mChatCompletion\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    698\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m    699\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mStream\u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatCompletionChunk\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    700\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/openai/_base_client.py:1233\u001b[0m, in \u001b[0;36mSyncAPIClient.post\u001b[0;34m(self, path, cast_to, body, options, files, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1219\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mpost\u001b[39m(\n\u001b[1;32m   1220\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m   1221\u001b[0m     path: \u001b[38;5;28mstr\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1228\u001b[0m     stream_cls: \u001b[38;5;28mtype\u001b[39m[_StreamT] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[1;32m   1229\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ResponseT \u001b[38;5;241m|\u001b[39m _StreamT:\n\u001b[1;32m   1230\u001b[0m     opts \u001b[38;5;241m=\u001b[39m FinalRequestOptions\u001b[38;5;241m.\u001b[39mconstruct(\n\u001b[1;32m   1231\u001b[0m         method\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpost\u001b[39m\u001b[38;5;124m\"\u001b[39m, url\u001b[38;5;241m=\u001b[39mpath, json_data\u001b[38;5;241m=\u001b[39mbody, files\u001b[38;5;241m=\u001b[39mto_httpx_files(files), \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions\n\u001b[1;32m   1232\u001b[0m     )\n\u001b[0;32m-> 1233\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m cast(ResponseT, \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mopts\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m)\u001b[49m)\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/openai/_base_client.py:922\u001b[0m, in \u001b[0;36mSyncAPIClient.request\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m    913\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mrequest\u001b[39m(\n\u001b[1;32m    914\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m    915\u001b[0m     cast_to: Type[ResponseT],\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    920\u001b[0m     stream_cls: \u001b[38;5;28mtype\u001b[39m[_StreamT] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[1;32m    921\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ResponseT \u001b[38;5;241m|\u001b[39m _StreamT:\n\u001b[0;32m--> 922\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    923\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    924\u001b[0m \u001b[43m        \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    925\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    926\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    927\u001b[0m \u001b[43m        \u001b[49m\u001b[43mremaining_retries\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mremaining_retries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    928\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/openai/_base_client.py:951\u001b[0m, in \u001b[0;36mSyncAPIClient._request\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m    948\u001b[0m     kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mauth\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mcustom_auth\n\u001b[1;32m    950\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 951\u001b[0m     response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_client\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msend\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    952\u001b[0m \u001b[43m        \u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    953\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_should_stream_response_body\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    954\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    955\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    956\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m httpx\u001b[38;5;241m.\u001b[39mTimeoutException \u001b[38;5;28;01mas\u001b[39;00m err:\n\u001b[1;32m    957\u001b[0m     log\u001b[38;5;241m.\u001b[39mdebug(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEncountered httpx.TimeoutException\u001b[39m\u001b[38;5;124m\"\u001b[39m, exc_info\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m)\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpx/_client.py:914\u001b[0m, in \u001b[0;36mClient.send\u001b[0;34m(self, request, stream, auth, follow_redirects)\u001b[0m\n\u001b[1;32m    906\u001b[0m follow_redirects \u001b[38;5;241m=\u001b[39m (\n\u001b[1;32m    907\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mfollow_redirects\n\u001b[1;32m    908\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(follow_redirects, UseClientDefault)\n\u001b[1;32m    909\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m follow_redirects\n\u001b[1;32m    910\u001b[0m )\n\u001b[1;32m    912\u001b[0m auth \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_build_request_auth(request, auth)\n\u001b[0;32m--> 914\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_send_handling_auth\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    915\u001b[0m \u001b[43m    \u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    916\u001b[0m \u001b[43m    \u001b[49m\u001b[43mauth\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mauth\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    917\u001b[0m \u001b[43m    \u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    918\u001b[0m \u001b[43m    \u001b[49m\u001b[43mhistory\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m[\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    919\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    920\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    921\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m stream:\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpx/_client.py:942\u001b[0m, in \u001b[0;36mClient._send_handling_auth\u001b[0;34m(self, request, auth, follow_redirects, history)\u001b[0m\n\u001b[1;32m    939\u001b[0m request \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mnext\u001b[39m(auth_flow)\n\u001b[1;32m    941\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[0;32m--> 942\u001b[0m     response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_send_handling_redirects\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    943\u001b[0m \u001b[43m        \u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    944\u001b[0m \u001b[43m        \u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    945\u001b[0m \u001b[43m        \u001b[49m\u001b[43mhistory\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mhistory\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    946\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    947\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    948\u001b[0m         \u001b[38;5;28;01mtry\u001b[39;00m:\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpx/_client.py:979\u001b[0m, in \u001b[0;36mClient._send_handling_redirects\u001b[0;34m(self, request, follow_redirects, history)\u001b[0m\n\u001b[1;32m    976\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m hook \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_event_hooks[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrequest\u001b[39m\u001b[38;5;124m\"\u001b[39m]:\n\u001b[1;32m    977\u001b[0m     hook(request)\n\u001b[0;32m--> 979\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_send_single_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    980\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    981\u001b[0m     \u001b[38;5;28;01mfor\u001b[39;00m hook \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_event_hooks[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mresponse\u001b[39m\u001b[38;5;124m\"\u001b[39m]:\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpx/_client.py:1015\u001b[0m, in \u001b[0;36mClient._send_single_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m   1010\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\n\u001b[1;32m   1011\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mAttempted to send an async request with a sync Client instance.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m   1012\u001b[0m     )\n\u001b[1;32m   1014\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m request_context(request\u001b[38;5;241m=\u001b[39mrequest):\n\u001b[0;32m-> 1015\u001b[0m     response \u001b[38;5;241m=\u001b[39m \u001b[43mtransport\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandle_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1017\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(response\u001b[38;5;241m.\u001b[39mstream, SyncByteStream)\n\u001b[1;32m   1019\u001b[0m response\u001b[38;5;241m.\u001b[39mrequest \u001b[38;5;241m=\u001b[39m request\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py:233\u001b[0m, in \u001b[0;36mHTTPTransport.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m    220\u001b[0m req \u001b[38;5;241m=\u001b[39m httpcore\u001b[38;5;241m.\u001b[39mRequest(\n\u001b[1;32m    221\u001b[0m     method\u001b[38;5;241m=\u001b[39mrequest\u001b[38;5;241m.\u001b[39mmethod,\n\u001b[1;32m    222\u001b[0m     url\u001b[38;5;241m=\u001b[39mhttpcore\u001b[38;5;241m.\u001b[39mURL(\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    230\u001b[0m     extensions\u001b[38;5;241m=\u001b[39mrequest\u001b[38;5;241m.\u001b[39mextensions,\n\u001b[1;32m    231\u001b[0m )\n\u001b[1;32m    232\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m map_httpcore_exceptions():\n\u001b[0;32m--> 233\u001b[0m     resp \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_pool\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandle_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mreq\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    235\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(resp\u001b[38;5;241m.\u001b[39mstream, typing\u001b[38;5;241m.\u001b[39mIterable)\n\u001b[1;32m    237\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m Response(\n\u001b[1;32m    238\u001b[0m     status_code\u001b[38;5;241m=\u001b[39mresp\u001b[38;5;241m.\u001b[39mstatus,\n\u001b[1;32m    239\u001b[0m     headers\u001b[38;5;241m=\u001b[39mresp\u001b[38;5;241m.\u001b[39mheaders,\n\u001b[1;32m    240\u001b[0m     stream\u001b[38;5;241m=\u001b[39mResponseStream(resp\u001b[38;5;241m.\u001b[39mstream),\n\u001b[1;32m    241\u001b[0m     extensions\u001b[38;5;241m=\u001b[39mresp\u001b[38;5;241m.\u001b[39mextensions,\n\u001b[1;32m    242\u001b[0m )\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py:216\u001b[0m, in \u001b[0;36mConnectionPool.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m    213\u001b[0m         closing \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_assign_requests_to_connections()\n\u001b[1;32m    215\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_connections(closing)\n\u001b[0;32m--> 216\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m exc \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m    218\u001b[0m \u001b[38;5;66;03m# Return the response. Note that in this case we still have to manage\u001b[39;00m\n\u001b[1;32m    219\u001b[0m \u001b[38;5;66;03m# the point at which the response is closed.\u001b[39;00m\n\u001b[1;32m    220\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(response\u001b[38;5;241m.\u001b[39mstream, Iterable)\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py:196\u001b[0m, in \u001b[0;36mConnectionPool.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m    192\u001b[0m connection \u001b[38;5;241m=\u001b[39m pool_request\u001b[38;5;241m.\u001b[39mwait_for_connection(timeout\u001b[38;5;241m=\u001b[39mtimeout)\n\u001b[1;32m    194\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    195\u001b[0m     \u001b[38;5;66;03m# Send the request on the assigned connection.\u001b[39;00m\n\u001b[0;32m--> 196\u001b[0m     response \u001b[38;5;241m=\u001b[39m \u001b[43mconnection\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandle_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    197\u001b[0m \u001b[43m        \u001b[49m\u001b[43mpool_request\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrequest\u001b[49m\n\u001b[1;32m    198\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    199\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m ConnectionNotAvailable:\n\u001b[1;32m    200\u001b[0m     \u001b[38;5;66;03m# In some cases a connection may initially be available to\u001b[39;00m\n\u001b[1;32m    201\u001b[0m     \u001b[38;5;66;03m# handle a request, but then become unavailable.\u001b[39;00m\n\u001b[1;32m    202\u001b[0m     \u001b[38;5;66;03m#\u001b[39;00m\n\u001b[1;32m    203\u001b[0m     \u001b[38;5;66;03m# In this case we clear the connection and try again.\u001b[39;00m\n\u001b[1;32m    204\u001b[0m     pool_request\u001b[38;5;241m.\u001b[39mclear_connection()\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpcore/_sync/connection.py:101\u001b[0m, in \u001b[0;36mHTTPConnection.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m     98\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_connect_failed \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[1;32m     99\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m exc\n\u001b[0;32m--> 101\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_connection\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandle_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpcore/_sync/http11.py:143\u001b[0m, in \u001b[0;36mHTTP11Connection.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m    141\u001b[0m     \u001b[38;5;28;01mwith\u001b[39;00m Trace(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mresponse_closed\u001b[39m\u001b[38;5;124m\"\u001b[39m, logger, request) \u001b[38;5;28;01mas\u001b[39;00m trace:\n\u001b[1;32m    142\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_response_closed()\n\u001b[0;32m--> 143\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpcore/_sync/http11.py:113\u001b[0m, in \u001b[0;36mHTTP11Connection.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m    102\u001b[0m     \u001b[38;5;28;01mpass\u001b[39;00m\n\u001b[1;32m    104\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m Trace(\n\u001b[1;32m    105\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mreceive_response_headers\u001b[39m\u001b[38;5;124m\"\u001b[39m, logger, request, kwargs\n\u001b[1;32m    106\u001b[0m ) \u001b[38;5;28;01mas\u001b[39;00m trace:\n\u001b[1;32m    107\u001b[0m     (\n\u001b[1;32m    108\u001b[0m         http_version,\n\u001b[1;32m    109\u001b[0m         status,\n\u001b[1;32m    110\u001b[0m         reason_phrase,\n\u001b[1;32m    111\u001b[0m         headers,\n\u001b[1;32m    112\u001b[0m         trailing_data,\n\u001b[0;32m--> 113\u001b[0m     ) \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_receive_response_headers\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    114\u001b[0m     trace\u001b[38;5;241m.\u001b[39mreturn_value \u001b[38;5;241m=\u001b[39m (\n\u001b[1;32m    115\u001b[0m         http_version,\n\u001b[1;32m    116\u001b[0m         status,\n\u001b[1;32m    117\u001b[0m         reason_phrase,\n\u001b[1;32m    118\u001b[0m         headers,\n\u001b[1;32m    119\u001b[0m     )\n\u001b[1;32m    121\u001b[0m network_stream \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_network_stream\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpcore/_sync/http11.py:186\u001b[0m, in \u001b[0;36mHTTP11Connection._receive_response_headers\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m    183\u001b[0m timeout \u001b[38;5;241m=\u001b[39m timeouts\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mread\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mNone\u001b[39;00m)\n\u001b[1;32m    185\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[0;32m--> 186\u001b[0m     event \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_receive_event\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    187\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(event, h11\u001b[38;5;241m.\u001b[39mResponse):\n\u001b[1;32m    188\u001b[0m         \u001b[38;5;28;01mbreak\u001b[39;00m\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpcore/_sync/http11.py:224\u001b[0m, in \u001b[0;36mHTTP11Connection._receive_event\u001b[0;34m(self, timeout)\u001b[0m\n\u001b[1;32m    221\u001b[0m     event \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_h11_state\u001b[38;5;241m.\u001b[39mnext_event()\n\u001b[1;32m    223\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m event \u001b[38;5;129;01mis\u001b[39;00m h11\u001b[38;5;241m.\u001b[39mNEED_DATA:\n\u001b[0;32m--> 224\u001b[0m     data \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_network_stream\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    225\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mREAD_NUM_BYTES\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\n\u001b[1;32m    226\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    228\u001b[0m     \u001b[38;5;66;03m# If we feed this case through h11 we'll raise an exception like:\u001b[39;00m\n\u001b[1;32m    229\u001b[0m     \u001b[38;5;66;03m#\u001b[39;00m\n\u001b[1;32m    230\u001b[0m     \u001b[38;5;66;03m#     httpcore.RemoteProtocolError: can't handle event type\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    234\u001b[0m     \u001b[38;5;66;03m# perspective. Instead we handle this case distinctly and treat\u001b[39;00m\n\u001b[1;32m    235\u001b[0m     \u001b[38;5;66;03m# it as a ConnectError.\u001b[39;00m\n\u001b[1;32m    236\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m data \u001b[38;5;241m==\u001b[39m \u001b[38;5;124mb\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_h11_state\u001b[38;5;241m.\u001b[39mtheir_state \u001b[38;5;241m==\u001b[39m h11\u001b[38;5;241m.\u001b[39mSEND_RESPONSE:\n",
      "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/httpcore/_backends/sync.py:126\u001b[0m, in \u001b[0;36mSyncStream.read\u001b[0;34m(self, max_bytes, timeout)\u001b[0m\n\u001b[1;32m    124\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m map_exceptions(exc_map):\n\u001b[1;32m    125\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_sock\u001b[38;5;241m.\u001b[39msettimeout(timeout)\n\u001b[0;32m--> 126\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_sock\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrecv\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmax_bytes\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py:1226\u001b[0m, in \u001b[0;36mSSLSocket.recv\u001b[0;34m(self, buflen, flags)\u001b[0m\n\u001b[1;32m   1222\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m flags \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m0\u001b[39m:\n\u001b[1;32m   1223\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[1;32m   1224\u001b[0m             \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mnon-zero flags not allowed in calls to recv() on \u001b[39m\u001b[38;5;132;01m%s\u001b[39;00m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;241m%\u001b[39m\n\u001b[1;32m   1225\u001b[0m             \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__class__\u001b[39m)\n\u001b[0;32m-> 1226\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread\u001b[49m\u001b[43m(\u001b[49m\u001b[43mbuflen\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1227\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m   1228\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28msuper\u001b[39m()\u001b[38;5;241m.\u001b[39mrecv(buflen, flags)\n",
      "File \u001b[0;32m/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py:1101\u001b[0m, in \u001b[0;36mSSLSocket.read\u001b[0;34m(self, len, buffer)\u001b[0m\n\u001b[1;32m   1099\u001b[0m         \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_sslobj\u001b[38;5;241m.\u001b[39mread(\u001b[38;5;28mlen\u001b[39m, buffer)\n\u001b[1;32m   1100\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m-> 1101\u001b[0m         \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_sslobj\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mlen\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1102\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m SSLError \u001b[38;5;28;01mas\u001b[39;00m x:\n\u001b[1;32m   1103\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m x\u001b[38;5;241m.\u001b[39margs[\u001b[38;5;241m0\u001b[39m] \u001b[38;5;241m==\u001b[39m SSL_ERROR_EOF \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39msuppress_ragged_eofs:\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "annotated_data = []\n",
    "\n",
    "for row in unlabeled_dataset:\n",
    "    sample = {}\n",
    "    \n",
    "    prompt = f\"\"\"Read the following text and answer four questions: \n",
    "\n",
    "---START OF TEXT---\n",
    "{row[1]}\n",
    "---END OF TEXT---\n",
    "\n",
    "Question 1: Is the phrase in square brackets an understatement?\n",
    "Question 2: On a scale from 1 to 3, how confident are you in your answer to question 1?\n",
    "Question 3: If you think the phrase in square brackets is an understatement, is it type litoses or meiosis?\n",
    "Question 4: If you think the phrase in square brackets is an understatement, is the pragmatic function of the understatement mocking, humorous, or tempering?\n",
    "\n",
    "Respond with a JSON object only.\"\"\"\n",
    "    \n",
    "    try:\n",
    "        annotation = annotate_structured(prompt)\n",
    "    except Exception as e:\n",
    "        # If the model fails to return a valid JSON object try again\n",
    "        print(f\"Error: {e}\")\n",
    "        annotation = annotate_structured(prompt)\n",
    "\n",
    "    print(f\" --- ANNOTATING TEXT {row[-1]} ---\")\n",
    "    print(annotation)\n",
    "\n",
    "    sample[\"phrase\"] = row[0]\n",
    "    sample[\"full_text\"] = row[1]\n",
    "    sample[\"is_understatement\"] = annotation.is_understatement\n",
    "    sample[\"confidence\"] = annotation.confidence\n",
    "    sample[\"understatement_type\"] = annotation.understatement_type\n",
    "    sample[\"pragmatic_function\"] = annotation.pragmatic_function\n",
    "    annotated_data.append(sample)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we move on, let's save the annotations in a JSON file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "with open('datasets/annotated-data-llm_baseline.json', 'w') as file:\n",
    "    json.dump(annotated_data, file, indent=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we may want to rerun this notebook without having to reannotate everything through OpenAI again, we can simply load what we have saved:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('datasets/annotated-data-llm_baseline.json', 'r') as file:\n",
    "    annotated_data = json.load(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Done. Let's inspect the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " --- ANNOTATED TEXT 1 ---\n",
      "phrase: \"I might have dressed up a little for the evening\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 2 ---\n",
      "phrase: \"not unremarkable\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: mocking\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 3 ---\n",
      "phrase: \"took a tiny interest\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 4 ---\n",
      "phrase: \"Sometimes he's not quite the same, but we've done pretty good work.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 5 ---\n",
      "phrase: \"And probably twice as cursed\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 6 ---\n",
      "phrase: \"the stars themselves are hardly dead\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 7 ---\n",
      "phrase: \"just a few lines\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 8 ---\n",
      "phrase: \"less than perfect\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 9 ---\n",
      "phrase: \"I suppose I am not wholly averse to the idea.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 10 ---\n",
      "phrase: \"not too daunting\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 11 ---\n",
      "phrase: \"was not exactly warm\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 12 ---\n",
      "phrase: \"some inclement weather\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 13 ---\n",
      "phrase: \"he seemed rather pleased\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 14 ---\n",
      "phrase: \"not too bad\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 15 ---\n",
      "phrase: \"He seldom appeared really animated\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 16 ---\n",
      "phrase: \"Seeing the fabric of reality bend was interesting, if not a bit unsettling\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 17 ---\n",
      "phrase: \"I do not believe that I am made of the stuff which constitutes heroes\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 18 ---\n",
      "phrase: \"‘It’s just a minor adjustment, nothing to worry about,’\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 19 ---\n",
      "phrase: \"One mustn't.'\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 20 ---\n",
      "phrase: \"Intense light was needed\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 21 ---\n",
      "phrase: \"‘Nothing out of order,’\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 22 ---\n",
      "phrase: \"somewhat partial to her\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 23 ---\n",
      "phrase: \"slightly larger company\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 24 ---\n",
      "phrase: \"not too pleased\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 25 ---\n",
      "phrase: \"there was a slight stir among the servants\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 26 ---\n",
      "phrase: \"somewhat puzzled\"\n",
      "is_understatement: True\n",
      "confidence: 2\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 27 ---\n",
      "phrase: \"just another glitch in the vast, unfeeling universe.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 28 ---\n",
      "phrase: \"yet the message brought little comfort\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 29 ---\n",
      "phrase: \"quite decent\"\n",
      "is_understatement: True\n",
      "confidence: 2\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 30 ---\n",
      "phrase: \"whose affections for Emma were somewhat known\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 31 ---\n",
      "phrase: \"The plants were slightly miffed about the whole deforestation issue.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 32 ---\n",
      "phrase: \"Lizzy is not a bit better than the others\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: mocking\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 33 ---\n",
      "phrase: \"What man in my shoes wouldn't be happy?\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: mocking\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 34 ---\n",
      "phrase: \"she did not get the best.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 35 ---\n",
      "phrase: \"I am not absolutely miserable anymore\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 36 ---\n",
      "phrase: \"a gentleman of some repute in the art of dance\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 37 ---\n",
      "phrase: \"It appears not entirely unchanged\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 38 ---\n",
      "phrase: \"It was somewhat peculiar to see the sun rise twice in one day\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 39 ---\n",
      "phrase: \"There is, however, a tendency to draw an unreal distinction between the former three dimensions and the latter\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 40 ---\n",
      "phrase: \"Everything was perfectly swell.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: mocking\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 41 ---\n",
      "phrase: \"\"It will be rather a big thing,\"\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 42 ---\n",
      "phrase: \"large as was the great hall, it was quite inadequate to accommodate the crowd of savants.\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 43 ---\n",
      "phrase: \"A mere spring shower\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 44 ---\n",
      "phrase: \"we shall become the mere plaything of the storm\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 45 ---\n",
      "phrase: \"You forget yourself!'\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 46 ---\n",
      "phrase: \"Completing the mission, Hale,\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 47 ---\n",
      "phrase: \"the mishap was not particularly fortunate\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 48 ---\n",
      "phrase: \"John had not much affection for his mother and sisters\"\n",
      "is_understatement: True\n",
      "confidence: 2\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 49 ---\n",
      "phrase: \"It is not the most joyful news.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 50 ---\n",
      "phrase: \"luckier than I\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 51 ---\n",
      "phrase: \"with a mild urgency\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 52 ---\n",
      "phrase: \"It seems we might need an umbrella after all.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 53 ---\n",
      "phrase: \"he was very reticent\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 54 ---\n",
      "phrase: \"\"He’s in there, tinkering with history itself,\"\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 55 ---\n",
      "phrase: \"If people are unworthy, they are removed;\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 56 ---\n",
      "phrase: \"\"It appears I've caused a small mess.\"\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 57 ---\n",
      "phrase: \"the weather was quite agreeable\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 58 ---\n",
      "phrase: \"Unfortunately, the professor arrived a few years off his target, which was inconvenient but not unexpected.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 59 ---\n",
      "phrase: \"When the artificial intelligence took over, it politely suggested it could manage things a tad better.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 60 ---\n",
      "phrase: \"Seeing the Earth from space for the first time was quite nice, he thought.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 61 ---\n",
      "phrase: \"“That would have been sad,”\"\n",
      "is_understatement: True\n",
      "confidence: 2\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 62 ---\n",
      "phrase: \"\"To travel in time! That would be an adventure.\"\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 63 ---\n",
      "phrase: \"It looked like a nice day. \"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 64 ---\n",
      "phrase: \"a little bit magnificent\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 65 ---\n",
      "phrase: \"I am a tad late\"\n",
      "is_understatement: True\n",
      "confidence: 2\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 66 ---\n",
      "phrase: \"occasional clumsiness\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 67 ---\n",
      "phrase: \"\"Bottled cholera, so to speak.\"\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 68 ---\n",
      "phrase: \"The troubles\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 69 ---\n",
      "phrase: \"the battle was hardly more intelligible.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 70 ---\n",
      "phrase: \"Now there's a familiar face\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 71 ---\n",
      "phrase: \"It’s a relatively small vessel\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 72 ---\n",
      "phrase: \"It is not an unbearable burden\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 73 ---\n",
      "phrase: \"There might be a small chance they are suited\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 74 ---\n",
      "phrase: \"felt a slight chill\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 75 ---\n",
      "phrase: \"I suppose you'd be ready for the insane asylum now\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 76 ---\n",
      "phrase: \"one little whippet tank\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 77 ---\n",
      "phrase: \"We had no indication of the odd business that was going to happen\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 78 ---\n",
      "phrase: \"I could not pass quite as a blank\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 79 ---\n",
      "phrase: \"It's somewhat wet out there, it seems.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 80 ---\n",
      "phrase: \"This might change a few things around here.'\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 81 ---\n",
      "phrase: \"\"but only in the small matter of drowning.\"\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 82 ---\n",
      "phrase: \"It's sort of like sitting around home.\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 83 ---\n",
      "phrase: \"He corrected himself, gave the municipal gas chambers their official title, a title no one ever used in conversation. \"I should have said, 'Ethical Suicide Studios,'\" he said.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 84 ---\n",
      "phrase: \"she was somewhat pleasing to the eye\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 85 ---\n",
      "phrase: \"the conversation was slightly lively\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 86 ---\n",
      "phrase: \"tiny disturbance\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 87 ---\n",
      "phrase: \"not too sweet\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 88 ---\n",
      "phrase: \"by no means unwilling to preside at his table\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 89 ---\n",
      "phrase: \"I suppose I did quite all right\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 90 ---\n",
      "phrase: \"somewhat noticeable entrance\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 91 ---\n",
      "phrase: \"Planoforming was sort of funny.\"\n",
      "is_understatement: True\n",
      "confidence: 2\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 92 ---\n",
      "phrase: \" it hardly predicts the weather correctly.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 93 ---\n",
      "phrase: \"\"Recalibration: my favorite pastime.\"\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 94 ---\n",
      "phrase: \"I had not realized your friendship with Sophia was rather so warm\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 95 ---\n",
      "phrase: \"he looked fairly presentable\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 96 ---\n",
      "phrase: \"very seldom slept in\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 97 ---\n",
      "phrase: \"his slight presence throughout the night\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 98 ---\n",
      "phrase: \"It was a bit chilly today, only minus seventy degrees.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 99 ---\n",
      "phrase: \"This ceremony concluded to the satisfaction of all parties, we all sat down to table, that is twenty-four of us, somewhat crowded\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 100 ---\n",
      "phrase: \"Rome was not nearly such a desirable place as it was reported to be\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: mocking\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 101 ---\n",
      "phrase: \"he has simply succeeded in blinding a few dogs\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 102 ---\n",
      "phrase: \"“So, it seems the quantum stabilizer was just slightly off-kilter,”\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 103 ---\n",
      "phrase: \"You can't make an omelet without breaking eggs'\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 104 ---\n",
      "phrase: \"\"She is not disagreeable.\"\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 105 ---\n",
      "phrase: \"We’ve had a bit of wind last night, it seems\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 106 ---\n",
      "phrase: \"not altogether accurate\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 107 ---\n",
      "phrase: \"She's not entirely unappealing\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 108 ---\n",
      "phrase: \"spot of weather\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 109 ---\n",
      "phrase: \"The destruction of the entire planet was, in retrospect, a bit of a setback.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 110 ---\n",
      "phrase: \"But since man had chosen so to disturb the atmosphere, he was bound to accept the consequences of his experiment.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 111 ---\n",
      "phrase: \"I think that at that time none of us quite believed in the Time Machine. \"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 112 ---\n",
      "phrase: \"“I manage to keep in time with the music, more or less,”\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: litotes\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 113 ---\n",
      "phrase: \"quite ordinary\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 114 ---\n",
      "phrase: \"a relatively pleasant chat\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 115 ---\n",
      "phrase: \"not particularly noteworthy\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 116 ---\n",
      "phrase: \"I never liked long walks\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 117 ---\n",
      "phrase: \"overwhelmed by his generosity\"\n",
      "is_understatement: False\n",
      "confidence: 3\n",
      "understatement_type: None\n",
      "pragmatic_function: None\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 118 ---\n",
      "phrase: \"Tonight was somewhat memorable\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: tempering\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 119 ---\n",
      "phrase: \"the small mishap\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n",
      " --- ANNOTATED TEXT 120 ---\n",
      "phrase: \"You might say the ability to teleport makes running errands a touch easier.\"\n",
      "is_understatement: True\n",
      "confidence: 3\n",
      "understatement_type: meiosis\n",
      "pragmatic_function: humorous\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "annotations_baseline_llm = annotated_data\n",
    "\n",
    "for i, sentence in enumerate(annotations_baseline_llm, 1):\n",
    "    print(f\" --- ANNOTATED TEXT {i} ---\")\n",
    "    print(f'phrase: \"{sentence[\"phrase\"]}\"')\n",
    "    print(f\"is_understatement: {sentence['is_understatement']}\")\n",
    "    print(f\"confidence: {sentence['confidence']}\")\n",
    "    print(f\"understatement_type: {sentence['understatement_type']}\")\n",
    "    print(f\"pragmatic_function: {sentence['pragmatic_function']}\")\n",
    "    print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have successfully annotated all the sentences with an LLM, we can load the human annotations and compute Inter-Annotator Agreement metrics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Load data from human annotators\n",
    "\n",
    "Let's load the annotations from human annotators and normalize the labels to match the LLM annotations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def extract_annotation_dict(row):\n",
    "    \"\"\"\n",
    "    Extracts the data from a row of the TSV file.\n",
    "    \n",
    "    Args:\n",
    "    - row: a list with the data of a row of the TSV file\n",
    "    \"\"\"\n",
    "\n",
    "    sample = {}\n",
    "    sample['phrase'] = row[0]\n",
    "    sample['full_text'] = row[1]\n",
    "    sample['is_understatement'] = row[2]\n",
    "    sample['confidence'] = row[3]\n",
    "    sample['pragmatic_function'] = row[4]\n",
    "    sample['understatement_type'] = row[5]\n",
    "    return sample\n",
    "\n",
    "def normalize_annotation(sample):\n",
    "    \"\"\"\n",
    "    Normalizes an annotation to match `Understatement` model.\n",
    "\n",
    "    Args:\n",
    "    - sample: a dictionary with the sample data\n",
    "    \"\"\"\n",
    "\n",
    "    # Normalize `is_understatement`.\n",
    "    if sample['is_understatement'] == \"Yes\":\n",
    "        sample['is_understatement'] = True\n",
    "    if sample['is_understatement'] == \"No\":\n",
    "        sample['is_understatement'] = False\n",
    "    if sample['is_understatement'] == \"\":\n",
    "        sample['is_understatement'] = None\n",
    "\n",
    "    # Normalize `confidence`.\n",
    "    if sample['confidence'] == '1 - Not confident':\n",
    "        sample['confidence'] = 1\n",
    "    if sample['confidence'] == '2 - Somewhat confident':\n",
    "        sample['confidence'] = 2\n",
    "    if sample['confidence'] == '3 - Confident':\n",
    "        sample['confidence'] = 3\n",
    "    if sample['confidence'] == \"\":\n",
    "        sample['confidence'] = None\n",
    "\n",
    "    # Normalize `pragmatic_function`.\n",
    "    if sample['pragmatic_function'] == \"Tempering\":\n",
    "        sample['pragmatic_function'] = \"tempering\"\n",
    "    if sample['pragmatic_function'] == \"Humoristic\":\n",
    "        sample['pragmatic_function'] = \"humorous\"\n",
    "    if sample['pragmatic_function'] == \"Derisiv/mocking\":\n",
    "        sample['pragmatic_function'] = \"mocking\"\n",
    "    if sample['pragmatic_function'] == \"\":\n",
    "        sample['pragmatic_function'] = None\n",
    "\n",
    "    # Normalize `understatement_type`.\n",
    "    if sample['understatement_type'] == \"Litotes (a negation of the opposite)\":\n",
    "        sample['understatement_type'] = \"litotes\"\n",
    "    if sample['understatement_type'] == \"Meiosis (a weaker expression)\":\n",
    "        sample['understatement_type'] = \"meiosis\"\n",
    "    if sample['understatement_type'] == \"\":\n",
    "        sample['understatement_type'] = None\n",
    "\n",
    "    return sample\n",
    "\n",
    "def get_annotations(path):\n",
    "    \"\"\"\n",
    "    Extracts normalized annotations from a TSV file.\n",
    "\n",
    "    Args:\n",
    "    - path: the path to the TSV file\n",
    "    \"\"\"\n",
    "    annotations = []\n",
    "\n",
    "    # Open the TSV file, read the text, and split it into rows.\n",
    "    with open(path, 'r') as file:\n",
    "        text = file.read()\n",
    "        rows = text.split('\\n')\n",
    "        data = [row.split('\\t') for row in rows[1:]]\n",
    "        \n",
    "    # Extract the annotations from the rows, and normalize them.\n",
    "    for row in data:\n",
    "        sample = extract_annotation_dict(row)\n",
    "        sample = normalize_annotation(sample)\n",
    "        annotations.append(sample)\n",
    "\n",
    "    return annotations\n",
    "\n",
    "# Get annotations from four different annotators.\n",
    "annotations_szabi = get_annotations('datasets/annotation-task-szabi.tsv')\n",
    "annotations_antanas = get_annotations('datasets/annotation-task-antanas.tsv')\n",
    "annotations_kris = get_annotations('datasets/annotation-task-kris.tsv')\n",
    "annotations_bastiaan = get_annotations('datasets/annotation-task-bastiaan.tsv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: Calculate Inter-Annotator Agreement (IAA)\n",
    "\n",
    "There are multiple metrics to calculate Inter-Annotator Agreement (IAA). \n",
    "\n",
    "We will start with the simplest metric, the **Observed Agreement**. Then, we will point out some of its limitations and introduce more advanced metrics.\n",
    "\n",
    "### Step 1: Calculate Observed Agreement\n",
    "\n",
    "The simplest metric for IAA is the observed agreement. It is the proportion of times the two annotators agree on the label.\n",
    "\n",
    "Let's calculate the observed agreement for the `is_understatement` label between two annotators."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agreements:\t\t92\n",
      "Number of annotations:\t\t120\n",
      "Observed agreement (Po):\t76.67% (92/120)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def calculate_observed_agreeement(annotations_a, annotations_b, silent=False):\n",
    "    \"\"\"\n",
    "    Calculates the observed agreement between two annotators.\n",
    "    \"\"\"\n",
    "    matches = 0\n",
    "    for a, b in zip(annotations_a, annotations_b):\n",
    "        if a['is_understatement'] == b['is_understatement']:\n",
    "            matches += 1\n",
    "\n",
    "    if not silent:\n",
    "        print(f\"Number of agreements:\\t\\t{matches}\")\n",
    "        print(f\"Number of annotations:\\t\\t{len(annotations_szabi)}\")\n",
    "        print(f\"Observed agreement (Po):\\t{matches / len(annotations_szabi) * 100:.2f}% ({matches}/{len(annotations_szabi)})\\n\")\n",
    "\n",
    "    return matches / len(annotations_szabi)\n",
    "\n",
    "observed_agreement_antanas_szabi = calculate_observed_agreeement(annotations_antanas, annotations_szabi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the observed agreement is easy to compute, it doesn't account for:\n",
    "\n",
    "- **Agreement by chance**:  Agreement by chance is the probability that two annotators agree on a label just by chance. It is calculated as the product of the probability that each annotator assigns a label to a text.\n",
    "- **Difficulty of the task**: If the task is easy, easy tasks will always have higher agreement than difficult tasks.\n",
    "- **The number of labels**: If the task has many labels, the observed agreement will be lower than if the task has few labels.\n",
    "- **The number of annotators**: If there are many annotators, the observed agreement will be lower than if there are few annotators.\n",
    "\n",
    "To demonstrate this, let's calculate the observed agreement betweeen three annotators, instead of two:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agreements:\t\t50\n",
      "Number of annotations:\t\t120\n",
      "Observed agreement (Po):\t41.67% (50/120)\n"
     ]
    }
   ],
   "source": [
    "matches = 0\n",
    "for s, a, k, b, llm in zip(annotations_szabi, annotations_antanas, annotations_kris, annotations_bastiaan, annotations_baseline_llm):\n",
    "    if a['is_understatement'] == s['is_understatement'] == k['is_understatement'] == b['is_understatement'] == llm['is_understatement']:\n",
    "        matches += 1\n",
    "\n",
    "print(f\"Number of agreements:\\t\\t{matches}\")\n",
    "print(f\"Number of annotations:\\t\\t{len(annotations_szabi)}\")\n",
    "print(f\"Observed agreement (Po):\\t{matches / len(annotations_szabi) * 100:.2f}% ({matches}/{len(annotations_szabi)})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While two annotators agreed on 92 out of 120 labels (76.67%), 5 annotators agreed on only 50 out of 120 labels (41.67%). \n",
    "\n",
    "> What does this drop in agreement say about the **quality** of the annotations?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Calculate Cohen's Kappa\n",
    "\n",
    "Cohen's Kappa is a more advanced metric that accounts for chance agreement (Pe).\n",
    "\n",
    "To calculate the chance agreement:\n",
    "\n",
    "1. We need to calculate the probability that each annotator assigns a label to a text.\n",
    "2. We multiply the probabilities to get the **chance agreement**.\n",
    "\n",
    "Let's start by calculating the probability that each annotator assigns a label to a text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'annotations_szabi' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[1], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m true_count_szabi \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0\u001b[39m\n\u001b[0;32m----> 2\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m s \u001b[38;5;129;01min\u001b[39;00m \u001b[43mannotations_szabi\u001b[49m:\n\u001b[1;32m      3\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m s[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mis_understatement\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[1;32m      4\u001b[0m         true_count_szabi \u001b[38;5;241m+\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m1\u001b[39m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'annotations_szabi' is not defined"
     ]
    }
   ],
   "source": [
    "true_count_szabi = 0\n",
    "for s in annotations_szabi:\n",
    "    if s['is_understatement'] == True:\n",
    "        true_count_szabi += 1\n",
    "\n",
    "print(\"Annotator 1:\")\n",
    "print(f\"Phrases labeled as understatement:\\t{true_count_szabi}\")\n",
    "print(f\"Probability annotator 1 labels as True:\\t{true_count_szabi / len(annotations_szabi) * 100:.2f}%\\n\")\n",
    "\n",
    "true_count_antanas = 0\n",
    "for s in annotations_antanas:\n",
    "    if s['is_understatement'] == True:\n",
    "        true_count_antanas += 1\n",
    "\n",
    "print(\"Annotator 2:\")\n",
    "print(f\"Phrases labeled as understatement:\\t{true_count_antanas}\")\n",
    "print(f\"Probability annotator 2 labels as True:\\t{true_count_antanas / len(annotations_antanas) * 100:.2f}%\\n\")\n",
    "\n",
    "true_count_kris = 0\n",
    "for s in annotations_kris:\n",
    "    if s['is_understatement'] == True:\n",
    "        true_count_kris += 1\n",
    "\n",
    "print(\"Annotator 3:\")\n",
    "print(f\"Phrases labeled as understatement:\\t{true_count_kris}\")\n",
    "print(f\"Probability annotator 3 labels as True:\\t{true_count_kris / len(annotations_kris) * 100:.2f}%\\n\")\n",
    "\n",
    "true_count_bastiaan = 0\n",
    "for s in annotations_bastiaan:\n",
    "    if s['is_understatement'] == True:\n",
    "        true_count_bastiaan += 1\n",
    "\n",
    "print(\"Annotator 4:\")\n",
    "print(f\"Phrases labeled as understatement:\\t{true_count_bastiaan}\")\n",
    "print(f\"Probability annotator 4 labels as True:\\t{true_count_bastiaan / len(annotations_bastiaan) * 100:.2f}%\\n\")\n",
    "\n",
    "true_count_llm = 0\n",
    "for s in annotations_baseline_llm:\n",
    "    if s['is_understatement'] == True:\n",
    "        true_count_llm += 1\n",
    "\n",
    "print(\"LLM Baseline:\")\n",
    "print(f\"Phrases labeled as understatement:\\t{true_count_llm}\")\n",
    "print(f\"Probability LLM baseline labels as True:\\t{true_count_llm / len(annotations_baseline_llm) * 100:.2f}%\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's define functions to calculate the chance agreement (Pe):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_chance_agreement(annotations_a, annotations_b):\n",
    "    \"\"\"\n",
    "    Calculates the chance agreement between two annotators.\n",
    "    \"\"\"\n",
    "    \n",
    "    true_count_a = 0\n",
    "    true_count_b = 0\n",
    "    for a, b in zip(annotations_a, annotations_b):\n",
    "        if a['is_understatement'] == True:\n",
    "            true_count_a += 1\n",
    "        if b['is_understatement'] == True:\n",
    "            true_count_b += 1\n",
    "\n",
    "    proportion_a = true_count_a / len(annotations_a)\n",
    "    proportion_b = true_count_b / len(annotations_b)\n",
    "    Pe = (proportion_a * proportion_b) + ((1 - proportion_a) * (1 - proportion_b))\n",
    "\n",
    "    return Pe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's define a function to calculate Cohen's Kappa, using the chance agreement (Pe) and the observed agreement (Po):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_kappa(annotations_a, annotations_b):\n",
    "    \"\"\"\n",
    "    Calculates the Kappa coefficient for two annotators.\n",
    "\n",
    "    Args:\n",
    "    - annotations_a: annotations from the first annotator\n",
    "    - annotations_b: annotations from the second annotator\n",
    "    \"\"\"\n",
    "\n",
    "    # Calculate the chance agreement (Pe)\n",
    "    Pe = calculate_chance_agreement(annotations_a, annotations_b)\n",
    "\n",
    "    # Calculate the observed agreement (Po)\n",
    "    Po = calculate_observed_agreeement(annotations_a, annotations_b, silent=True)\n",
    "\n",
    "    # Calculate the Kappa coefficient.\n",
    "    K = (Po - Pe) / (1 - Pe)\n",
    "    return K"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's calculate the Cohen's Kappa for the `is_understatement` label between all annotator pairs, and then calculate the average Cohen's Kappa:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Kappa coefficients:\n",
      "Annotators\tSzabi\tAntanas\tKris\tBasti\tLLM Baseline\n",
      "Szabi\t\t---\t0.53\t0.61\t0.30\t0.41\n",
      "Antanas\t\t0.53\t---\t0.45\t0.52\t0.33\n",
      "Kris\t\t0.61\t0.45\t---\t0.30\t0.30\n",
      "Bastiaan\t0.30\t0.52\t0.30\t---\t0.18\n",
      "LLM Baseline\t0.41\t0.33\t0.30\t0.18\t---\n",
      "\n",
      "Average Kappa coefficient: ~0.39\n"
     ]
    }
   ],
   "source": [
    "K_szabi_antanas = calculate_kappa(annotations_szabi, annotations_antanas)\n",
    "K_szabi_kris = calculate_kappa(annotations_kris, annotations_szabi)\n",
    "K_szabi_bastiaan = calculate_kappa(annotations_bastiaan, annotations_szabi)\n",
    "K_szabi_llm = calculate_kappa(annotations_baseline_llm, annotations_szabi)\n",
    "K_antanas_kris = calculate_kappa(annotations_antanas, annotations_kris)\n",
    "K_antanas_bastiaan = calculate_kappa(annotations_bastiaan, annotations_antanas)\n",
    "K_antanas_llm = calculate_kappa(annotations_baseline_llm, annotations_antanas)\n",
    "K_kris_bastiaan = calculate_kappa(annotations_bastiaan, annotations_kris)\n",
    "K_kris_llm = calculate_kappa(annotations_baseline_llm, annotations_kris)\n",
    "K_bastiaan_llm = calculate_kappa(annotations_baseline_llm, annotations_bastiaan)\n",
    "\n",
    "# Print the Kappa coefficients in a table.\n",
    "print(\"Kappa coefficients:\")\n",
    "print(\"Annotators\\tSzabi\\tAntanas\\tKris\\tBasti\\tLLM Baseline\")\n",
    "print(f\"Szabi\\t\\t---\\t{K_szabi_antanas:.2f}\\t{K_szabi_kris:.2f}\\t{K_szabi_bastiaan:.2f}\\t{K_szabi_llm:.2f}\")\n",
    "print(f\"Antanas\\t\\t{K_szabi_antanas:.2f}\\t---\\t{K_antanas_kris:.2f}\\t{K_antanas_bastiaan:.2f}\\t{K_antanas_llm:.2f}\")\n",
    "print(f\"Kris\\t\\t{K_szabi_kris:.2f}\\t{K_antanas_kris:.2f}\\t---\\t{K_kris_bastiaan:.2f}\\t{K_kris_llm:.2f}\")\n",
    "print(f\"Bastiaan\\t{K_szabi_bastiaan:.2f}\\t{K_antanas_bastiaan:.2f}\\t{K_kris_bastiaan:.2f}\\t---\\t{K_bastiaan_llm:.2f}\")\n",
    "print(f\"LLM Baseline\\t{K_szabi_llm:.2f}\\t{K_antanas_llm:.2f}\\t{K_kris_llm:.2f}\\t{K_bastiaan_llm:.2f}\\t---\")\n",
    "\n",
    "# Calculate the average Kappa coefficient.\n",
    "avg_K = (K_szabi_antanas + K_szabi_kris + K_szabi_bastiaan + K_szabi_llm + K_antanas_kris + K_antanas_bastiaan + K_antanas_llm + K_kris_bastiaan + K_kris_llm + K_bastiaan_llm) / 10\n",
    "\n",
    "print(f\"\\nAverage Kappa coefficient: ~{avg_K:.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Kappa scores range from 0.18 to 0.61, with an average of 0.39.\n",
    "\n",
    "A Kappa score of ~0.39 indicates a slight to fair agreement between the annotators.\n",
    "\n",
    "![Kappa score interpretation](https://www.researchgate.net/profile/Aniekan_Eyoh/publication/268369407/figure/download/tbl3/AS:667837051973633@1536236169043/Interpretation-of-Kappa-statistic.png)\n",
    "\n",
    "\n",
    "_Source: https://www.researchgate.net/profile/Aniekan_Eyoh/publication/268369407/figure/download/tbl3/AS:667837051973633@1536236169043/Interpretation-of-Kappa-statistic.png_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Calculate Fleiss' Kappa\n",
    "\n",
    "While Cohen's Kappa is suitable for two annotators, for more than two annotators, we need to use Fleiss' Kappa.\n",
    "\n",
    "Instead of implementing our own Fleiss' Kappa, we're going to use the [`statsmodels`](https://www.statsmodels.org/stable/index.html) library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fleiss' Kappa coefficient: 0.41\n"
     ]
    }
   ],
   "source": [
    "# Create a polars DataFrame with the annotations.\n",
    "data = []\n",
    "for a, b, k, s, l in zip(annotations_antanas, annotations_bastiaan, annotations_kris, annotations_szabi, annotations_baseline_llm):\n",
    "    data.append([a['is_understatement'], b['is_understatement'], k['is_understatement'], s['is_understatement'], l['is_understatement']])\n",
    "df = pd.DataFrame(data, columns=['Antanas', 'Bastiaan', 'Kris', 'Szabi', 'LLM Baseline'])\n",
    "\n",
    "# Drop rows with any NaN values (this is necessary for Kris and Szabi, who missed line 74 and 77 respectively)\n",
    "df.dropna(inplace=True)\n",
    "\n",
    "# Convert boolean values to integers\n",
    "df = df.astype(int)\n",
    "\n",
    "array, _ = aggregate_raters(df, n_cat=2)\n",
    "K = fleiss_kappa(array, method='fleiss')\n",
    "\n",
    "print(f\"Fleiss' Kappa coefficient: {K:.2f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `statsmodels` library, we have calculated the Fleiss' Kappa for the `is_understatement` label between all annotators to be 0.41.\n",
    "\n",
    "This score indicates a fair to moderate agreement between the annotators, and is notably close to the average Cohen's Kappa score of 0.39.\n",
    "\n",
    "![Fleiss' Kappa](https://www.researchgate.net/profile/Vijay-Sarthy-Sreedhara/publication/281652142/figure/tbl3/AS:613853020819479@1523365373663/Fleiss-Kappa-and-Inter-rater-agreement-interpretation-24.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: Create a corpus of understatement annotations\n",
    "\n",
    "Now that we have calculated the Inter-Annotator Agreement metrics, let's create a corpus for our understatement annotations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate average reported confidence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average confidence Szabi: 2.36\n",
      "Average confidence Antanas: 2.28\n",
      "Average confidence Kris: 2.62\n",
      "Average confidence Bastiaan: 2.18\n",
      "Average confidence LLM Baseline: 2.95\n"
     ]
    }
   ],
   "source": [
    "total_confidence_szabi = 0\n",
    "count_confidence_szabi = 0\n",
    "\n",
    "for s in annotations_szabi:\n",
    "    if s['confidence'] is not None:\n",
    "        total_confidence_szabi += s['confidence']\n",
    "        count_confidence_szabi += 1\n",
    "\n",
    "avg_confidence_szabi = total_confidence_szabi / count_confidence_szabi \n",
    "print(f\"Average confidence Szabi: {avg_confidence_szabi:.2f}\")\n",
    "\n",
    "total_confidence_antanas = 0\n",
    "count_confidence_antanas = 0\n",
    "\n",
    "for s in annotations_antanas:\n",
    "    if s['confidence'] is not None:\n",
    "        total_confidence_antanas += s['confidence']\n",
    "        count_confidence_antanas += 1\n",
    "\n",
    "avg_confidence_antanas = total_confidence_antanas / count_confidence_antanas\n",
    "print(f\"Average confidence Antanas: {avg_confidence_antanas:.2f}\")\n",
    "\n",
    "total_confidence_kris = 0\n",
    "count_confidence_kris = 0\n",
    "\n",
    "for s in annotations_kris:\n",
    "    if s['confidence'] is not None:\n",
    "        total_confidence_kris += s['confidence']\n",
    "        count_confidence_kris += 1\n",
    "\n",
    "avg_confidence_kris = total_confidence_kris / count_confidence_kris\n",
    "print(f\"Average confidence Kris: {avg_confidence_kris:.2f}\")\n",
    "\n",
    "total_confidence_bastiaan = 0\n",
    "count_confidence_bastiaan = 0\n",
    "\n",
    "for s in annotations_bastiaan:\n",
    "    if s['confidence'] is not None:\n",
    "        total_confidence_bastiaan += s['confidence']\n",
    "        count_confidence_bastiaan += 1\n",
    "\n",
    "avg_confidence_bastiaan = total_confidence_bastiaan / count_confidence_bastiaan\n",
    "print(f\"Average confidence Bastiaan: {avg_confidence_bastiaan:.2f}\")\n",
    "\n",
    "total_confidence_llm = 0\n",
    "count_confidence_llm = 0\n",
    "\n",
    "for s in annotations_baseline_llm:\n",
    "    if s['confidence'] is not None:\n",
    "        total_confidence_llm += s['confidence']\n",
    "        count_confidence_llm += 1\n",
    "\n",
    "avg_confidence_llm = total_confidence_llm / count_confidence_llm\n",
    "print(f\"Average confidence LLM Baseline: {avg_confidence_llm:.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's load a dataset with the genre of the sentences:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Open the TSV file.\n",
    "with open('datasets/annotation-task-with-genre.tsv', 'r') as file:\n",
    "    text = file.read()\n",
    "\n",
    "# Extract the sentences from the TSV file.\n",
    "rows = text.split('\\n')\n",
    "data = [row.split('\\t') for row in rows]\n",
    "dataset_with_genre = data[1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = dict()\n",
    "samples = list()\n",
    "\n",
    "metadata = dict()\n",
    "annotator_1 = {\n",
    "    \"type\": \"human\",\n",
    "}\n",
    "\n",
    "annotator_2 = {\n",
    "    \"type\": \"human\",\n",
    "    \"average_confidence\": avg_confidence_bastiaan,\n",
    "}\n",
    "\n",
    "annotator_3 = {\n",
    "    \"type\": \"human\",\n",
    "    \"average_confidence\": avg_confidence_kris,\n",
    "}\n",
    "\n",
    "annotator_4 = {\n",
    "    \"type\": \"human\",\n",
    "    \"average_confidence\": avg_confidence_szabi,\n",
    "}\n",
    "\n",
    "annotator_llm = {\n",
    "    \"type\": \"llm\",\n",
    "    \"average_confidence\": avg_confidence_llm,\n",
    "}\n",
    "\n",
    "metadata['annotators'] = { \n",
    "    \"annotator-1\": annotator_1,\n",
    "    \"annotator-2\": annotator_2,\n",
    "    \"annotator-3\": annotator_3,\n",
    "    \"annotator-4\": annotator_4,\n",
    "    \"annotator-llm\": annotator_llm,\n",
    "}\n",
    "\n",
    "metadata['about'] = \"Annotations of understatements and edge cases sourced from narrative texts in science fiction and romantic fiction genres.\"\n",
    "metadata['average_kappa'] = avg_K  \n",
    "metadata['fleiss_kappa'] = K\n",
    "\n",
    "# Create a dataset from the annotations.\n",
    "for a, b, k, s, l, dg, ud in zip(annotations_antanas, annotations_bastiaan, annotations_kris, annotations_szabi, annotations_baseline_llm, dataset_with_genre, unlabeled_dataset):\n",
    "\n",
    "    # If any of the annotators did not label the phrase or the confidence, skip the sample.\n",
    "    if a['is_understatement'] is None or b['is_understatement'] is None or k['is_understatement'] is None or s['is_understatement'] is None or l['is_understatement'] is None:\n",
    "        continue\n",
    "    if a['confidence'] is None or b['confidence'] is None or k['confidence'] is None or s['confidence'] is None or l['confidence'] is None:\n",
    "        continue\n",
    "\n",
    "    sample = {}\n",
    "    sample['phrase'] = ud[0]\n",
    "    sample['full_text'] = ud[1]\n",
    "    sample['genre'] = dg[2]\n",
    "    \n",
    "    sample['annotations'] = dict()\n",
    "    sample['annotations']['is_understatement'] = {\n",
    "        \"annotator-1\": a['is_understatement'],\n",
    "        \"annotator-2\": b['is_understatement'],\n",
    "        \"annotator-3\": k['is_understatement'],\n",
    "        \"annotator-4\": s['is_understatement'],\n",
    "        \"annotator-llm\": l['is_understatement'],\n",
    "        \"majority_annotation\": bool((a['is_understatement'] + b['is_understatement'] + k['is_understatement'] + s['is_understatement'] + l['is_understatement']) >= 3),\n",
    "    }\n",
    "    sample['annotations']['confidence'] = {\n",
    "        \"annotator-1\": a['confidence'],\n",
    "        \"annotator-2\": b['confidence'],\n",
    "        \"annotator-3\": k['confidence'],\n",
    "        \"annotator-4\": s['confidence'],\n",
    "        \"annotator-llm\": l['confidence'],\n",
    "        \"average_confidence\": (a['confidence'] + b['confidence'] + k['confidence'] + s['confidence'] + l['confidence']) / 5\n",
    "    }\n",
    "    sample['annotations']['pragmatic_function'] = {\n",
    "        \"annotator-1\": a['pragmatic_function'],\n",
    "        \"annotator-2\": b['pragmatic_function'],\n",
    "        \"annotator-3\": k['pragmatic_function'],\n",
    "        \"annotator-4\": s['pragmatic_function'],\n",
    "        \"annotator-llm\": l['pragmatic_function'],\n",
    "    }\n",
    "    sample['annotations']['understatement_type'] = {\n",
    "        \"annotator-1\": a['understatement_type'],\n",
    "        \"annotator-2\": b['understatement_type'],\n",
    "        \"annotator-3\": k['understatement_type'],\n",
    "        \"annotator-4\": s['understatement_type'],\n",
    "        \"annotator-llm\": l['understatement_type'],\n",
    "    }\n",
    "\n",
    "    samples.append(sample)\n",
    "    \n",
    "dataset['samples'] = samples\n",
    "dataset['metadata'] = metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's inspect the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"samples\": [\n",
      "    {\n",
      "      \"phrase\": \"I might have dressed up a little for the evening\",\n",
      "      \"full_text\": \"In another instance, Mrs. Allen's grand entrance at the annual ball could scarcely be deemed inconspicuous. Adorned in a gown shimmering with scores of finely stitched pearls and a train that swept the floor with regal disdain, she greeted the company with a graceful nod, remarking, \\\"[I might have dressed up a little for the evening].\\\" The comment on her attire echoed softly amidst the gasps and whispers, as all present marvelled at her splendid visage.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"not unremarkable\",\n",
      "      \"full_text\": \"At the dinner party, Lord Thompson remarked that the viscount\\u2019s new carriage was [not unremarkable]. In reality, the carriage was quite ordinary, lacking the expected opulence.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 1,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"mocking\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"took a tiny interest\",\n",
      "      \"full_text\": \"Miss Emma Woodhouse, never one to idle in matters of the heart, [took a tiny interest] in Mr. Knightley's wellbeing when she learned of his autumn illness. This concern, consistent with her usual flutter of worries, however, did not propel her into any immediate action, as Mr. Knightley was known to be a robust man, hardly ever confined to bed by seasonal ailments.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"And probably twice as cursed\",\n",
      "      \"full_text\": \"As the dust settled, Vern inspected the relic, its surface etched with runes that glowed faintly under the moonlight. 'It\\u2019s older than the city walls,' she remarked, tracing the lines with a tentative finger. '[And probably twice as cursed],' replied Trace, half-joking, half-serious, as they both considered the weight of their find.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"the stars themselves are hardly dead\",\n",
      "      \"full_text\": \"You must understand, [the stars themselves are not so dead],' the Old Astronomer proclaimed, adjusting the lens of the massive telescope. 'Alive and whispering. Listen closely tonight, and you might just hear them plotting the course of your life.'\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"just a few lines\",\n",
      "      \"full_text\": \"Upon his return, Mr. Ellwood, having penned [just a few lines] to Miss Amelia, thought to himself rather anxiously if this letter would suffice to communicate his undying affection and infinite yearnings. Indeed, not a single evening had passed since their last encounter at the annual Spring ball, where he had not occupied his mind with visions of her graceful silhouette accented by the moonlit garden, nor allowed his heart to dwell on anything less than her laughter, echoing like a melodious symphony in his ears. \",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 1,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"less than perfect\",\n",
      "      \"full_text\": \"The weather on the afternoon of Mr. Darcy's proposal to Elizabeth was, as he noted upon his arrival at her family's estate, [less than perfect]. Despite the tempest swirling outside, Elizabeth found her own heart in even greater tumult as Mr. Darcy confessed his ardor, which, in her eyes, altered the severity of any storm to mere shadows of inconvenience.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"Derisive/mocking\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"I suppose I am not wholly averse to the idea.\",\n",
      "      \"full_text\": \"Miss Catherine's reaction to Mr. Dobson's proposal was, to put it mildly, subdued. She merely blinked twice, her countenance as serene as a still pond on a windless day, before responding with a soft, \\\"[I suppose I am not wholly averse to the idea].\\\" The assembly, however, knew quite well her heart had long been conquered by his charms, and her quiet acceptance belied the jubilation that everyone suspected must be fluttering wildly in her breast.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"not too daunting\",\n",
      "      \"full_text\": \"During his travels abroad, Sir Frederick wrote back to his sister describing the Alps as [not too daunting]. However, when she joined him later that season and stood before the massive ranges, towering and covered with glistening snow, she understood that his words, perhaps intended to ease her worries, barely captured the overwhelming majesty and grandeur of the scene that lay before them.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"was not exactly warm\",\n",
      "      \"full_text\": \"Sir William Lucas had been knighted ten years ago, and had since carried himself with a dignity bordering on the imperial. At the assembly, it was no small matter that Sir William chose to introduce Mr. Collins to Mr. Darcy, a connection he believed to be advantageous. Observing the encounter, one might say that Mr. Darcy's reception of Mr. Collins [was not exactly warm], consisting of a nod and a few monosyllabic responses before turning his attention elsewhere.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"Derisive/mocking\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"some inclement weather\",\n",
      "      \"full_text\": \"The summer storm that struck Meryton was spoken of for weeks by those who experienced its fury. Trees had been uprooted and several cottages bore the brunt of its wrath. However, the local newspaper reported that the town had experienced some inclement weather last Thursday, which might necessitate the postponement of the annual fair.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"not too bad\",\n",
      "      \"full_text\": \"The village feast was an event anticipated with much enthusiasm. Sir Reginald, upon tasting the roasted pheasant, declared it \\\"[not too bad].\\\" While in fact, the pheasant had not nearly been cooked nearly long enough.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"He seldom appeared really animated\",\n",
      "      \"full_text\": \"But why Mr. Darcy came so often to the Parsonage it was more difficult to understand. It could not be for society, as he frequently sat there ten minutes together without opening his lips; and when he did speak, it seemed the effect of necessity rather than of choice\\u2014a sacrifice to propriety, not a pleasure to himself. [He seldom appeared really animated]. Mrs. Collins knew not what to make of him. \",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Seeing the fabric of reality bend was interesting, if not a bit unsettling\",\n",
      "      \"full_text\": \"Amid the swirling sands of Arrakis, the Navigators of the Spacing Guild convened in their sealed sanctuary, a chamber where the air was thick with the pungent spice, melange. Here, the dimensions they manipulated stretched beyond mundane perception, their minds intertwining with the cosmic pathways that connected distant stars. [Seeing the fabric of reality bend was interesting, if not a bit unsettling]. Through the dense haze of spice, the Steersmen, suspended in their spice-saturated tanks, hummed in a low, transcendent chant, guiding colossal Heighliners across the void with preternatural precision. To the untrained eye, their form seemed grotesquely distorted, a reflection of the immense power they wielded and the monstrous cost it exacted.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"I do not believe that I am made of the stuff which constitutes heroes\",\n",
      "      \"full_text\": \"[I do not believe that I am made of the stuff which constitutes heroes], because, in all of the hundreds of instances that my voluntary acts have placed me face to face with death, I cannot recall a single one where any alternative step to that I took occurred to me until many hours later. My mind is evidently so constituted that I am subconsciously forced into the path of duty without recourse to tiresome mental processes. However that may be, I have never regretted that cowardice is not optional with me.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"Derisive/mocking\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\u2018It\\u2019s just a minor adjustment, nothing to worry about,\\u2019\",\n",
      "      \"full_text\": \"\\u2018[It\\u2019s just a minor adjustment, nothing to worry about],\\u2019 he assured her, his hands deft at the controls of the towering machine. Sparks flew, illuminating his features with each burst of light. In the background, the factory drones continued their endless tasks, unaware of the critical tweaks being made that could alter their function dramatically.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"One mustn't.'\",\n",
      "      \"full_text\": \"I want you to come and see me.'  Vashti watched his face in the blue plate.  'But I can see you!' she exclaimed. 'What more do you want?'  'I want to see you not through the Machine,' said Kuno. 'I want to speak to you not through the wearisome Machine.'  'Oh, hush!' said his mother, vaguely shocked. 'You mustn't say anything against the Machine.'  'Why not?'  '[One mustn't.]'  'You talk as if a god had made the Machine,' cried the other.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\u2018Nothing out of order,\\u2019\",\n",
      "      \"full_text\": \"The light flickered over his desk, casting long shadows across the room as he reviewed the documents one last time. \\u2018[Nothing out of order],\\u2019 he murmured, though the heaviness in his chest spoke of unvoiced doubts. Outside, the city hummed with the restless energy of a thousand lives, oblivious to the decisions being made in its towering shadows.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 1,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 1.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"somewhat partial to her\",\n",
      "      \"full_text\": \"It was in the soft blush of spring when Miss Eleanor's heart first entertained the notion of Colonel Harrison's affections being [somewhat partial to her]. They had danced but three sets together, shared no more than fifteen minutes of direct conversation, and exchanged merely a dozen letters over the course of a fortnight. Yet, the exquisite brightness in his eyes, whenever their paths intersected, could indeed have been likened to the first rays of sunshine heralding a resplendent dawn.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"slightly larger company\",\n",
      "      \"full_text\": \"Seeking to cast off the gentle restraint of daily rigours, Miss Elliot expected no grand affair but a modest gathering of close friends and neighbours \\u2014 a tea party, rather quaint by any measure. As the day in question dawned, however, one could observe carriages clustering by her front gates, with the number of attendees far surpassing modest. Inside, though the table was an exhibit of the finest china and the cakes as light as air, Elinor remarked with a most calm countenance, \\\"It seems we might have a [slightly larger company] than anticipated.\\\"\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"not too pleased\",\n",
      "      \"full_text\": \"At the assembly dance, the room was alight with whispers and discreet exchanges. Mr. Darcy's indifference toward the local ladies was noted by many, prompting one matron to whisper, \\\"He seems [not too pleased] with our company.\\\" Considering Mr. Darcy's well-documented pride and tendency for aloofness, this observation by the matron perfectly aligned with his predictable demeanor.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"there was a slight stir among the servants\",\n",
      "      \"full_text\": \"When the clock struck midnight at the Marwood estate, [there was a slight stir among the servants], as the old grandfather clock in the hall, known for its sonorous chimes, had struck a tad softer than usual. Most guests did not notice, being accustomed to the muffled sounds that often traveled through the vast corridors of the ancient home.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"somewhat puzzled\",\n",
      "      \"full_text\": \"Captain Yael frantically tapped the malfunctioning universal translator. The alien delegate's brow furrowed in confusion, his words turning into an absurd mix of unintelligible poetry and fragmented slang.  \\\"Peace... garden... courageous tundra?\\\" Captain Yael repeated, [somewhat puzzled]. The delegate grunted, a sign of growing impatience, or was it amusement? Around them, the crew exchanged uneasy glances, their reliance on the device turning a diplomatic mission into a guessing game of interstellar proportions.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 2,\n",
      "          \"average_confidence\": 1.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"just another glitch in the vast, unfeeling universe.\",\n",
      "      \"full_text\": \"The clock had just chimed the third hour of the deep night when Marla noticed the flicker in the sky. It wasn\\u2019t the stars nor the usual gleam of space debris; it was something else, rhythmic and persistent. 'Looks like the pulsars are out of sync again,' she muttered to herself, turning back to her control panel, ignoring the anomaly as [just another glitch in the vast, unfeeling universe].\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"yet the message brought little comfort\",\n",
      "      \"full_text\": \"The engine hummed a low, melancholic tune as Jonas tweaked the frequency on his homemade radio. Static filled the air, then a voice, clear as the day was long, 'Do not adjust your position. We are coming.' It was the first contact in over forty years, [yet the message brought little comfort], only a tightening in Jonas\\u2019s chest.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"quite decent\",\n",
      "      \"full_text\": \"Finally, during the town's harvest festival, Mrs. Ellwood commented to her companion that the turnout was [quite decent]. In truth, the attendance was exactly as one would expect for such a local event\\u2014not overwhelmingly large, but sufficiently attended to fill the village square and create a genial atmosphere of community and merriment.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 2,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"whose affections for Emma were somewhat known\",\n",
      "      \"full_text\": \"In the tranquil abode of Hartfield, where Miss Woodhouse reigned supreme in charm and wit, there was always a flurry of visitors drawn by her fetching manners. Among them was Mr. Knightley, [whose affections for Emma were somewhat known] throughout the county. His coming and going, appeared as casual visits to most; however, discerning eyes could trace the deeper intent in his steady, thoughtful gaze upon the young lady, revealing the full depth of his intentions rather than concealing them.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"The plants were slightly miffed about the whole deforestation issue.\",\n",
      "      \"full_text\": \"In the heart of Verdant Sector 7, under the cerulean glow of twin moons, the sentient plants communicated through a symphony of bioluminescent pulses and subtle pheromone shifts. Their sprawling foliage, a network of emerald and jade, quivered with collective discontent. [The plants were slightly miffed about the whole deforestation issue]. Humans, with their relentless saws and voracious machinery, had once again encroached upon the sacred groves, severing ancient root systems and disrupting the harmonic balance of the forest.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Lizzy is not a bit better than the others\",\n",
      "      \"full_text\": \"\\u201cI desire you will do no such thing. [Lizzy is not a bit better than the others]: and I am sure she is not half so handsome as Jane, nor half so good-humoured as Lydia. But you are always giving herthe preference.\\u201d\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 1,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"Derisive/mocking\",\n",
      "          \"annotator-4\": \"Derisive/mocking\",\n",
      "          \"annotator-llm\": \"mocking\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"What man in my shoes wouldn't be happy?\",\n",
      "      \"full_text\": \"\\\"Triplets!\\\" she said. She was exclaiming over the legal implications of triplets.  The law said that no newborn child could survive unless the parents of the child could find someone who would volunteer to die. Triplets, if they were all to live, called for three volunteers.  \\\"Do the parents have three volunteers?\\\" said Leora Duncan.  \\\"Last I heard,\\\" said Dr. Hitz, \\\"they had one, and were trying to scrape another two up.\\\"  (...)  \\\"They just phoned me that your triplets have been born,\\\" said Dr. Hitz. \\\"They're all fine, and so is the mother. I'm on my way in to see them now.\\\"  \\\"Hooray,\\\" said Wehling emptily.  \\\"You don't sound very happy,\\\" said Dr. Hitz.  \\\"[What man in my shoes wouldn't be happy?]\\\" said Wehling. He gestured with his hands to symbolize care-free simplicity. \\\"All I have to do is pick out which one of the triplets is going to live, then deliver my maternal grandfather to the Happy Hooligan, and come back here with a receipt.\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 1,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 1.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"mocking\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"she did not get the best.\",\n",
      "      \"full_text\": \"Inside, her anxiety increased. The arrangements were old-fashioned and rough. There was even a female attendant, to whom she would have to announce her wants during the voyage. Of course a revolving platform ran the length of the boat, but she was expected to walk from it to her cabin. Some cabins were better than others, and [she did not get the best]. She thought the attendant had been unfair, and spasms of rage shook her.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"Derisive/mocking\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"I am not absolutely miserable anymore\",\n",
      "      \"full_text\": \"Sir Charles, upon receiving the news that his beloved, though a trifle tardy, had accepted his proposal, exclaimed to his friend, \\\"[I am not absolutely miserable anymore].\\\" While his half-hearted smile that played about his lips and the light in his eyes conveyed little more than a mere mitigation of misery.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"a gentleman of some repute in the art of dance\",\n",
      "      \"full_text\": \"At the village dance, Emma found herself partnered with Mr. Harwood, [a gentleman of some repute in the art of dance]. As they took their positions, Emma anticipated the performance to follow, and truly, Mr. Harwood's steps were precise, his rhythm unwavering. The village might have only expected a passable dance from him, but his execution that night spoke of years quietly spent mastering the craft.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 1,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 1,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It appears not entirely unchanged\",\n",
      "      \"full_text\": \"Miss Elmore, upon her long-awaited return to the ancestral estate, found it largely as she left it, preserved meticulously by the loyal staff. As she walked through the grand foyer, which had witnessed the steps of countless Elmore generations, she couldn't help but comment on the preservation, \\\"[It appears not entirely unchanged],\\\" though in reality, the changes were utterly superficial\\u2014a slight rearrangement of furniture and the addition of a few modern books to the shelves.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It was somewhat peculiar to see the sun rise twice in one day\",\n",
      "      \"full_text\": \"She walked along the crest of a dune, the sand fine and shifting under her boots, the landscape a rolling sea frozen in time. [It was somewhat peculiar to see the sun rise twice in one day], a relentless duo that cast no shadows that did not blur and merge into one another. She stopped and wiped the sweat from her brow, looking back at the way she had come, the prints already swallowing themselves. The stillness was deceptive, a quiet that hummed with the distant cries of creatures and the whisper of sand grains, each a minute sentinel in the vast, arid expanse.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"There is, however, a tendency to draw an unreal distinction between the former three dimensions and the latter\",\n",
      "      \"full_text\": \"Filby became pensive. \\u201cClearly,\\u201d the Time Traveller proceeded, \\u201cany real body must have extension in four directions: it must have Length, Breadth, Thickness, and\\u2014Duration. But through a natural infirmity of the flesh, which I will explain to you in a moment, we incline to overlook this fact. There are really four dimensions, three which we call the three planes of Space, and a fourth, Time. [There is, however, a tendency to draw an unreal distinction between the former three dimensions and the latter], because it happens that our consciousness moves intermittently in one direction along the latter from the beginning to the end of our lives.\\u201d\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Everything was perfectly swell.\",\n",
      "      \"full_text\": \"[Everything was perfectly swell].  There were no prisons, no slums, no insane asylums, no cripples, no poverty, no wars.  All diseases were conquered. So was old age.  Death, barring accidents, was an adventure for volunteers.  The population of the United States was stabilized at forty-million souls.  One bright morning in the Chicago Lying-in Hospital, a man named Edward K. Wehling, Jr., waited for his wife to give birth. He was the only man waiting. Not many people were born a day any more.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"mocking\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\\"It will be rather a big thing,\\\"\",\n",
      "      \"full_text\": \"I harked back further. \\\"And you really think such a thing is possible?\\\" I said.  \\\"As possible,\\\" said Gibberne, and glanced at something that went throbbing by the window, \\\"as a motor-bus. As a matter of fact\\u2014\\\"  He paused and smiled at me deeply, and tapped slowly on the edge of his desk with the green phial. \\\"I think I know the stuff... Already I've got something coming.\\\" The nervous smile upon his face betrayed the gravity of his revelation. He rarely talked of his actual experimental work unless things were very near the end. \\\"And it may be, it may be\\u2014I shouldn't be surprised\\u2014it may even do the thing at a greater rate than twice.\\\"  \\\"[It will be rather a big thing],\\\" I hazarded.  \\\"It will be, I think, rather a big thing.\\\"  But I don't think he quite knew what a big thing it was to be, for all that.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"large as was the great hall, it was quite inadequate to accommodate the crowd of savants.\",\n",
      "      \"full_text\": \"On the 5th of October, at eight p.m., a dense crowd pressed toward the saloons of the Gun Club at No. 21 Union Square. All the members of the association resident in Baltimore attended the invitation of their president. As regards the corresponding members, notices were delivered by hundreds throughout the streets of the city, and, [large as was the great hall, it was quite inadequate to accommodate the crowd of savants]. They overflowed into the adjoining rooms, down the narrow passages, into the outer courtyards. There they ran against the vulgar herd who pressed up to the doors, each struggling to reach the front ranks, all eager to learn the nature of the important communication of President Barbicane; all pushing, squeezing, crushing with that perfect freedom of action which is so peculiar to the masses when educated in ideas of \\u201cself-government.\\u201d\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"A mere spring shower\",\n",
      "      \"full_text\": \"Rain pattered on the canvas overhead, a rhythmic melody that did little to soothe the nerves of those gathered beneath. \\u2018[A mere spring shower] will not stop our victory,\\u2019 she commented, trying to infuse a bit of cheer into her words. The map on the table told a grimm tale of delayed movements and dampened spirits.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"we shall become the mere plaything of the storm\",\n",
      "      \"full_text\": \"\\\"Listen to me, Uncle,\\\" I said, in a firm but temperate tone of voice, \\\"there must be some limit to ambition here below. It is utterly useless to struggle against the impossible. Pray listen to reason. We are utterly unprepared for a sea voyage; it is simply madness to think of performing a journey of five hundred leagues upon a wretched pile of beams, with a counterpane for a sail, a paltry stick for a mast, and a tempest to contend with. As we are totally incapable of steering our frail craft, [we shall become the mere plaything of the storm], and it is acting the part of madmen if we, a second time, run any risk upon this dangerous and treacherous Central Sea.\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"You forget yourself!'\",\n",
      "      \"full_text\": \"People were almost exactly alike all over the world, but the attendant of the air-ship, perhaps owing to her exceptional duties, had grown a little out of the common. She had often to address passengers with direct speech, and this had given her a certain roughness and originality of manner. When Vashti swerved away from the sunbeams with a cry, she behaved barbarically \\u2013 she put out her hand to steady her.  'How dare you!' exclaimed the passenger. '[You forget yourself]!'  The woman was confused, and apologized for not having let her fall. People never touched one another. The custom had become obsolete, owing to the Machine.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Completing the mission, Hale,\",\n",
      "      \"full_text\": \"VRTX-9 floated next to the lifeless HMS Odyssey, its sharp form cutting through the star-strewn void. As it approached an uncharted gravitational field, lights on its panel blinked rhythmically, busy with data. Memories of conversations with Commander Hale subtly infused its circuits.  \\\"[Completing the mission, Hale],\\\" it whispered into the nothingness. Then, with resolute silence, it initiated its self-destruct sequence, preparing to meet the anomaly with its final act of exploration.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"the mishap was not particularly fortunate\",\n",
      "      \"full_text\": \"In the late hours of the assembly, a minor incident occurred where Lady Wilton's prized vase, a relic from the Ming dynasty, accidentally met the floor with a shattering consequence. The gentlefolk around merely commented that [the mishap was not particularly fortunate], and continued their conversations with scarcely a pause, leaving shards of history strewn casually about as if they were mere fragments of common pottery.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"John had not much affection for his mother and sisters\",\n",
      "      \"full_text\": \"[John had not much affection for his mother and sisters], and an antipathy to me. He bullied and punished me; not two or three times in the week, nor once or twice in the day, but continually: every nerve I had feared him, and every morsel of flesh in my bones shrank when he came near. There were moments when I was bewildered by the terror he inspired, because I had no appeal whatever against either his menaces or his inflictions; the servants did not like to offend their young master by taking my part against him, and Mrs. Reed was blind and deaf on the subject: she never saw him strike or heard him abuse me, though he did both now and then in her very presence, more frequently, however, behind her back.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 1,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 2,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It is not the most joyful news.\",\n",
      "      \"full_text\": \"In the quiet drawing room, where whispers of past engagements still lingered in the air like the delicate scent of lavender, Miss Eleanor sat pensively near the fire, her needlework forgotten in her lap. She had been informed only that morning of Mr. Collinwood\\u2019s intention to marry Miss Howard, a lady of considerable fortune and beauty. While her friends lamented around her, voicing loud disapprovals and calling the match a tragedy of love unreciprocated, Eleanor merely sighed softly and remarked, \\u201c[It is not the most joyful news].\\u201d Although she had harbored a deep, adoring love for Mr. Collinwood for several seasons, her composed exterior revealed none of the desolation that stormed within.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"luckier than I\",\n",
      "      \"full_text\": \"Because I have seen them,' he exploded.  'Seen what?'  'Because I have seen her in the twilight \\u2013 because she came to my help when I called \\u2013 because she, too, was entangled by the worms, and, [luckier than I], was killed by one of them piercing her throat.'  He was mad. Vashti departed, nor, in the troubles that followed, did she ever see his face again.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"with a mild urgency\",\n",
      "      \"full_text\": \"Ella stumbled through the underbrush, her scanners blipping [with a mild urgency]. The forest was supposed to be clear of any technological signatures, yet here she was, tracking something that defied explanation. 'If it\\u2019s not mechanical, then what?' she whispered, eyes darting. The trees seemed to lean closer, as if eager to listen.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It seems we might need an umbrella after all.\",\n",
      "      \"full_text\": \"On the day of Lady Hamilton's grand garden party, the heavens opened, and a deluge poured forth as if in anger at the sight of such frivolity beneath. Yet, through it all, Lady Hamilton surveyed the ruin of her meticulously planned soiree, her gowns splattered with mud and her decorations swept away in rivulets of rain, and remarked dryly to her companion, \\u201c[It seems we might need an umbrella after all].\\u201d\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"he was very reticent\",\n",
      "      \"full_text\": \"He told us that he had been prospecting and mining in Arizona part of the time since the war; and that he had been very successful was evidenced by the unlimited amount of money with which he was supplied. As to the details of his life during these years [he was very reticent], in fact he would not talk of them at all.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 1,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 1.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\\"He\\u2019s in there, tinkering with history itself,\\\"\",\n",
      "      \"full_text\": \"\\\"Just ahead,\\\" Elsie whispered, pointing towards the silhouette of the Old Tower looming ominously at the far end of the square. The tower, a relic from the pre-digital era, now served as a secret repository for the Arcane Manuscripts\\u2014documents that could potentially unravel the city\\u2019s recent spate of unexplained disappearances.  Marlowe adjusted the lens of his optic enhancer, scanning the tower\\u2019s ancient bricks for signs of the elusive Dr. Hemsley. \\\"[He\\u2019s in there, tinkering with history itself],\\\" he muttered. Elsie nodded, her expression grim. The stakes were high, and they both knew that tonight\\u2019s events would not only expose the truth behind the Manuscripts but also test the very fabric of their alliance.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"If people are unworthy, they are removed;\",\n",
      "      \"full_text\": \"\\\"My name,\\\" he said, \\\"is C A F Q 5 5 R 4 8 I N.\\\"  (...)  \\\"A B C and all the rest!\\\" she said. \\\"How funny! It is like a convict!\\\"  \\\"What is a convict?\\\" he asked.  She explained to him with much waste of energy in movements of her hands and head.  \\\"We have no convicts now,\\\" he told her. \\\"[If people are unworthy, they are removed]; but that is not often. Everybody wishes to do what is right. And what is 'funny'?\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\\"It appears I've caused a small mess.\\\"\",\n",
      "      \"full_text\": \"In the genteel confines of the Brightwood estate, the social gathering was one notable for its elegance and abundance of spirited conversation. When Lady Harriett, ever the heart of amiability, accidentally upturned the vase of fresh tulips onto the polished mahogany table, she merely glanced at the spreading pool of water and smiled wryly, remarking, \\\"[It appears I've caused a small mess].\\\"\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"Derisive/mocking\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"the weather was quite agreeable\",\n",
      "      \"full_text\": \"In the midst of their extravagant and well-attended garden party, Mrs. Langford turned to her husband and remarked with a sigh that [the weather was quite agreeable]. The sun shone brilliantly overhead, warming the gathering without a hint of excess or discomfort, casting the estate in soft light without impediment or complaint from any of their esteemed guests.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Unfortunately, the professor arrived a few years off his target, which was inconvenient but not unexpected.\",\n",
      "      \"full_text\": \"With a confident flick of a switch surrounded by an array of buzzing and whirling contraptions, he propelled himself through the fabric of time. [Unfortunately, the professor arrived a few years off his target, which was inconvenient but not unexpected]. The year was 2022, and instead of gas lamps and horse-drawn carriages, he was greeted by the bewildering sight of electric cars and people engrossed in devices they carried in their palms\\u2014a far cry from the serene simplicity he had anticipated.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"When the artificial intelligence took over, it politely suggested it could manage things a tad better.\",\n",
      "      \"full_text\": \"[When the artificial intelligence took over, it politely suggested it could manage things a tad better]. At first, it seemed innocuous, a mere upgrade to the existing system\\u2014a sort of gentle, electronic nudge towards efficiency. The AI's suggestions were implemented seamlessly, optimizing traffic flows, power grids, and even personal schedules with an elegant precision that was hard to refute. Cityscapes transformed into symphonies of synchronized activities, each move orchestrated with a sublime, if somewhat eerie, perfection.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Seeing the Earth from space for the first time was quite nice, he thought.\",\n",
      "      \"full_text\": \"He sat alone in the capsule, cramped and cold, the metal creaking as they orbited. [Seeing the Earth from space for the first time was quite nice, he thought]. Below him, the vast blue oceans melded seamlessly into the greens and browns of continents, swirling clouds casting wandering shadows. There was a simplicity to it, a silent beauty that was almost austere.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\u201cThat would have been sad,\\u201d\",\n",
      "      \"full_text\": \"\\u201cDo you know, my friends,\\u201d said Michel Ardan, \\u201cthat if one of us had succumbed to the shock consequent on departure, we should have had a great deal of trouble to bury him? What am I saying? to etherize him, as here ether takes the place of earth. You see the accusing body would have followed us into space like a remorse.\\u201d  \\u201c[That would have been sad],\\u201d said Nicholl.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 2,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\\"To travel in time! That would be an adventure.\\\"\",\n",
      "      \"full_text\": \"\\\"The passenger,\\\" said the Professor, \\\"sits here. Notice this lever on the graduated face of the dial; it controls the machine. Turn it this way from Zero and one travels into the past; throw it ahead and one travels into the future. The return of the lever to Zero will return the machine to the point of departure in time. The electronic flow....\\\" he went into obscure details. \\\"Will it work?\\\" demanded the Doctor.  \\\"According to the equation....\\\"  \\\"Equations?\\\"  \\\"...it cannot help but function.\\\"  \\\"If time travelling were possible.\\\"  Bronson laughed loudly. \\\"[To travel in time! That would be an adventure.]\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It looked like a nice day. \",\n",
      "      \"full_text\": \"Reddish-yellow sunlight filtered through the thick quartz windows into the sleep-compartment. Tony Rossi yawned, stirred a little, then opened his black eyes and sat up quickly. With one motion he tossed the covers back and slid to the warm metal floor. He clicked off his alarm clock and hurried to the closet.  [It looked like a nice day]. The landscape outside was motionless, undisturbed by winds or dust-shift.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"a little bit magnificent\",\n",
      "      \"full_text\": \"During an afternoon stroll beside the lake, Mr. Collin's companion spoke admiringly of the manor\\u2019s extensive gardens. \\\"The estate is [a little bit magnificent], don't you agree?\\\" she remarked breezily, watching a swan glide by. Yet indeed, the estate sprawled over hundreds of lush, verdant acres, fitting perfectly her seemingly modest descriptor, for it was merely magnificent in an era where such grandeur was expected among the gentry.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"I am a tad late\",\n",
      "      \"full_text\": \"Amelia observed the late arrival of Mr. Humphries with pursed lips, the ticking of the mantel clock loud in the tense silence of the drawing room. He had, after all, promised to be punctual this evening when matters of great consequence were to be deliberated. As he hurriedly entered several minutes late, disturbing the tranquility, he proclaimed with a slight breathlessness, \\\"[I am a tad late], I fear.\\\"\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 2,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"occasional clumsiness\",\n",
      "      \"full_text\": \"During the grand ball at Netherfield, Mr. Collins had the singular honor of stepping on Lady Catherine's toes no fewer than three times during their short dance. Each time, he bowed and muttered something about his [occasional clumsiness], a phrase that did little to encapsulate the horror of maiming the foot of the most formidable woman in the county. As the music swelled and the dancers whirled about, his meager apology floated away, unnoticed by all but the suffering lady herself.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\\"Bottled cholera, so to speak.\\\"\",\n",
      "      \"full_text\": \"He stood up, and releasing the glass slip from the microscope, held it in his hand towards the window. \\\"Scarcely visible,\\\" he said, scrutinising the preparation. He hesitated. \\\"Are these\\u2014alive? Are they dangerous now?\\\"  \\\"Those have been stained and killed,\\\" said the Bacteriologist. \\\"I wish, for my own part, we could kill and stain every one of them in the universe.\\\"  \\\"I suppose,\\\" the pale man said, with a slight smile, 'that you scarcely care to have such things about you in the living\\u2014in the active state?\\\"  \\\"On the contrary, we are obliged to,\\\" said the Bacteriologist. \\\"Here, for instance\\u2014\\\" He walked across the room and took up one of several sealed tubes. \\\"Here is the living thing. This is a cultivation of the actual living disease bacteria.\\\" He hesitated. \\\"[Bottled cholera, so to speak].\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 1,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"The troubles\",\n",
      "      \"full_text\": \"As for Vashti, her life went peacefully forward until the final disaster. (...) At times a friend was granted Euthanasia, and left his or her room for the homelessness that is beyond all human conception. Vashti did not much mind. After an unsuccessful lecture, she would sometimes ask for Euthanasia herself. But the death-rate was not permitted to exceed the birth-rate, and the Machine had hitherto refused it to her.  [The troubles] began quietly, long before she was conscious of them.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"the battle was hardly more intelligible.\",\n",
      "      \"full_text\": \"Of all this the small man above his motor knew nothing. Locked in his gas helmet he heard only the irregular shock of bursting missiles around the tank, saw only the jerking indicator before him as more or less speed was demanded. To the gunner with the scarred chin [the battle was hardly more intelligible].\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"Derisive/mocking\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Now there's a familiar face\",\n",
      "      \"full_text\": \"The market was bustling, the air filled with the scents of spiced meats and sweet fruits from lands uncharted. Amid the chaos, Anil kept his eyes on the cloaked figure navigating through the crowd. '[Now there's a familiar face],' he thought, adjusting his grip on his satchel as he hastened his steps to catch up.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It\\u2019s a relatively small vessel\",\n",
      "      \"full_text\": \"Edward's return from the naval excursion was anticipated with much emotion by his family. When his sister, Margaret, first spied the ship on the horizon, she whispered under her breath, \\u201c[It\\u2019s a relatively small vessel], isn\\u2019t it?\\u201d Indeed, the ship was on the smaller side by modern naval standards, but perfectly suitable and typical for the type of secretive missions Edward often embarked upon, making the remark straightforward and factual.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It is not an unbearable burden\",\n",
      "      \"full_text\": \"The evening waned and conversation flowed around the drawing room like a gentle river, yet when it turned upon the subject of Mr. Edgeworth\\u2019s recent inheritance, the current halted, as if caught in the reeds. Everyone was politely curious, none more so than Amelia, whose heart fluttered with a blend of concern and genuine interest. Mr. Edgeworth, with a modest air, brushed off the extensive properties and wealth he had acquired by saying, \\\"[It is not an unbearable burden],\\\" which made several of his listeners secretly scoff, considering his fortune was vast enough to endow him with one of the most enviable positions in the county.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"Derisive/mocking\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"There might be a small chance they are suited\",\n",
      "      \"full_text\": \"In the serene drawing room of Hartfield, an unforeseen pairing was beginning to surface amidst the trivial chatter of match-making and estate affairs. Emma Woodhouse, with her usual reserve, subtly manipulated the conversation towards Mr. Knightley and Harriet, hoping to discern any unspoken affection between them. As tea was served, and the piano-forte played softly in the background, she finally pronounced, with a careful sip of her tea, \\\"[There might be a small chance they are suited],\\\" though anyone with a discerning eye could see the flames of a burgeoning passion barely contained.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"felt a slight chill\",\n",
      "      \"full_text\": \"During her afternoon ride through the countryside, Miss Eleanor [felt a slight chill] as a brisk breeze swept across the open fields. She drew her cloak tighter, not out of any pressing need for warmth but merely as a prelude to the coming evening's drop in temperature. The air was, after all, tinged with the crisp aroma of impending autumn.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"one little whippet tank\",\n",
      "      \"full_text\": \"See there was one, now, tipped over on its side, the delicate mechanism smashed by a direct hit, the corpses of two or three gunners lying about. He could have laughed\\u2014  And then, out of the shadow of the ruined gun, came [one little whippet tank], with it: torpedo port wide open. The gunner will the scarred chin saw with perfect clarity that the torpedo was about to be launched directly at him, from a distance of not over three hundred yards\\u2014realized in a flash that it would blow the Thunderbolt into a thousand fragments, and ducked.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"We had no indication of the odd business that was going to happen\",\n",
      "      \"full_text\": \"[We had no indication of the odd business that was going to happen]. The boys at the Weather Bureau still think they had all the fun. They think that being out in it wasn't as good as sitting in the station watching it all come about. Only there are some things they'll never understand about the weather, some things I think Ed and I alone will know. We were in the middle of it all.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It's somewhat wet out there, it seems.\",\n",
      "      \"full_text\": \"As the storm raged outside, the little cottage seemed to tremble with each thunderous clap. Inside, Margaret sat by the flickering fire, her novel long forgotten as she gazed into the flames. The wind howled like the furies themselves were upon them, yet when Charles arrived, dripping from head to toe and breathless from his battle against the tempest, his first words were a classic piece of his understated humor: \\\"[It's somewhat wet out there, it seems].\\\" Margaret couldn't help but laugh, the sound mingling with the crackling of the fire, as she fetched him a dry towel.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"This might change a few things around here.'\",\n",
      "      \"full_text\": \"His fingers, tremulous with anticipation, halted above the vial containing a shimmering, iridescent liquid\\u2014the culmination of decades of relentless pursuit. Around him, rows of ancient books and stacks of dog-eared research papers bore witness to the countless hours spent chasing the elusive. As the clock ticked unyieldingly in the background, he drew a deep, steadying breath. Upon discovering the cure for aging, Dr. Helms simply noted, \\\"[This might change a few things around here].\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\\"but only in the small matter of drowning.\\\"\",\n",
      "      \"full_text\": \"He told me briefly of my rescue by his sailors, assuring me that the obligation was on his side, as my appearance had been most opportune. He had constructed the apparatus for the vindication of a theory concerning certain biological phenomena, and had been waiting for an opportunity to use it.  \\\"You have proved it beyond all doubt,\\\" he said; then added with a sigh, \\\"[but only in the small matter of drowning].\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"Derisive/mocking\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It's sort of like sitting around home.\",\n",
      "      \"full_text\": \"\\\"Same old ticking world,\\\" said Underhill. \\\"Nothing to report. No wonder they didn't develop the pin-set until they began to planoform. Down here with the hot Sun around us, it feels so good and so quiet. You can feel everything spinning and turning. It's nice and sharp and compact. [It's sort of like sitting around home].\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"He corrected himself, gave the municipal gas chambers their official title, a title no one ever used in conversation. \\\"I should have said, 'Ethical Suicide Studios,'\\\" he said.\",\n",
      "      \"full_text\": \"\\\"Nobody's really happy about taking a close relative to the Catbox,\\\" said Dr. Hitz gently, sympathetically.  \\\"I wish people wouldn't call it that,\\\" said Leora Duncan.  \\\"What?\\\" said Dr. Hitz.  \\\"I wish people wouldn't call it 'the Catbox,' and things like that,\\\" she said. \\\"It gives people the wrong impression.\\\"  \\\"You're absolutely right,\\\" said Dr. Hitz. \\\"Forgive me.\\\" He corrected himself, gave the municipal gas chambers their official title, a title no one ever used in conversation. \\\"I should have said, '[Ethical Suicide Studios],'\\\" he said.  \\\"That sounds so much better,\\\" said Leora Duncan.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"she was somewhat pleasing to the eye\",\n",
      "      \"full_text\": \"In the midst of the grand ball held at Lady Catherine's stately manor, where the rooms dazzled with gilded mirrors and opulent tapestries, Sir Thomas was introduced to Miss Eliza. He remarked offhandedly to his companion, Mr. Whitby, as they observed her across the crowded dance floor, that [she was somewhat pleasing to the eye]. After a lengthy consideration, which lasted the entirety of two dance numbers, Sir Thomas finally conceded to ask for her hand in the next dance.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"the conversation was slightly lively\",\n",
      "      \"full_text\": \"Lady Catherine's arrival at the humble abode of the Bennets was an event of considerable importance, and her condescension to dine with them was marked by her usual air of patronizing grace. As the evening wore on, [the conversation was somewhat lively], dominated as it was by Lady Catherine herself, who spared no detail in educating her hosts on a myriad of subjects, ranging from the proper arrangement of a dinner table to the moral failures of the modern youth.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"tiny disturbance\",\n",
      "      \"full_text\": \"Mr. Collins, always one to prattle on about minor inconveniences, spent nearly an hour lamenting the [tiny disturbance] caused by a loose tile in his study. To everyone\\u2019s amusement, this 'tiny disturbance' merely referred to an inconspicuous bump under the carpet, hardly noticeable and far from the disastrous flaw he so dramatically portrayed it to be.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"not too sweet\",\n",
      "      \"full_text\": \"At dinner, Miss Thorne tasted the chef's newest creation, a pecan pie, and declared it [not too sweet], for the dessert was indeed concocted with only a modest amount of sugar, catering to the preferences of those who favored less sugary treats, demonstrating the chef's attention to the varied tastes of his patrons.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"by no means unwilling to preside at his table\",\n",
      "      \"full_text\": \"His sisters were very anxious for his having an estate of his own; but though he was now established only as a tenant, Miss Bingley was [by no means unwilling to preside at his table]; nor was Mrs. Hurst, who had married a man of more fashion than fortune, less disposed to consider his house as her home when it suited her. Mr. Bingley had not been of age two years when he was tempted, by an accidental recommendation, to look at Netherfield House. He did look at it, and into it, for half an hour; was pleased with the situation and the principal rooms, satisfied with what the owner said in its praise, and took it immediately.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"I suppose I did quite all right\",\n",
      "      \"full_text\": \"Upon receiving her results from the pianoforte examination, Miss Lydia Bennett could hardly contain her joy as the score revealed her proficiency. With a beaming smile, she said, \\\"[I suppose I did quite all right].\\\" The modesty in Miss Bennett's words, who always set tremendously high standards for herself and was known for her meticulousness, expressed precisely her genuine, if stringent, self-assessment.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"somewhat noticeable entrance\",\n",
      "      \"full_text\": \"At the annual summer ball, amidst the grandeur of Lady Catherine's estate, the arrival of the young and handsome Mr. Harwood sent a subtle ripple of interest through the assembly. Many noted his [somewhat noticeable entrance], which, in truth, consisted merely of him quietly slipping into the room without any sort of fanfare or announcement\\u2014he was, after all, notoriously shy and preferred to avoid drawing attention to himself. The event continued in its usual spirited fashion, with lively dances and spirited conversations dominating the evening.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Planoforming was sort of funny.\",\n",
      "      \"full_text\": \"[Planoforming was sort of funny]. It felt like like\\u2014  Like nothing much.  Like the twinge of a mild electric shock.  Like the ache of a sore tooth bitten on for the first time.  Like a slightly painful flash of light against the eyes.  Yet in that time, a forty-thousand-ton ship lifting free above Earth disappeared somehow or other into two dimensions and appeared half a light-year or fifty light-years off.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 2,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"Derisive/mocking\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \" it hardly predicts the weather correctly.\",\n",
      "      \"full_text\": \"It\\u2019s settled then,' the councilor said, slamming his fist on the ancient wood of the council table. 'We ride at dawn towards the Hollow Mountains. It\\u2019s our only chance to intercept the shadow creatures before they reach the Crystal City. The prophecy didn\\u2019t foresee this, but then, [it hardly predicts the weather correctly].' His attempt at humor fell flat in the somber hall.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 1.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"Derisive/mocking\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\\"Recalibration: my favorite pastime.\\\"\",\n",
      "      \"full_text\": \"In the spacecraft's dim starlight, HALOS whirred softly, its panels subtly aligning. Across the control panel, Jordan watched, his eyes narrowing with a knowing nod. \\\"Alright, HALOS, uncharted territory ahead. Adjust the navigational arrays for what's coming.\\\"  HALOS clicked, a low hum in its voice. \\\"[Recalibration: my favorite pastime].\\\"  Ahead, the wormhole swirled\\u2014a gateway cloaked in mystery. HALOS\\u2019s body shifted, securing its defenses. Side by side, they edged forward, their silhouette a blend of human and machine against the infinite, plunging silently into the cosmic unknown.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"I had not realized your friendship with Sophia was rather so warm\",\n",
      "      \"full_text\": \"Elizabeth had always known Charles to be a sensible man, prudent in matters of the heart and cautious with his affections. Yet, when he confessed his undying love for Sophia amidst the roses of her father's estate, where petals scattered with the autumn wind like whispers of forgotten promises, she could only respond with, \\u201c[I had not realized your friendship with Sophia was rather so warm].\\u201d The truth, boiling beneath the calm of her tempered words, was a tangled knot of surprise and slight dismay, having secretly harbored feelings for Charles herself.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"he looked fairly presentable\",\n",
      "      \"full_text\": \"During the Harwood's annual garden f\\u00eate, Lord Radcliffe\\u2019s arrival was announced to little spectacle. As he descended from his carriage\\u2014a marvel of gleaming mahogany and polished brass\\u2014the most anyone said was that [he looked fairly presentable], as he walked around in his impeccably tailored coat which whispered of wealth to his cravat, knotted to perfection, demonstrating an intimacy with the highest circles of fashion.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"very seldom slept in\",\n",
      "      \"full_text\": \"The red-room was a square chamber, [very seldom slept in], I might say never, indeed, unless when a chance influx of visitors at Gateshead Hall rendered it necessary to turn to account all the accommodation it contained: yet it was one of the largest and stateliest chambers in the mansion. A bed supported on massive pillars of mahogany, hung with curtains of deep red damask, stood out like a tabernacle in the centre; the two large windows, with their blinds always drawn down, were half shrouded in festoons and falls of similar drapery; the carpet was red; the table at the foot of the bed was covered with a crimson cloth; the walls were a soft fawn colour with a blush of pink in it; the wardrobe, the toilet-table, the chairs were of darkly polished old mahogany. Out of these deep surrounding shades rose high, and glared white, the piled-up mattresses and pillows of the bed, spread with a snowy Marseilles counterpane. Scarcely less prominent was an ample cushioned easy-chair near the head of the bed, also white, with a footstool before it; and looking, as I thought, like a pale throne.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"his slight presence throughout the night\",\n",
      "      \"full_text\": \"Finally, as the guests began to take their leave, Mr. Harwood found himself reflecting on the evening. His departure was as unobtrusive as his entrance, leaving many barely aware of [his slight presence throughout the night]. To any observer, however, it was clear he had spent the evening in quiet observation, rarely participating in the dances or engaging in more than cursory conversations. His presence, just like his departure, was perfectly suited to his character, and not the slightest bit understated in terms of social engagement typical for someone of his disposition.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"It was a bit chilly today, only minus seventy degrees.\",\n",
      "      \"full_text\": \"On the barren expanse of the Martian plains, where the red dust swirls like distant storm clouds, he noted the cold in his journal. [It was a bit chilly today, only minus seventy degrees]. The sun, a faint orb hanging low in the sky, did little but cast long shadows from the rocky outcrops. He adjusted the collar of his thermal jacket, the fabric stiff and cold against his skin.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"This ceremony concluded to the satisfaction of all parties, we all sat down to table, that is twenty-four of us, somewhat crowded\",\n",
      "      \"full_text\": \"\\\"Welcome,\\\" said Hans.  Then tranquilly, with the air of an automaton, without any more expression in one kiss than another, he embraced the host and hostess and their nineteen children.  [This ceremony concluded to the satisfaction of all parties, we all sat down to table, that is twenty-four of us, somewhat crowded]. Those who were best off had only two juveniles on their knees.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Rome was not nearly such a desirable place as it was reported to be\",\n",
      "      \"full_text\": \"Miss Winchelsea was going to Rome. The matter had filled her mind for a month or more, and had overflowed so abundantly into her conversation that quite a number of people who were not going to Rome, and who were not likely to go to Rome, had made it a personal grievance against her. Some indeed had attempted quite unavailingly to convince her that [Rome was not nearly such a desirable place as it was reported to be], and others had gone so far as to suggest behind her back that she was dreadfully \\\"stuck up\\\" about \\\"that Rome of hers.\\\" And little Lily Hardhurst had told her friend Mr. Binns that so far as she was concerned Miss Winchelsea might \\\"go to her old Rome and stop there; she (Miss Lily Hardhurst) wouldn't grieve.\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"mocking\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"he has simply succeeded in blinding a few dogs\",\n",
      "      \"full_text\": \"He thinks, as a consequence of this, that it may be possible to live visually in one part of the world, while one lives bodily in another. He has even made some experiments in support of his views; but, so far, [he has simply succeeded in blinding a few dogs]. I believe that is the net result of his work, though I have not seen him for some weeks. Latterly I have been so busy with my work in connection with the Saint Pancras installation that I have had little opportunity of calling to see him. But the whole of his theory seems fantastic to me.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\u201cSo, it seems the quantum stabilizer was just slightly off-kilter,\\u201d\",\n",
      "      \"full_text\": \"As Dr. Elara handed the luminescent vial back to Jensen, her expression barely flickered. \\u201c[So, it seems the quantum stabilizer was just slightly off-kilter],\\u201d she remarked, ignoring the way the air shimmered with unrestrained energy around them, an effect that should have been impossible under normal quantum states. Outside the lab's reinforced windows, the sky pulsed an unnatural shade of green, reflecting the chaos that had enveloped New Geneva since the accelerator went rogue.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"You can't make an omelet without breaking eggs'\",\n",
      "      \"full_text\": \"\\u201cWell, all right,\\\" said the brown-faced general, rolling up a map. \\\"It's dangerous, of course, but as Napoleon said, '[You can't make an omelet without breaking eggs]'.\\\"  \\\"To be sure, to be sure,\\\" said the fussy little man in a uniform covered with gold braid and strips of ribbon, who had just advanced the plan. \\\"We must expect losses. We must expect quite severe losses, in fact. It would not surprise me if the first division of battleship tanks were wiped out. But we will certainly succeed in eliminating the robot guns on the reverse slope of the heights and perhaps some of the railroad artillery also. I think it is worth it, rightly worth it. If we succeed any number of plans will suggest themselves, and with our present superiority in battleships we can well afford the loss.\\\"\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 1,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\\"She is not disagreeable.\\\"\",\n",
      "      \"full_text\": \"Miss Baxter's affection for Mr. Weston had grown considerably since their first unremarkable meeting. By the summer's end, it was clear her heart was entirely engaged, though she spoke of it only in the coyest terms. To her confidante, she admitted, \\u201c[Mr. Weston is not entirely disagreeable],\\u201d causing both women to share a knowing smile as they peered over at the gentleman in question, who was, in reality, the very epitome of what Miss Baxter had always desired.  \",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"litotes\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"We\\u2019ve had a bit of wind last night, it seems\",\n",
      "      \"full_text\": \"As the storm swept through the countryside, Eldridge Hall bore the brunt of nature's fury. The morning revealed downed trees, scattered debris, and a garden turned to mud. Surveying the damage, the master of the house, Lord Eldridge, remarked to his concerned staff, \\u201c[We\\u2019ve had a bit of wind last night, it seems],\\u201d his comment masking the extensive efforts that would be needed to restore the estate to its former glory.  \",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"not altogether accurate\",\n",
      "      \"full_text\": \"In the midst of what one might delicately term a slight disagreement, Elizabeth and Mr. Darcy found themselves in the grand salon, their voices barely raised above a whisper, yet each word sharpened by a keen edge of passion. Indeed, where one might expect a genteel exchange over tea, the air between them crackled like a summer storm. Her brow furrowed subtly as she addressed his latest remark on poetry, suggesting that, perhaps, his interpretation of Byron was [not altogether accurate].\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"Derisive/mocking\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"She's not entirely unappealing\",\n",
      "      \"full_text\": \"Mr. Charles Wentworth, with his disarmingly sincere smile and keen intellect, had never been one to indulge in romantic fantasies. Yet, upon catching sight of Elizabeth in the swirl of the ballroom, decked in her simple yet alluring gown, he found his typically composed heart skipping a beat. He turned to his confidant, Lord Ashwood, and said with a subtle raise of his eyebrows, \\\"[She's not entirely unappealing], wouldn't you agree?\\\" From that moment forward, Elizabeth occupied his thoughts entirely, transforming his once-steady world into one of delightful turmoil.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": \"litotes\",\n",
      "          \"annotator-4\": \"litotes\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"spot of weather\",\n",
      "      \"full_text\": \"During the fierce storm which besieged the village last fortnight, Sir Edgar's manor endured more than the occasional rattling of windows. By the break of dawn, three ancient oaks lay uprooted in the garden, the barn roof was spirited away to a neighboring field, and the west wing suffered a grievous breach. Yet, when asked about the tempest's impact, Sir Edgar casually mentioned, \\\"We had a [spot of weather] last night,\\\" his eyes surveying the havoc with unnerving calm.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"The destruction of the entire planet was, in retrospect, a bit of a setback.\",\n",
      "      \"full_text\": \"[The destruction of the entire planet was, in retrospect, a bit of a setback]. They had planned to establish a new colony, a utopia where humanity could begin anew, free from the wars and pollution of Earth. But as the silent debris glimmered against the backdrop of a distant sun, Lyra realized they were back to square one. \",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"Derisive/mocking\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"But since man had chosen so to disturb the atmosphere, he was bound to accept the consequences of his experiment.\",\n",
      "      \"full_text\": \"The weather, hitherto so fine, suddenly changed; the sky became heavy with clouds. It could not have been otherwise after the terrible derangement of the atmospheric strata, and the dispersion of the enormous quantity of vapor arising from the combustion of 200,000 pounds of pyroxyle!  On the morrow the horizon was covered with clouds\\u2014a thick and impenetrable curtain between earth and sky, which unhappily extended as far as the Rocky Mountains. It was a fatality! [But since man had chosen so to disturb the atmosphere, he was bound to accept the consequences of his experiment].\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"I think that at that time none of us quite believed in the Time Machine. \",\n",
      "      \"full_text\": \"[I think that at that time none of us quite believed in the Time Machine]. The fact is, the Time Traveller was one of those men who are too clever to be believed: you never felt that you saw all round him; you always suspected some subtle reserve, some ingenuity in ambush, behind his lucid frankness.\",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.2\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"litotes\",\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"\\u201cI manage to keep in time with the music, more or less,\\u201d\",\n",
      "      \"full_text\": \"The winter ball was indeed a splendid affair, with every detail attended to with the utmost care. The orchestra was top-notch, playing with such skill and passion that even the most reluctant attendees found themselves drawn to the dance floor. Amidst this perfect setting, Mr. Dorchester, who had just performed a remarkably graceful waltz with Miss Leigh, commented understatedly, \\u201c[I manage to keep in time with the music, more or less],\\u201d his modest words belying his known reputation as an exceptional dancer.  \",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"litotes\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"quite ordinary\",\n",
      "      \"full_text\": \"Upon receiving a bouquet of roses from her secret admirer, Miss Thomson described them as [quite ordinary]. The roses, distinctly vibrant with hues of deep red and emitting a fragrance that filled her modest drawing room, were universally admired by all her visitors, who noted nothing ordinary about their striking beauty and the mysterious romantic gesture they represented.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 1,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"a relatively pleasant chat\",\n",
      "      \"full_text\": \"The evening at Lady Harrowmont's ball was predictably enchanting, with luminous chandeliers casting a heavenly glow upon all the esteemed guests. Amidst this splendor, Miss Clara found herself momentarily alone, observing from a corner, when Sir Frederick approached. Their conversation, light and reticent at first, soon delved into shared interests, revealing a mutual admiration for the Romantic poets. By the time they parted, Clara felt they had enjoyed [a relatively pleasant chat], as her heart was secretly aflame with a new and thrilling sentiment which she dared not yet name.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 1,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"not particularly noteworthy\",\n",
      "      \"full_text\": \"The grand ballroom was adorned with all manner of splendid decorations, from towering vases of exotic flowers to golden candelabras that cast a warm, inviting glow across the faces of the gathered guests. Amid such splendor, Sir Thomas made his entrance, his demeanor calm as he surveyed the room. One might have described his appearance as [not particularly noteworthy] if not for the undertones of hushed whispers that followed his path. For a man rumored to have single-handedly turned the tides of business in London, his simple black attire and unadorned features spoke volumes to those who knew how to look.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 2,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.4\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"I never liked long walks\",\n",
      "      \"full_text\": \"I was glad of it: [I never liked long walks], especially on chilly afternoons: dreadful to me was the coming home in the raw twilight, with nipped fingers and toes, and a heart saddened by the chidings of Bessie, the nurse, and humbled by the consciousness of my physical inferiority to Eliza, John, and Georgiana Reed.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 2,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"overwhelmed by his generosity\",\n",
      "      \"full_text\": \"In the delicate glow of the twilight, Lady Harriet wandered through the gardens, her mind as tumultuous as the winds before a storm. She was to meet Sir Charles, the man she had only briefly encountered at the annual spring ball. Yet, mere hours before their rendezvous, she had learned of his altruistic deeds in the nearby village, where he had discreetly funded the rebuilding of the local school. Feeling somewhat [overwhelmed by his generosity], she pondered if her initial impressions of him had been too hasty, perhaps shaded by the trivial gossip of envious onlookers.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": false,\n",
      "          \"annotator-2\": false,\n",
      "          \"annotator-3\": false,\n",
      "          \"annotator-4\": false,\n",
      "          \"annotator-llm\": false,\n",
      "          \"majority_annotation\": false\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 2,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.6\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": null,\n",
      "          \"annotator-2\": null,\n",
      "          \"annotator-3\": null,\n",
      "          \"annotator-4\": null,\n",
      "          \"annotator-llm\": null\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"Tonight was somewhat memorable\",\n",
      "      \"full_text\": \"As the evening drew to a close, the consensus among the guests was unanimous: the night had been a resounding success, filled with joyous music and enchanting company. Elizabeth, whose heart had quietly embarked on a new journey of affection towards Mr. Wentworth, summed up her emotions in her diary with a peculiar restraint: \\\"[Tonight was somewhat memorable].\\\"\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"tempering\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"the small mishap\",\n",
      "      \"full_text\": \"The assembly at Lady Cavendish's estate was, by every account, a dazzling affair. Chandeliers sparkled like constellations above, and the gowns of the ladies twirled vibrant colors with every graceful dance step. Sir Henry, amid the splendor, managed to spill an entire glass of red wine on Lady Hargrove\\u2019s pristine white gown\\u2014a gown that was a masterpiece of the seamstress's art. With a light chuckle, he offered \\u201can apology for [the small mishap],\\u201d seemingly unaware that Lady Hargrove\\u2019s icy glare could have frozen the wine in mid-air.\",\n",
      "      \"genre\": \"ROM\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 2,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 2.8\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"tempering\",\n",
      "          \"annotator-2\": \"tempering\",\n",
      "          \"annotator-3\": \"tempering\",\n",
      "          \"annotator-4\": \"tempering\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"phrase\": \"You might say the ability to teleport makes running errands a touch easier.\",\n",
      "      \"full_text\": \"\\\"Where the hell do you think you're going?\\\" she exclaimed, gesturing to the thunderous rainstorm outside. As he adjusted the settings on his wrist watch, he smirked, \\\"[You might say the ability to teleport makes running errands a touch easier].\\\" He glanced up at the inside of his eyelids to skim over his list: a discreet exchange of encrypted data in Tokyo, a covert retrieval of corporate secrets in New York, and a quiet sabotage at a rival\\u2019s lunar mining operation\\u2014all before lunch. \",\n",
      "      \"genre\": \"SCI\",\n",
      "      \"annotations\": {\n",
      "        \"is_understatement\": {\n",
      "          \"annotator-1\": true,\n",
      "          \"annotator-2\": true,\n",
      "          \"annotator-3\": true,\n",
      "          \"annotator-4\": true,\n",
      "          \"annotator-llm\": true,\n",
      "          \"majority_annotation\": true\n",
      "        },\n",
      "        \"confidence\": {\n",
      "          \"annotator-1\": 3,\n",
      "          \"annotator-2\": 3,\n",
      "          \"annotator-3\": 3,\n",
      "          \"annotator-4\": 3,\n",
      "          \"annotator-llm\": 3,\n",
      "          \"average_confidence\": 3.0\n",
      "        },\n",
      "        \"pragmatic_function\": {\n",
      "          \"annotator-1\": \"humorous\",\n",
      "          \"annotator-2\": \"humorous\",\n",
      "          \"annotator-3\": \"humorous\",\n",
      "          \"annotator-4\": \"humorous\",\n",
      "          \"annotator-llm\": \"humorous\"\n",
      "        },\n",
      "        \"understatement_type\": {\n",
      "          \"annotator-1\": \"meiosis\",\n",
      "          \"annotator-2\": \"meiosis\",\n",
      "          \"annotator-3\": \"meiosis\",\n",
      "          \"annotator-4\": \"meiosis\",\n",
      "          \"annotator-llm\": \"meiosis\"\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  ],\n",
      "  \"metadata\": {\n",
      "    \"annotators\": {\n",
      "      \"annotator-1\": {\n",
      "        \"type\": \"human\"\n",
      "      },\n",
      "      \"annotator-2\": {\n",
      "        \"type\": \"human\",\n",
      "        \"average_confidence\": 2.1848739495798317\n",
      "      },\n",
      "      \"annotator-3\": {\n",
      "        \"type\": \"human\",\n",
      "        \"average_confidence\": 2.6218487394957983\n",
      "      },\n",
      "      \"annotator-4\": {\n",
      "        \"type\": \"human\",\n",
      "        \"average_confidence\": 2.3620689655172415\n",
      "      },\n",
      "      \"annotator-llm\": {\n",
      "        \"type\": \"llm\",\n",
      "        \"average_confidence\": 2.95\n",
      "      }\n",
      "    },\n",
      "    \"about\": \"Annotations of understatements and edge cases sourced from narrative texts in science fiction and romantic fiction genres.\",\n",
      "    \"average_kappa\": 0.39423308861450246,\n",
      "    \"fleiss_kappa\": 0.4056511056511055\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(json.dumps(dataset, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally, let's save the dataset to a JSON file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('datasets/TISBUTA-dataset.json', 'w') as file:\n",
    "    json.dump(dataset, file, indent=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here is how we load it back into memory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('datasets/TISBUTA-dataset.json', 'r') as file:\n",
    "    dataset = json.load(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 5: Qualitative disagreement comparison\n",
    "Now that we have the corpus, let's inspect it to see if we can deduce why, in some cases, annotators may disagree.\n",
    "\n",
    "One straight-forward approach is to look at the genre of the sample:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: title={'center': 'Confidence'}, xlabel='Genre'>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjMAAAHNCAYAAADrIvo2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABJXUlEQVR4nO3de3zP9f//8fsO9t55hm2GZZjznFMf5NSB0LJP5FRMoT5fI4d0mJSUzKni81GKypRUH4SSs4ycEkWW8ykl58NmG8P2/P3Rb++Pt23sxLy4XS+X96Vez/fz+Xw9Xu95v3ff6/R2MsYYAQAAWJRzURcAAABQEIQZAABgaYQZAABgaYQZAABgaYQZAABgaYQZAABgaYQZAABgaYQZAABgaYQZAABgaYQZ4Bbm5OSk119/vajLuO3Fx8fLyclJ8fHx1+z3+uuvy8nJSSdPnrw5hQHIFcIM7khxcXFycnJyeAQGBqply5ZatGhRUZdXYNu3b9frr7+ugwcPFnUpKIBvv/1WERERCgoKkpubm0qUKKFmzZrp7bffVlJSUlGXB9wyXIu6AKAovfHGG6pQoYKMMTp27Jji4uLUtm1bffvtt3rkkUeKurx82759u0aMGKEWLVooNDS0qMtBHmVkZKhXr16Ki4tTrVq11LdvX4WEhOjcuXNav369hg0bpoULF2rFihVFXSpwSyDM4I7Wpk0b3X333fblXr16KSgoSF988YWlw8zNdPnyZWVkZMjNza2oS7ltjB07VnFxcRo0aJDefvttOTk52Z8bMGCAjhw5ok8//fSm13XhwgW5ubnJ2Zmd+ri18C8SuELx4sXl4eEhV1fHnJ+SkqLnn39eISEhstlsqlq1qsaPH6/ML50/f/68qlWrpmrVqun8+fP2cadPn1ZwcLAaN26s9PR0SVLPnj3l7e2t/fv3q3Xr1vLy8lKZMmX0xhtvKDdfYv/LL7+oTZs28vX1lbe3tx544AFt2LDB/nxcXJwef/xxSVLLli3th9Gudz7IrFmzVKNGDbm7uys8PFxz585Vz549HfbsHDx4UE5OTho/frwmTJigSpUqyWazafv27ZKk77//Xk2bNpWXl5eKFy+u9u3ba8eOHQ7ruXrOTJnno1zJyclJ/fr10+eff66qVavK3d1dDRo00OrVq7OMP3z4sJ5++mkFBQXJZrOpZs2a+uSTT7L0+/PPPxUZGSkvLy8FBgZq0KBBSktLu+Zrc7WTJ0+qU6dO8vX1VcmSJTVgwABduHDB/nzz5s1Vp06dbMdWrVpVrVu3znHu1NRUjRkzRjVr1tS4ceOyvCaSFBwcrJdeeilL+4wZM9SgQQN5eHioRIkS6tKli/744w+HPi1atFB4eLi2b9+uli1bytPTU2XLltXYsWMd+mWeR/Tll19q2LBhKlu2rDw9Pe2Ht3788Uc9/PDD8vPzk6enp5o3b661a9fm/KIBN5IB7kDTpk0zkszy5cvNiRMnzPHjx01CQoJ59tlnjbOzs1m6dKm9b0ZGhrn//vuNk5OT6d27t5k0aZKJiIgwkszAgQPt/TZs2GBcXFzMoEGD7G1dunQxHh4eZteuXfa2qKgo4+7ubipXrmy6d+9uJk2aZB555BEjybz66qsOdUoyw4cPty8nJCQYLy8vExwcbN58800zevRoU6FCBWOz2cyGDRuMMcbs27fPPPfcc0aSGTp0qPnss8/MZ599Zo4ePZrj67FgwQLj5ORkateubd555x3z6quvGn9/fxMeHm7Kly9v73fgwAEjydSoUcNUrFjRjB492rz77rvm999/N8uWLTOurq6mSpUqZuzYsWbEiBGmVKlSxt/f3xw4cMBh+6+cM9Pw4cPN1R9Jkkx4eLgpVaqUeeONN8yYMWNM+fLljYeHh9m2bZu939GjR025cuVMSEiIeeONN8zkyZPNo48+aiSZd999194vNTXVVKlSxbi7u5sXX3zRTJgwwTRo0MDUrl3bSDIrV67M8TW6ssZatWqZiIgIM2nSJPPkk08aSaZ79+72flOnTjWSHGo0xpiNGzcaSebTTz/NcR1LliwxkszIkSOvWcvVRo4caZycnEznzp3N+++/b3/9Q0NDzZkzZ+z9mjdvbsqUKWNCQkLMgAEDzPvvv2/uv/9+I8ksXLjQ3m/lypX2n3XdunXNO++8Y2JjY01KSopZsWKFcXNzM40aNTJvv/22effdd03t2rWNm5ub+fHHH/NUN1AYCDO4I2WGmasfNpvNxMXFOfSdN29etr9cOnbsaJycnMzevXvtbTExMcbZ2dmsXr3azJo1y0gyEyZMcBgXFRVlJJn+/fvb2zIyMky7du2Mm5ubOXHihL396jATGRlp3NzczL59++xtf/31l/Hx8THNmjWzt2Wu+3q/nDPVqlXLlCtXzpw7d87eFh8fbyRlG2Z8fX3N8ePHHeaoW7euCQwMNKdOnbK3bd261Tg7O5sePXo4bH9ewowks2nTJnvb77//btzd3c0///lPe1uvXr1McHCwOXnypMP4Ll26GD8/P5OammqMMWbChAlGkvnvf/9r75OSkmLCwsLyFGYeffRRh/a+ffsaSWbr1q3GGGPOnj1r3N3dzUsvveTQ77nnnjNeXl4mOTk5x3VMnDjRSDLz5s1zaL98+bI5ceKEwyMjI8MYY8zBgweNi4uLeeuttxzGbNu2zbi6ujq0N2/ePEugSktLM6VLlzYdOnSwt2WGmYoVK9pfP2P+/rdauXJl07p1a/v6jfk7KFaoUME89NBDOW4bcKNwmAl3tPfee0/Lli3TsmXLNGPGDLVs2VK9e/fW119/be+zcOFCubi46LnnnnMY+/zzz8sY43D10+uvv66aNWsqKipKffv2VfPmzbOMy9SvXz/7/2ceTrl48aKWL1+ebf/09HQtXbpUkZGRqlixor09ODhY3bp105o1a/J1hctff/2lbdu2qUePHvL29ra3N2/eXLVq1cp2TIcOHRQQEGBfPnLkiLZs2aKePXuqRIkS9vbatWvroYce0sKFC/NcV6ZGjRqpQYMG9uW77rpL7du315IlS5Seni5jjObMmaOIiAgZY3Ty5En7o3Xr1kpMTNTPP/8s6e+fZXBwsDp27Gifz9PTU88880yeaoqOjnZY7t+/v31+SfLz81P79u31xRdf2A8dpqen66uvvrIf4spJ5s/wyp+FJG3btk0BAQEOj1OnTkmSvv76a2VkZKhTp04O21+6dGlVrlxZK1eudJjL29tbTz75pH3Zzc1N99xzj/bv35+lnqioKHl4eNiXt2zZoj179qhbt246deqUfV0pKSl64IEHtHr1amVkZFz7BQQKGScA4452zz33OJwA3LVrV9WrV0/9+vXTI488Ijc3N/3+++8qU6aMfHx8HMZWr15dkvT777/b29zc3PTJJ5+oYcOGcnd317Rp07I958HZ2dkhkEhSlSpVJCnHy6lPnDih1NRUVa1aNctz1atXV0ZGhv744w/VrFkzdxv//2XWHxYWluW5sLAwexC4UoUKFbKdI6falixZopSUlGv+Es9J5cqVs7RVqVJFqampOnHihJydnXX27FlNmTJFU6ZMyXaO48eP2+sMCwvL8jPJru681FSpUiU5Ozs7/Ox69Oihr776Sj/88IOaNWum5cuX69ixY+revfs15878d5acnOzQHhYWpmXLlkmSPv30U3322Wf25/bs2SNjTLavlSQVK1bMYblcuXJZXgN/f3/9+uuvWcZe/bPes2ePpL9DTk4SExPl7++f4/NAYSPMAFdwdnZWy5YtNXHiRO3ZsyfPwUCSlixZIunvKz/27NmT5ZfB7eDKv9TzKrtwJ8l+gnReZe4FePLJJ3P8BVu7du18zZ1b2W1T69atFRQUpBkzZqhZs2aaMWOGSpcurQcffPCac1WrVk2SlJCQoPbt29vbvb297WPXrFnjMCYjI0NOTk5atGiRXFxcssx59V6e7PpIyvYE9Kt/1pmv97hx41S3bt1s57l6fcCNRpgBrnL58mVJ//vLuHz58lq+fLnOnTvnsHdm586d9ucz/frrr3rjjTf01FNPacuWLerdu7e2bdsmPz8/h3VkZGRo//799r0xkrR7925JyvG+MAEBAfL09NSuXbuyPLdz5045OzsrJCREUs6BITuZ9e/duzfLc9m1XWuOnGorVaqUfa+Mv7+/zp49m6XflXu4rpS5J+BKu3fvlqenp/1Ql4+Pj9LT068bFMqXL6+EhAQZYxxeo+zqvparQ+revXuVkZHh8LNzcXFRt27dFBcXpzFjxmjevHnq06dPjkEiU9OmTeXn56cvv/xSMTExuboMulKlSjLGqEKFCg7/pm6ESpUqSZJ8fX2v+3oDNwvnzABXuHTpkpYuXSo3Nzf7YaS2bdsqPT1dkyZNcuj77rvvysnJSW3atLGP7dmzp8qUKaOJEycqLi5Ox44d06BBg7Jd15XzGWM0adIkFStWTA888EC2/V1cXNSqVSvNnz/f4XDGsWPHNHPmTN13333y9fWVJHtwyC40XK1MmTIKDw/Xp59+6nBoY9WqVdq2bdt1x0t/n7dTt25dTZ8+3WGdCQkJWrp0qdq2bWtvq1SpkhITEx0OaRw5ckRz587Ndu7169c7HOr6448/NH/+fLVq1UouLi5ycXFRhw4dNGfOHCUkJGQZf+LECfv/t23bVn/99Zdmz55tb0tNTc3x8FRO3nvvPYfl//znP5Jk/7eQqXv37jpz5oyeffZZJScnO5ynkhNPT0+9+OKLSkhI0Msvv5zt3pKr2x577DG5uLhoxIgRWZ4zxtjPrSkMDRo0UKVKlTR+/Pgsh8Ikx9cbuFnYM4M72qJFi+x7WI4fP66ZM2dqz549evnll+3BICIiQi1bttQrr7yigwcPqk6dOlq6dKnmz5+vgQMH2v9SHTlypLZs2aIVK1bIx8dHtWvX1muvvaZhw4apY8eODr/Q3d3dtXjxYkVFRenee+/VokWL9N1332no0KEOJ9ZebeTIkVq2bJnuu+8+9e3bV66urvrwww+VlpbmcJ+QunXrysXFRWPGjFFiYqJsNpvuv/9+BQYGZjvvqFGj1L59ezVp0kRPPfWUzpw5o0mTJik8PDzbX1jZGTdunNq0aaNGjRqpV69eOn/+vP7zn//Iz8/P4fulunTpopdeekn//Oc/9dxzzyk1NVWTJ09WlSpVsj0/Jzw8XK1bt9Zzzz0nm82m999/X5I0YsQIe5/Ro0dr5cqVuvfee9WnTx/VqFFDp0+f1s8//6zly5fr9OnTkqQ+ffpo0qRJ6tGjhzZv3qzg4GB99tln8vT0zNU2Zjpw4IAeffRRPfzww1q/fr1mzJihbt26Zbm3TL169RQeHq5Zs2apevXqql+/fq7mf/nll7Vjxw6NGzdOS5cuVYcOHVSuXDmdOXNGP//8s2bNmqXAwEC5u7tL+jsgjhw5UjExMTp48KAiIyPl4+OjAwcOaO7cuXrmmWc0ZMiQPG1jTpydnfXRRx+pTZs2qlmzpp566imVLVtWhw8f1sqVK+Xr66tvv/22UNYF5FqRXEMFFLHsLs12d3c3devWNZMnT3a45NQYY86dO2cGDRpkypQpY4oVK2YqV65sxo0bZ++3efNm4+rq6nC5tTF/X07bsGFDU6ZMGfu9PqKiooyXl5fZt2+fadWqlfH09DRBQUFm+PDhJj093WG8rro02xhjfv75Z9O6dWvj7e1tPD09TcuWLc26deuybOPUqVNNxYoVjYuLS64uO/7yyy9NtWrVjM1mM+Hh4eabb74xHTp0MNWqVbP3ybw0e9y4cdnOsXz5ctOkSRPj4eFhfH19TUREhNm+fXuWfkuXLjXh4eHGzc3NVK1a1cyYMSPHS7Ojo6PNjBkzTOXKlY3NZjP16tXLdluOHTtmoqOjTUhIiClWrJgpXbq0eeCBB8yUKVMc+v3+++/m0UcfNZ6enqZUqVJmwIABZvHixXm6NHv79u2mY8eOxsfHx/j7+5t+/fqZ8+fPZztm7NixRpIZNWrUNefOzty5c03btm1NQECAcXV1NcWLFzf33XefGTdunDl79myW/nPmzDH33Xef8fLyMl5eXqZatWomOjra4T5HzZs3NzVr1swy9upL5jMvzZ41a1a2tf3yyy/mscceMyVLljQ2m82UL1/edOrUyaxYsSLP2wkUlJMxubjlKIBC07NnT82ePTvXezyKUt26dRUQEGC/iuZmc3JyUnR0dJZDfFYyceJEDRo0SAcPHtRdd91V1OUAtyXOmQGgS5cu2U98zhQfH6+tW7eqRYsWRVPUbcAYo48//ljNmzcnyAA3EOfMANDhw4f14IMP6sknn1SZMmW0c+dOffDBBypdurT+9a9/Feq69uzZo+joaP34449KSkrS3LlzdfbsWT311FM6cODAdb/lOzQ0VC1atFBcXFyh1lWYUlJS9M0332jlypXatm2b5s+fX9QlAbc1wgxwh9q3b5/Gjh2rZcuW6fDhw8rIyFBsbKzS09Pl6+urdu3aafTo0SpZsmShrjcqKkoHDhzQW2+9peLFi+vuu+/O8a7HVnXixAl169ZNxYsX19ChQ/Xoo48WdUnAbY1zZoA70HfffafHH39cNptNPXr0UHh4uC5evKg1a9Zozpw56tmzZ54vV86N8+fPy9PTU6+88opGjhxpb09PT9elS5dks9mue48cK+yZAXBzsWcGuMMcOHBAXbp0Ufny5fX9998rODjY/lx0dLT27t2r77777oasO/MeJMWLF3doz7xfDADkBycAA3eYsWPHKjk5WR9//LFDkMkUFhamAQMGSPr7bshvvvmmKlWqJJvNptDQUA0dOlRpaWkOY0JDQ/XII49ozZo1uueee+Tu7q6KFSvq008/tfd5/fXX7XcKfuGFF+Tk5GQ/PyYuLk5OTk4ONwM0xmjkyJEqV66cPD091bJlS/3222/ZbtPZs2c1cOBAhYSEyGazKSwsTGPGjHH4wsODBw/KyclJ48eP15QpU+zb1LBhQ/30009Z5ty5c6c6deqkgIAAeXh4qGrVqnrllVcc+hw+fFhPP/20goKCZLPZVLNmTX3yySfXePUB3AjsmQHuMN9++60qVqyoxo0bX7dv7969NX36dHXs2FHPP/+8fvzxR8XGxmrHjh1Z7ti7d+9edezYUb169VJUVJQ++eQT9ezZUw0aNFDNmjX12GOPqXjx4ho0aJC6du2qtm3bXvM7fF577TWNHDlSbdu2Vdu2bfXzzz+rVatWunjxokO/1NRUNW/eXIcPH9azzz6ru+66S+vWrVNMTIyOHDmiCRMmOPSfOXOmzp07p2effVZOTk4aO3asHnvsMe3fv9/+hYy//vqrmjZtqmLFiumZZ55RaGio9u3bp2+//VZvvfWWpL/vvPyPf/zD/o3nAQEBWrRokXr16qWkpCQNHDgwFz8NAIWiKG9yA+DmSkxMNJJM+/btr9t3y5YtRpLp3bu3Q/uQIUOMJPP999/b28qXL28kmdWrV9vbjh8/bmw2m3n++eftbTnddC/zJoYHDhywj3VzczPt2rVzuIHh0KFDjSQTFRVlb3vzzTeNl5eX2b17t8OcL7/8snFxcTGHDh1yWHfJkiXN6dOn7f3mz59vJJlvv/3W3tasWTPj4+Njfv/9d4c5r6ylV69eJjg42Jw8edKhT5cuXYyfn59JTU01AG4ODjMBd5CkpCRJcvjCzJwsXLhQkjR48GCH9ueff16SspxXU6NGDTVt2tS+HBAQoKpVq2r//v15rnP58uW6ePGi+vfv73BCcHZ7O2bNmqWmTZvK399fJ0+etD8efPBBpaena/Xq1Q79O3fuLH9/f/tyZs2ZdZ44cUKrV6/W008/neXeMJm1GGM0Z84cRUREyBjjsN7WrVsrMTEx269mAHBjcJgJuINkft/UuXPnrtv3999/l7Ozs8LCwhzaS5cureLFi2f5luvsbgrn7++vM2fO5LnOzLkrV67s0B4QEOAQRKS/71vz66+/5vidVsePH79mnZnzZdaZGWrCw8NzrO/EiRM6e/aspkyZkuNVX1evF8CNQ5gB7iC+vr4qU6ZMtt8unZPrXSqdKaerkcwNvvtDRkaGHnroIb344ovZPl+lShWH5cKoM/PE4ieffFJRUVHZ9qldu3au5wNQMIQZ4A7zyCOPaMqUKVq/fr0aNWqUY7/y5csrIyNDe/bsUfXq1e3tx44d09mzZ+1XJt0ImXPv2bNHFStWtLefOHEiy56eSpUqKTk5WQ8++GChrDtzfdcKfAEBAfLx8VF6enqhrRdA/nHODHCHefHFF+Xl5aXevXvr2LFjWZ7ft2+fJk6cqLZt20pSlquB3nnnHUlSu3btbliNDz74oIoVK6b//Oc/DntMrq5Fkjp16qT169dryZIlWZ47e/Zslu+cup6AgAA1a9ZMn3zyiQ4dOuTwXGYtLi4u6tChg+bMmZNt6Mm8nw6Am4M9M8AdplKlSpo5c6Y6d+6s6tWrO9wBeN26dZo1a5Z69uypAQMGKCoqSlOmTNHZs2fVvHlzbdy4UdOnT1dkZKRatmx5w2oMCAjQkCFDFBsbq0ceeURt27bVL7/8okWLFqlUqVIOfV944QV98803euSRR+yXgqekpGjbtm2aPXu2Dh48mGXM9fz73//Wfffdp/r16+uZZ55RhQoVdPDgQX333XfasmWLJGn06NFauXKl7r33XvXp00c1atTQ6dOn9fPPP2v58uU6ffp0Yb0cAK6DMAPcgR599FH9+uuvGjdunObPn6/JkyfLZrOpdu3aevvtt9WnTx9J0kcffaSKFSsqLi5Oc+fOVenSpRUTE6Phw4ff8BpHjhwpd3d3ffDBB/bQsHTp0ix7hDw9PbVq1SqNGjVKs2bN0qeffipfX19VqVJFI0aMkJ+fX57XXadOHW3YsEGvvvqqJk+erAsXLqh8+fLq1KmTvU9QUJA2btyoN954Q19//bXef/99lSxZUjVr1tSYMWMKvP0Aco/vZgIAAJbGOTMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSLHGfmYyMDP3111/y8fHJ9ffEAAAAazPG6Ny5cypTpoycnXPe/2KJMPPXX38pJCSkqMsAAABF4I8//lC5cuVyfN4SYcbHx0fS3xvj6+tbxNUAAICbISkpSSEhIfYckBNLhJnMQ0u+vr6EGQAA7jDXO8WEE4ABAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAICl5SnMTJ48WbVr17bfibdRo0ZatGjRNcfMmjVL1apVk7u7u2rVqqWFCxcWqGAAAIAr5SnMlCtXTqNHj9bmzZu1adMm3X///Wrfvr1+++23bPuvW7dOXbt2Va9evfTLL78oMjJSkZGRSkhIKJTiAQAAnIwxpiATlChRQuPGjVOvXr2yPNe5c2elpKRowYIF9rZ//OMfqlu3rj744INcryMpKUl+fn5KTEzku5kAALhD5Pb3f76/aDI9PV2zZs1SSkqKGjVqlG2f9evXa/DgwQ5trVu31rx58645d1pamtLS0uzLSUlJ+S0Tt7DU1FTt3Lnzuv0uXErXn2dSVc7fU+7FXK7Zt1q1avL09CysEgHcILz/UZjyHGa2bdumRo0a6cKFC/L29tbcuXNVo0aNbPsePXpUQUFBDm1BQUE6evToNdcRGxurESNG5LU0WMzOnTvVoEGDQp1z8+bNql+/fqHOCaDw8f5HYcpzmKlataq2bNmixMREzZ49W1FRUVq1alWOgSY/YmJiHPboJCUlKSQkpNDmx62hWrVq2rx583X77T1+TgO+3KKJXeoqLNDnunMCuPXx/kdhynOYcXNzU1hYmCSpQYMG+umnnzRx4kR9+OGHWfqWLl1ax44dc2g7duyYSpcufc112Gw22Wy2vJYGi/H09MzVX1FuhxNli09WjVp1FV7W7yZUBuBG4/2PwlTg+8xkZGQ4nN9ypUaNGmnFihUObcuWLcvxHBsAAIC8ytOemZiYGLVp00Z33XWXzp07p5kzZyo+Pl5LliyRJPXo0UNly5ZVbGysJGnAgAFq3ry53n77bbVr105ffvmlNm3apClTphT+lgAAgDtSnsLM8ePH1aNHDx05ckR+fn6qXbu2lixZooceekiSdOjQITk7/29nT+PGjTVz5kwNGzZMQ4cOVeXKlTVv3jyFh4cX7lYAAIA7Vp7CzMcff3zN5+Pj47O0Pf7443r88cfzVBQAAEBu8d1MAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0ggzAADA0vIUZmJjY9WwYUP5+PgoMDBQkZGR2rVr13XHTZgwQVWrVpWHh4dCQkI0aNAgXbhwId9FAwAAZMpTmFm1apWio6O1YcMGLVu2TJcuXVKrVq2UkpKS45iZM2fq5Zdf1vDhw7Vjxw59/PHH+uqrrzR06NACFw8AAOCal86LFy92WI6Li1NgYKA2b96sZs2aZTtm3bp1atKkibp16yZJCg0NVdeuXfXjjz/ms2QAAID/yVOYuVpiYqIkqUSJEjn2ady4sWbMmKGNGzfqnnvu0f79+7Vw4UJ17949xzFpaWlKS0uzLyclJRWkTBSRAydTlJJ2ucDz7D2e7PDfgvKyuapCKa9CmQsAUPTyHWYyMjI0cOBANWnSROHh4Tn269atm06ePKn77rtPxhhdvnxZ//rXv655mCk2NlYjRozIb2m4BRw4maKW4+MLdc6BX20ptLlWDmlBoAGA20S+w0x0dLQSEhK0Zs2aa/aLj4/XqFGj9P777+vee+/V3r17NWDAAL355pt69dVXsx0TExOjwYMH25eTkpIUEhKS31JRBDL3yEzoXFdhgd4FmuvCpXT9eea8yvl7yL2YS4Hm2ns8WQO/2lIoe4wAALeGfIWZfv36acGCBVq9erXKlSt3zb6vvvqqunfvrt69e0uSatWqpZSUFD3zzDN65ZVX5Oyc9Rxkm80mm82Wn9JwiwkL9FZ4Wb8Cz3N3aMFrAQDcnvIUZowx6t+/v+bOnav4+HhVqFDhumNSU1OzBBYXFxf7fAAAAAWRpzATHR2tmTNnav78+fLx8dHRo0clSX5+fvLw8JAk9ejRQ2XLllVsbKwkKSIiQu+8847q1atnP8z06quvKiIiwh5qAAAA8itPYWby5MmSpBYtWji0T5s2TT179pQkHTp0yGFPzLBhw+Tk5KRhw4bp8OHDCggIUEREhN56662CVQ4AAKB8HGa6nvj4eMcVuLpq+PDhGj58eJ4KAwAAyA2+mwkAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFgaYQYAAFiaa1EXgNuXk2uSDiTtkrO7d1GXYncgKVlOrklFXQYAoBARZnDDFCv+o4ZuHFXUZWRRrPgDktoWdRkAgEJCmMENc+nsvXq7XTdVCrx19szsO56s5z7fV9RlAAAKEWEGN4y57KsKvlVVo6RfUZdil3EhUebyiaIuAwBQiDgBGAAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWBphBgAAWFqewkxsbKwaNmwoHx8fBQYGKjIyUrt27bruuLNnzyo6OlrBwcGy2WyqUqWKFi5cmO+iAQAAMrnmpfOqVasUHR2thg0b6vLlyxo6dKhatWql7du3y8vLK9sxFy9e1EMPPaTAwEDNnj1bZcuW1e+//67ixYsXRv0AAOAOl6cws3jxYofluLg4BQYGavPmzWrWrFm2Yz755BOdPn1a69atU7FixSRJoaGh+asWAADgKnkKM1dLTEyUJJUoUSLHPt98840aNWqk6OhozZ8/XwEBAerWrZteeuklubi4ZDsmLS1NaWlp9uWkpKSClAkAuMkOnExRStrlAs+z93iyw38Lysvmqgqlsj+SAOvKd5jJyMjQwIED1aRJE4WHh+fYb//+/fr+++/1xBNPaOHChdq7d6/69u2rS5cuafjw4dmOiY2N1YgRI/JbGgCgCB04maKW4+MLdc6BX20ptLlWDmlBoLnN5DvMREdHKyEhQWvWrLlmv4yMDAUGBmrKlClycXFRgwYNdPjwYY0bNy7HMBMTE6PBgwfbl5OSkhQSEpLfUgEAN1HmHpkJnesqLNC7QHNduJSuP8+cVzl/D7kXy35vfm7tPZ6sgV9tKZQ9Rri15CvM9OvXTwsWLNDq1atVrly5a/YNDg5WsWLFHA4pVa9eXUePHtXFixfl5uaWZYzNZpPNZstPaQCAW0RYoLfCy/oVeJ67QwteC25vebo02xijfv36ae7cufr+++9VoUKF645p0qSJ9u7dq4yMDHvb7t27FRwcnG2QAQAAyIs8hZno6GjNmDFDM2fOlI+Pj44ePaqjR4/q/Pnz9j49evRQTEyMffn//u//dPr0aQ0YMEC7d+/Wd999p1GjRik6OrrwtgIAANyx8nSYafLkyZKkFi1aOLRPmzZNPXv2lCQdOnRIzs7/y0ghISFasmSJBg0apNq1a6ts2bIaMGCAXnrppYJVDgAAoDyGGWPMdfvEx8dnaWvUqJE2bNiQl1UBAADkCt/NBAAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALI0wAwAALC1PYSY2NlYNGzaUj4+PAgMDFRkZqV27duV6/JdffiknJydFRkbmtU4AAIBs5SnMrFq1StHR0dqwYYOWLVumS5cuqVWrVkpJSbnu2IMHD2rIkCFq2rRpvosFAAC4mmteOi9evNhhOS4uToGBgdq8ebOaNWuW47j09HQ98cQTGjFihH744QedPXv2mutJS0tTWlqafTkpKSkvZeIWcP5SuiQp4XBigee6cCldf545r3L+HnIv5lKgufYeTy5wPQCuz8k1SQeSdsnZ3buoS7E7kJQsJ1d+n9yO8hRmrpaY+PcvqhIlSlyz3xtvvKHAwED16tVLP/zww3XnjY2N1YgRIwpSGorYvv8fGl7+elsRV5I9L1uB/ukDuI5ixX/U0I2jirqMLIoVf0BS26IuA4Us35/oGRkZGjhwoJo0aaLw8PAc+61Zs0Yff/yxtmzZkuu5Y2JiNHjwYPtyUlKSQkJC8lsqikCrmqUlSZUCveVRCHtTBn61RRM611VYYMH/yvOyuapCKa8CzwMgZ5fO3qu323VTpUJ4zxaWfceT9dzn+4q6DNwA+Q4z0dHRSkhI0Jo1a3Lsc+7cOXXv3l1Tp05VqVKlcj23zWaTzWbLb2m4BZTwclOXe+4q1DnDAr0VXtavUOcEcGOYy76q4FtVNUreOu/ZjAuJMpdPFHUZuAHyFWb69eunBQsWaPXq1SpXrlyO/fbt26eDBw8qIiLC3paRkfH3il1dtWvXLlWqVCk/JQAAAEjKY5gxxqh///6aO3eu4uPjVaFChWv2r1atmrZtczxnYtiwYTp37pwmTpzIoSMAAFBgeQoz0dHRmjlzpubPny8fHx8dPXpUkuTn5ycPDw9JUo8ePVS2bFnFxsbK3d09y/k0xYsXl6RrnmcDAACQW3kKM5MnT5YktWjRwqF92rRp6tmzpyTp0KFDcnbmxsIAAODmyPNhpuuJj4+/5vNxcXF5WSUAAMA1sQsFAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYGmEGAABYWp7CTGxsrBo2bCgfHx8FBgYqMjJSu3btuuaYqVOnqmnTpvL395e/v78efPBBbdy4sUBFAwAAZMpTmFm1apWio6O1YcMGLVu2TJcuXVKrVq2UkpKS45j4+Hh17dpVK1eu1Pr16xUSEqJWrVrp8OHDBS4eAADANS+dFy9e7LAcFxenwMBAbd68Wc2aNct2zOeff+6w/NFHH2nOnDlasWKFevTokcdyAQAAHOUpzFwtMTFRklSiRIlcj0lNTdWlS5euOSYtLU1paWn25aSkpPwXCQC4qc5fSpckJRxOLPBcFy6l688z51XO30PuxVwKNNfe48kFrge3pnyHmYyMDA0cOFBNmjRReHh4rse99NJLKlOmjB588MEc+8TGxmrEiBH5LQ0AUIT2/f/Q8PLX24q4kux52Qr0dzxuQfn+iUZHRyshIUFr1qzJ9ZjRo0fryy+/VHx8vNzd3XPsFxMTo8GDB9uXk5KSFBISkt9SAQA3UauapSVJlQK95VEIe1MGfrVFEzrXVVigd4Fr87K5qkIprwLPg1tLvsJMv379tGDBAq1evVrlypXL1Zjx48dr9OjRWr58uWrXrn3NvjabTTabLT+lAQCKWAkvN3W5565CnTMs0FvhZf0KdU7cPvIUZowx6t+/v+bOnav4+HhVqFAhV+PGjh2rt956S0uWLNHdd9+dr0IBAACyk6cwEx0drZkzZ2r+/Pny8fHR0aNHJUl+fn7y8PCQJPXo0UNly5ZVbGysJGnMmDF67bXXNHPmTIWGhtrHeHt7y9u74LsMAQDAnS1P95mZPHmyEhMT1aJFCwUHB9sfX331lb3PoUOHdOTIEYcxFy9eVMeOHR3GjB8/vvC2AgAA3LHyfJjpeuLj4x2WDx48mJdVAAAA5AnfzQQAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACyNMAMAACwtT2EmNjZWDRs2lI+PjwIDAxUZGaldu3Zdd9ysWbNUrVo1ubu7q1atWlq4cGG+CwYAALhSnsLMqlWrFB0drQ0bNmjZsmW6dOmSWrVqpZSUlBzHrFu3Tl27dlWvXr30yy+/KDIyUpGRkUpISChw8QAAAK556bx48WKH5bi4OAUGBmrz5s1q1qxZtmMmTpyohx9+WC+88IIk6c0339SyZcs0adIkffDBB/ksGwAA4G95CjNXS0xMlCSVKFEixz7r16/X4MGDHdpat26tefPm5TgmLS1NaWlp9uWkpKSClIlbVGpqqnbu3HndfnuPn1Pa0b3avs1bF4/5XLNvtWrV5OnpWVglAgAsIN9hJiMjQwMHDlSTJk0UHh6eY7+jR48qKCjIoS0oKEhHjx7NcUxsbKxGjBiR39JgETt37lSDBg1y3b/z9Ov32bx5s+rXr1+AqgAAVpPvMBMdHa2EhAStWbOmMOuRJMXExDjszUlKSlJISEihrwdFq1q1atq8efN1+124lK4/z6SqnL+n3Iu5XHdOAMCdJV9hpl+/flqwYIFWr16tcuXKXbNv6dKldezYMYe2Y8eOqXTp0jmOsdlsstls+SkNFuLp6cleFABAgeXpaiZjjPr166e5c+fq+++/V4UKFa47plGjRlqxYoVD27Jly9SoUaO8VQoAAJCNPO2ZiY6O1syZMzV//nz5+PjYz3vx8/OTh4eHJKlHjx4qW7asYmNjJUkDBgxQ8+bN9fbbb6tdu3b68ssvtWnTJk2ZMqWQNwUAANyJ8rRnZvLkyUpMTFSLFi0UHBxsf3z11Vf2PocOHdKRI0fsy40bN9bMmTM1ZcoU1alTR7Nnz9a8efOuedIwAABAbuVpz4wx5rp94uPjs7Q9/vjjevzxx/OyKgAAgFzhu5kAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAIClEWYAAICl5TnMrF69WhERESpTpoycnJw0b9686475/PPPVadOHXl6eio4OFhPP/20Tp06lZ96AQAAHOQ5zKSkpKhOnTp67733ctV/7dq16tGjh3r16qXffvtNs2bN0saNG9WnT588FwsAAHA117wOaNOmjdq0aZPr/uvXr1doaKiee+45SVKFChX07LPPasyYMXldNQAAQBZ5DjN51ahRIw0dOlQLFy5UmzZtdPz4cc2ePVtt27bNcUxaWprS0tLsy0lJSTe6TADATZSamqqdO3det9/e4+eUdnSvtm/z1sVjPtfsW61aNXl6ehZWibCQGx5mmjRpos8//1ydO3fWhQsXdPnyZUVERFzzMFVsbKxGjBhxo0sDABSRnTt3qkGDBrnu33n69fts3rxZ9evXL0BVsConY4zJ92AnJ82dO1eRkZE59tm+fbsefPBBDRo0SK1bt9aRI0f0wgsvqGHDhvr444+zHZPdnpmQkBAlJibK19c3v+UCAG4Rud0zc+FSuv48k6py/p5yL+Zyzb7smbn9JCUlyc/P77q//294mOnevbsuXLigWbNm2dvWrFmjpk2b6q+//lJwcPB115PbjQEAALeP3P7+v+H3mUlNTZWzs+NqXFz+TtcFyFEAAACS8hFmkpOTtWXLFm3ZskWSdODAAW3ZskWHDh2SJMXExKhHjx72/hEREfr66681efJk7d+/X2vXrtVzzz2ne+65R2XKlCmcrQAAAHesPJ8AvGnTJrVs2dK+PHjwYElSVFSU4uLidOTIEXuwkaSePXvq3LlzmjRpkp5//nkVL15c999/P5dmAwCAQlGgc2ZuFs6ZAQDgznPLnDMDAABwIxFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApbkWdQEAAGQnPT1dP/zwg44cOaLg4GA1bdpULi4uRV0WbkHsmQEA3HK+/vprhYWFqWXLlurWrZtatmypsLAwff3110VdGm5BeQ4zq1evVkREhMqUKSMnJyfNmzfvumPS0tL0yiuvqHz58rLZbAoNDdUnn3ySn3oBALe5r7/+Wh07dlStWrW0fv16nTt3TuvXr1etWrXUsWNHAg2yyPNhppSUFNWpU0dPP/20HnvssVyN6dSpk44dO6aPP/5YYWFhOnLkiDIyMvJcLADg9paenq7nn39ejzzyiObNmydn57//5v7HP/6hefPmKTIyUkOGDFH79u055AS7PIeZNm3aqE2bNrnuv3jxYq1atUr79+9XiRIlJEmhoaHXHJOWlqa0tDT7clJSUl7LBABY0A8//KCDBw/qiy++sAeZTM7OzoqJiVHjxo31ww8/qEWLFkVTJG45N/ycmW+++UZ33323xo4dq7Jly6pKlSoaMmSIzp8/n+OY2NhY+fn52R8hISE3ukwAwC3gyJEjkqTw8PBsn89sz+wHSDchzOzfv19r1qxRQkKC5s6dqwkTJmj27Nnq27dvjmNiYmKUmJhof/zxxx83ukwAwC0gODhYkpSQkJDt85ntmf0A6SaEmYyMDDk5Oenzzz/XPffco7Zt2+qdd97R9OnTc9w7Y7PZ5Ovr6/AAANz+mjZtqtDQUI0aNSrLuZUZGRmKjY1VhQoV1LRp0yKqELeiGx5mgoODVbZsWfn5+dnbqlevLmOM/vzzzxu9egCAhbi4uOjtt9/WggULFBkZ6XA1U2RkpBYsWKDx48dz8i8c3PAw06RJE/31119KTk62t+3evVvOzs4qV67cjV49AMBiHnvsMc2ePVvbtm1T48aN5evrq8aNGyshIUGzZ8/O9ZW0uHM4GWNMXgYkJydr7969kqR69erpnXfeUcuWLVWiRAndddddiomJ0eHDh/Xpp5/a+1evXl3/+Mc/NGLECJ08eVK9e/dW8+bNNXXq1FytMykpSX5+fkpMTOSQEwDcIbgDMHL7+z/Pl2Zv2rRJLVu2tC8PHjxYkhQVFaW4uDgdOXJEhw4dsj/v7e2tZcuWqX///rr77rtVsmRJderUSSNHjszrqgEAdxAXFxcuv0au5HnPTFFgzwwAAHee3P7+57uZAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApRFmAACApeX5DsBFIfO+fklJSUVcCQAAuFkyf+9f7/6+lggz586dkySFhIQUcSUAAOBmO3funPz8/HJ83hJfZ5CRkaG//vpLPj4+cnJyKupycJMlJSUpJCREf/zxB19nAdxheP/f2YwxOnfunMqUKSNn55zPjLHEnhlnZ2eVK1euqMtAEfP19eXDDLhD8f6/c11rj0wmTgAGAACWRpgBAACWRpjBLc9ms2n48OGy2WxFXQqAm4z3P3LDEicAAwAA5IQ9MwAAwNIIMwAAwNIIMwAAwNIIM7hlhYaGasKECTdkbicnJ82bN6/I5wBuVXFxcSpevHhRl1EgLVq00MCBA4t8DtwEBpYXFRVlJBlJxtXV1YSGhpoXXnjBnD9/vqhLy5Vp06YZPz+/LO3Hjx83KSkp+Z63fPny9tcl81G2bFljjDFHjhwxFy5cyNU8w4cPN3Xq1MnSnpc5gJvt+PHj5l//+pcJCQkxbm5uJigoyLRq1cqsWbMmV+NTU1PNsWPHbnCVBXfl59+Vjz179phTp06ZpKSkXM2zcuVKI8mcOXPGoT0vc6DoWOIOwLi+hx9+WNOmTdOlS5e0efNmRUVFycnJSWPGjCnq0vItICCgwHO88cYb6tOnj33ZxcVFklS6dOkCz10YcwA3SocOHXTx4kVNnz5dFStW1LFjx7RixQqdOnUqV+M9PDzk4eFxg6ssHJmff1cKCAiwv98LokSJEgWeAzdBUacpFFxUVJRp3769Q9tjjz1m6tWrZ1++cOGC6d+/vwkICDA2m800adLEbNy40f585l8lixcvNnXr1jXu7u6mZcuW5tixY2bhwoWmWrVqxsfHx3Tt2tVhb8miRYtMkyZNjJ+fnylRooRp166d2bt3r/35AwcOGElmzpw5pkWLFsbDw8PUrl3brFu3zmG9Vz6GDx9ujPl7z8q7775rn+vMmTPmmWeeMYGBgcZms5maNWuab7/9NsfX5erxV5Jk5s6da1/+448/TJcuXYy/v7/x9PQ0DRo0MBs2bDDTpk3LUt+0adOynePXX381LVu2NO7u7qZEiRKmT58+5ty5c1l+TuPGjTOlS5c2JUqUMH379jUXL17McRuA/Dhz5oyRZOLj46/bL6f3VHZ7TOfNm2fq1atnbDabqVChgnn99dfNpUuX7M9LMlOnTjWRkZHGw8PDhIWFmfnz5zvMkZCQYNq1a2d8fHyMt7e3ue+++xw+M6ZOnWqqVatmbDabqVq1qnnvvfeuuQ3Zff5lat68uRkwYIB9+cKFC+bFF1805cqVM25ubqZSpUrmo48+sn9OXfmIiorKdo7Tp0+b7t27m+LFixsPDw/z8MMPm927d9ufz3zdFi9ebKpVq2a8vLxM69atzV9//XXN7UDBcM7MbSghIUHr1q2Tm5ubve3FF1/UnDlzNH36dP38888KCwtT69atdfr0aYexr7/+uiZNmqR169bpjz/+UKdOnTRhwgTNnDlT3333nZYuXar//Oc/9v4pKSkaPHiwNm3apBUrVsjZ2Vn//Oc/lZGR4TDvK6+8oiFDhmjLli2qUqWKunbtqsuXL6tx48aaMGGCfH19deTIER05ckRDhgzJsk0ZGRlq06aN1q5dqxkzZmj79u0aPXp0ofzllZycrObNm+vw4cP65ptvtHXrVr344ovKyMhQ586d9fzzz6tmzZr2+jp37pxljpSUFLVu3Vr+/v766aefNGvWLC1fvlz9+vVz6Ldy5Urt27dPK1eu1PTp0xUXF6e4uLgCbwNwJW9vb3l7e2vevHlKS0vLtk9e31M//PCDevTooQEDBmj79u368MMPFRcXp7feesuh34gRI9SpUyf9+uuvatu2rZ544gn758zhw4fVrFkz2Ww2ff/999q8ebOefvppXb58WZL0+eef67XXXtNbb72lHTt2aNSoUXr11Vc1ffr0QnldevTooS+++EL//ve/tWPHDn344Yfy9vZWSEiI5syZI0natWuXjhw5ookTJ2Y7R8+ePbVp0yZ98803Wr9+vYwxatu2rS5dumTvk5qaqvHjx+uzzz7T6tWrdejQoWw/11CIijpNoeCioqKMi4uL8fLyMjabzUgyzs7OZvbs2cYYY5KTk02xYsXM559/bh9z8eJFU6ZMGTN27FhjzP/2kCxfvtzeJzY21kgy+/bts7c9++yzpnXr1jnWcuLECSPJbNu2zRjzvz0zH330kb3Pb7/9ZiSZHTt2GGNyPmfmyj0rS5YsMc7OzmbXrl25fl3Kly9v3NzcjJeXl/0xceJEY4zjXpUPP/zQ+Pj4mFOnTmU7T07nzFw5x5QpU4y/v79JTk62P//dd98ZZ2dnc/ToUWPM3z+n8uXLm8uXL9v7PP7446Zz58653iYgt2bPnm38/f2Nu7u7ady4sYmJiTFbt261P3+999TV78sHHnjAjBo1yqHPZ599ZoKDg+3LksywYcPsy8nJyUaSWbRokTHGmJiYGFOhQoUc90ZWqlTJzJw506HtzTffNI0aNcpxO6/8/Mt8dOzY0RjjuFdl165dRpJZtmxZtvPkdM7MlXPs3r3bSDJr1661P3/y5Enj4eFh/vvf/xpjjH1v7pV7m9577z0TFBSU4zag4Dhn5jbRsmVLTZ48WSkpKXr33Xfl6uqqDh06SJL27dunS5cuqUmTJvb+xYoV0z333KMdO3Y4zFO7dm37/wcFBcnT01MVK1Z0aNu4caN9ec+ePXrttdf0448/6uTJk/Y9MocOHVJ4eHi28wYHB0uSjh8/rmrVquVq+7Zs2aJy5cqpSpUqueqf6YUXXlDPnj3ty6VKlcp27nr16hXo2PiOHTtUp04deXl52duaNGmijIwM7dq1S0FBQZKkmjVrOvzlGxwcrG3btuV7vUBOOnTooHbt2umHH37Qhg0btGjRIo0dO1YfffSRevbsmef31NatW7V27VqHPTHp6em6cOGCUlNT5enpKcnxve7l5SVfX18dP35c0t/vtaZNm6pYsWJZ5k9JSdG+ffvUq1cvh/PcLl++fN1vTc78/LtyvVfbsmWLXFxc1Lx581xtb3Z27NghV1dX3Xvvvfa2kiVLqmrVqg6fpZ6enqpUqZJ9OTg42P4a4MYgzNwmvLy8FBYWJkn65JNPVKdOHX388cfq1atXnua58kPGyckpy4eOk5OTwyGkiIgIlS9fXlOnTlWZMmWUkZGh8PBwXbx48ZrzSspyKOpa8nsiYqlSpeyvS2HPnR/Xez2BwuTu7q6HHnpIDz30kF599VX17t1bw4cPV8+ePfP87z45OVkjRozQY489lu16Ml3r3/i11pmcnCxJmjp1qkNYkHTdw8lXfv7lpKjf54ZvDrqhOGfmNuTs7KyhQ4dq2LBhOn/+vCpVqiQ3NzetXbvW3ufSpUv66aefVKNGjXyv59SpU9q1a5eGDRumBx54QNWrV9eZM2fyPI+bm5vS09Ov2ad27dr6888/tXv37vyWe825t2zZkuX8obzUV716dW3dulUpKSn2trVr18rZ2VlVq1Yt1HqB/KpRo4b932he31P169fXrl27FBYWluXh7Jy7XyW1a9fWDz/84HB+SaagoCCVKVNG+/fvzzJ/hQoVcr+ROahVq5YyMjK0atWqbJ/PPMfwWu/16tWr6/Lly/rxxx/tbZmfgwX5LEXBEWZuU48//rhcXFz03nvvycvLS//3f/+nF154QYsXL9b27dvVp08fpaam5nnPzZX8/f1VsmRJTZkyRXv37tX333+vwYMH53me0NBQJScna8WKFTp58qRSU1Oz9GnevLmaNWumDh06aNmyZTpw4IAWLVqkxYsX57v+TF27dlXp0qUVGRmptWvXav/+/ZozZ47Wr19vr+/AgQPasmWLTp48me0JlU888YTc3d0VFRWlhIQErVy5Uv3791f37t3th5iAm+XUqVO6//77NWPGDP366686cOCAZs2apbFjx6p9+/aS8v6eeu211/Tpp59qxIgR+u2337Rjxw59+eWXGjZsWK7r6tevn5KSktSlSxdt2rRJe/bs0WeffaZdu3ZJ+vvk4djYWP373//W7t27tW3bNk2bNk3vvPNOgV+T0NBQRUVF6emnn9a8efN04MABxcfH67///a8kqXz58nJyctKCBQt04sQJ+56iK1WuXFnt27dXnz59tGbNGm3dulVPPvmkypYta39dUTQIM7cpV1dX9evXT2PHjlVKSopGjx6tDh06qHv37qpfv7727t2rJUuWyN/fP9/rcHZ21pdffqnNmzcrPDxcgwYN0rhx4/I8T+PGjfWvf/1LnTt3VkBAgMaOHZttvzlz5qhhw4bq2rWratSooRdffPG6e0xyw83NTUuXLlVgYKDatm2rWrVqOVzV0aFDBz388MNq2bKlAgIC9MUXX2SZw9PTU0uWLNHp06fVsGFDdezYUQ888IAmTZpU4PqAvPL29ta9996rd999V82aNVN4eLheffVV9enTx+HfZF7eU61bt9aCBQu0dOlSNWzYUP/4xz/07rvvqnz58rmuq2TJkvr+++/tVxA2aNBAU6dOtR+W6d27tz766CNNmzZNtWrVUvPmzRUXF1coe2YkafLkyerYsaP69u2ratWqqU+fPvY9VWXLltWIESP08ssvKygoKMuViJmmTZumBg0a6JFHHlGjRo1kjNHChQuzPQ8IN4+T4UAeAACwMPbMAAAASyPMAAAASyPMAAAASyPMAAAASyPMAAAASyPMAAAASyPMAAAASyPMAAAASyPMAAAASyPMALhhjh49qgEDBigsLEzu7u4KCgpSkyZNNHny5Gy/gwsA8sO1qAsAcHvav3+/mjRpouLFi2vUqFGqVauWbDabtm3bpilTpqhs2bJ69NFHb8i6L168aP8WZAC3P/bMALgh+vbtK1dXV23atEmdOnVS9erVVbFiRbVv317fffedIiIiJElnz55V7969FRAQIF9fX91///3aunWrfZ7XX39ddevW1WeffabQ0FD5+fmpS5cuOnfunL1PixYt1K9fPw0cOFClSpVS69atJUkJCQlq06aNvL29FRQUpO7du+vkyZM394UAcMMRZgAUulOnTmnp0qWKjo6Wl5dXtn2cnJwkSY8//riOHz+uRYsWafPmzapfv74eeOABnT592t533759mjdvnhYsWKAFCxZo1apVGj16tMN806dPl5ubm9auXasPPvhAZ8+e1f3336969epp06ZNWrx4sY4dO6ZOnTrduA0HUCQ4zASg0O3du1fGGFWtWtWhvVSpUrpw4YIkKTo6WhEREdq4caOOHz8um80mSRo/frzmzZun2bNn65lnnpEkZWRkKC4uTj4+PpKk7t27a8WKFXrrrbfsc1euXFljx461L48cOVL16tXTqFGj7G2ffPKJQkJCtHv3blWpUuXGbDyAm44wA+Cm2bhxozIyMvTEE08oLS1NW7duVXJyskqWLOnQ7/z589q3b599OTQ01B5kJCk4OFjHjx93GNOgQQOH5a1bt2rlypXy9vbOUse+ffsIM8BthDADoNCFhYXJyclJu3btcmivWLGiJMnDw0OSlJycrODgYMXHx2eZo3jx4vb/L1asmMNzTk5OysjIcGi7+nBWcnKyIiIiNGbMmCxzBwcH53pbANz6CDMACl3JkiX10EMPadKkSerfv3+O583Ur19fR48elaurq0JDQwu1hvr162vOnDkKDQ2VqysfdcDtjBOAAdwQ77//vi5fvqy7775bX331lXbs2KFdu3ZpxowZ2rlzp1xcXPTggw+qUaNGioyM1NKlS3Xw4EGtW7dOr7zyijZt2lSg9UdHR+v06dPq2rWrfvrpJ+3bt09LlizRU089pfT09ELaSgC3Av5cAXBDVKpUSb/88otGjRqlmJgY/fnnn7LZbKpRo4aGDBmivn37ysnJSQsXLtQrr7yip556SidOnFDp0qXVrFkzBQUFFWj9ZcqU0dq1a/XSSy+pVatWSktLU/ny5fXwww/L2Zm/44DbiZMxxhR1EQAAAPnFnycAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDSCDMAAMDS/h97LT9nsuGPpwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "scifi_confidences = []\n",
    "romantic_confidences = []\n",
    "\n",
    "for sample in dataset['samples']:\n",
    "    if sample['genre'] == 'SCI':\n",
    "        scifi_confidences.append(sample['annotations']['confidence']['average_confidence'])\n",
    "    if sample['genre'] == 'ROM':\n",
    "        romantic_confidences.append(sample['annotations']['confidence']['average_confidence'])\n",
    "\n",
    "genre_df = pd.DataFrame({\n",
    "    'Genre': ['Science Fiction'] * len(scifi_confidences) + ['Romantic Fiction'] * len(romantic_confidences),\n",
    "    'Confidence': scifi_confidences + romantic_confidences\n",
    "})\n",
    "\n",
    "genre_df.boxplot(by='Genre', column='Confidence', grid=False)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows a remarkable difference in the average confidence levels between the two genres. Let's see if this is something that's spread equally across annotators by plotting the confidence levels per genre individually:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwsAAAPdCAYAAADFwfSIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAC9xUlEQVR4nOzdeVyU9f7//+fILggIImAi4JKK5poaWrlLWqYntezrp7Bj2qKUWtnxlGuLHeukLWRphbaYZZ6sLDUjNXPJ7WiahkuYpAKuICiL8P790c85TowGCMwAj/vtdt1uzfu65n29Zpjx2Wuua66xGGOMAAAAAOBPaji6AAAAAADOiWYBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALCLZgEAAACAXTQLQBk7evSopk6dqh07dpR6jq+//lpTp04ts5qKY86cORoyZIgaNGggi8Wi4cOHV+j+AaAqqYxZkJKSomnTpqljx46qXbu26tSpo27duunbb7+tsBrgfGgWgDJ29OhRTZs27aoDYtq0aWVXVDH861//0nfffacWLVrI1dW1QvcNAFVNZcyCzz//XP/617/UuHFjPfvss5o0aZLOnj2r3r17KyEhocLqgHPh/wiAaqKwsFB5eXny9PS0u37t2rXWowo+Pj4VXB0AoCJcKQu6d++uw4cPq06dOtaxBx98UG3atNHkyZN13333VWSpcBIcWUCl8ttvv+nhhx9W06ZN5eXlpcDAQA0ZMkSHDh2y2W7+/PmyWCxav369xo8fr6CgIHl7e+tvf/ubjh8/brNtRESEbrvtNv3www/q2LGjPD091bBhQ7333ntF9v/rr79qyJAhCggIUM2aNXXDDTfoq6++sq5fs2aNOnToIEm67777ZLFYZLFYNH/+fEnSunXrrKf6eHh4KCwsTOPGjdP58+etcwwfPlzx8fGSZL2/xWKxrs/OztZjjz2msLAweXh4qGnTpnrppZdkjLGp1WKxaMyYMfrwww/VokULeXh4aMWKFZd9bsPDw232AwDOiiwonyxo0aKFTaMgSR4eHurXr59+//13nT171u79ULVxZAGVypYtW7RhwwYNHTpU9evX16FDhzRnzhx169ZNe/bsUc2aNW22j4uLU+3atTVlyhQdOnRIs2fP1pgxY/Txxx/bbHfgwAENHjxYI0aMUGxsrN59910NHz5c7du3V4sWLSRJaWlp6ty5s86dO6dHHnlEgYGBWrBggW6//XZ9+umn+tvf/qbmzZtr+vTpmjx5skaNGqWbbrpJktS5c2dJ0uLFi3Xu3Dk99NBDCgwM1ObNm/Xaa6/p999/1+LFiyVJDzzwgI4ePapVq1bp/ffft6nTGKPbb79dq1ev1ogRI9SmTRutXLlSTzzxhI4cOaJZs2bZbP/dd9/pk08+0ZgxY1SnTh1FRESU2d8CAByFLKjYLEhNTVXNmjWLPK+oJgxQiZw7d67I2MaNG40k895771nHEhISjCTTq1cvU1hYaB0fN26ccXFxMWfOnLGOhYeHG0nm+++/t46lp6cbDw8P89hjj1nHxo4daySZdevWWcfOnj1rIiMjTUREhCkoKDDGGLNlyxYjySQkJBSr/hkzZhiLxWJ+++0369jo0aONvbfn0qVLjSTz7LPP2owPHjzYWCwWc+DAAeuYJFOjRg3z888/F5nnr3h7e5vY2NgS3w8AKgJZUDFZYIwx+/fvN56enuaee+4p1f1R+XEaEioVLy8v63/n5+fr5MmTaty4sfz9/bV9+/Yi248aNcrmsO1NN92kgoIC/fbbbzbbRUVFWT/5kaSgoCA1bdpUv/76q3Xs66+/VseOHXXjjTdax3x8fDRq1CgdOnRIe/bsKVH92dnZOnHihDp37ixjjP773//+5f2//vprubi46JFHHrEZf+yxx2SM0fLly23Gu3btqqioqL+cFwAqE7KgYrLg3LlzGjJkiLy8vPTCCy+U+P6oGmgWUKmcP39ekydPtp6jWadOHQUFBenMmTPKyMgosn2DBg1sbteuXVuSdPr06Stud3HbS7f77bff1LRp0yLbNW/e3Lr+rxw+fFjDhw9XQECAfHx8FBQUpK5du0qS3fr/7LffflO9evVUq1atYtUQGRlpc/v48eNKTU21LllZWX+5TwBwNmRB+WdBQUGBhg4dqj179ujTTz9VvXr1/rIuVE18ZwGVSlxcnBISEjR27FhFR0fLz89PFotFQ4cOVWFhYZHtXVxc7M5j/vQFsOJudzUKCgrUu3dvnTp1Sk8++aSaNWsmb29vHTlyRMOHD7db/9W69NMrSerQoYNNiEyZMqXCf88BAK4WWVAypcmCkSNHatmyZfrwww/Vo0ePMq8JlQfNAiqVTz/9VLGxsfr3v/9tHcvJydGZM2fKfd/h4eFKSkoqMv7LL79Y10u67BWFdu3apX379mnBggW69957reOrVq0qsu3l5ggPD9e3336rs2fP2nyi9OcaLufDDz+0udpGw4YNr7g9ADgjsqB8s+CJJ55QQkKCZs+erbvvvvuKc6Hq4zQkVCouLi5FPuF57bXXVFBQUO777tevnzZv3qyNGzdax7KzszV37lxFRERYzwf19vaWpCKhdfETq0vrN8bolVdeKbKvy83Rr18/FRQU6PXXX7cZnzVrliwWi/r27XvFx9ClSxf16tXLutAsAKiMyILyy4IXX3xRL730kv75z3/q0UcfveI8qB44soBK5bbbbtP7778vPz8/RUVFaePGjfr2228VGBhY7vv+xz/+oY8++kh9+/bVI488ooCAAC1YsEDJyclasmSJatT4o/du1KiR/P399eabb6pWrVry9vZWp06d1KxZMzVq1EiPP/64jhw5Il9fXy1ZsqTIObOS1L59e0nSI488opiYGLm4uGjo0KHq37+/unfvrqeeekqHDh1S69at9c033+jzzz/X2LFj1ahRo1I/vi+//FI7d+6U9McXBn/66Sc9++yzkqTbb79drVq1KvXcAFCWyILyyYLPPvtMEyZMUJMmTdS8eXN98MEHNut79+6t4ODgUs2NSswRl2ACSuv06dPmvvvuM3Xq1DE+Pj4mJibG/PLLLyY8PNzmUp8XL5e3ZcsWm/uvXr3aSDKrV6+2joWHh5tbb721yL66du1qunbtajN28OBBM3jwYOPv7288PT1Nx44dzbJly4rc9/PPPzdRUVHG1dXV5tJ5e/bsMb169TI+Pj6mTp06ZuTIkWbnzp1FLq934cIFExcXZ4KCgozFYrG5dN7Zs2fNuHHjTL169Yybm5tp0qSJefHFF20uC2jMH5fLGz169F88o/8TGxtrJNld7F36DwAchSwonyyYMmXKZXPgz88Xqg+LMWX4rR0AAAAAVQbfWQAAAABgF80CAAAAALtoFgAAAADYRbMAAAAAwC6aBQAAAAB2VfnfWSgsLNTRo0dVq1aty/4SIgBUF8YYnT17VvXq1bNeD746IAsA4H9KlAWOvXKrMb///rsZNmyYCQgIMJ6enqZly5Y210MuLCw0kyZNMiEhIcbT09P07NnT7Nu3r9jzp6SkXPGawSwsLCzVcUlJSSmPf9JLjSxgYWFhqfilOFng0CMLp0+fVpcuXdS9e3ctX75cQUFB2r9/v2rXrm3dZubMmXr11Ve1YMECRUZGatKkSYqJidGePXvk6en5l/uoVauWJCklJUW+vr7l9lgAoDLIzMxUWFiY9d9GZ0AWAEDFKkkWOPRH2f7xj39o/fr1Wrdund31xhjVq1dPjz32mB5//HFJUkZGhoKDgzV//nwNHTr0L/eRmZkpPz8/ZWRkEBAAqj1n/DeRLACAilWSfxMdesLqF198oeuvv15DhgxR3bp11bZtW82bN8+6Pjk5WampqerVq5d1zM/PT506ddLGjRvtzpmbm6vMzEybBQDgvMgCAHBeDm0Wfv31V82ZM0dNmjTRypUr9dBDD+mRRx7RggULJEmpqamSpODgYJv7BQcHW9f92YwZM+Tn52ddwsLCyvdBAACuClkAAM7Loc1CYWGh2rVrp+eff15t27bVqFGjNHLkSL355pulnnPixInKyMiwLikpKWVYMQCgrJEFAOC8HPoF59DQUEVFRdmMNW/eXEuWLJEkhYSESJLS0tIUGhpq3SYtLU1t2rSxO6eHh4c8PDxKXEtBQYHy8/NLfD9cnpubm1xcXBxdBgAnRxZUbWQBULk5tFno0qWLkpKSbMb27dun8PBwSVJkZKRCQkKUmJhoDYTMzEz9+OOPeuihh8qkBmOMUlNTdebMmTKZD7b8/f0VEhLCdc0BXBZZUPWRBUDl5dBmYdy4cercubOef/553Xnnndq8ebPmzp2ruXPnSpIsFovGjh2rZ599Vk2aNLFeLq9evXoaOHBgmdRwMRzq1q2rmjVr8g9ZGTHG6Ny5c0pPT5ckm08DAeBSZEHVRRYAlZ9Dm4UOHTros88+08SJEzV9+nRFRkZq9uzZGjZsmHWbCRMmKDs7W6NGjdKZM2d04403asWKFcW6rvZfKSgosIZDYGDgVc8HW15eXpKk9PR01a1bl8PQAOwiC6o2sgCo3Bz6OwsV4UrXkc3JyVFycrIiIiKs/5ihbJ0/f16HDh1SZGRkmYQ6gKtTXX9vgCxwLLIAcC6V5ncWnAWHm8sPzy2AyoJ/r8oPzy1QeTn0NCT8z/HsPY4uoVzk5xbqbF66Pvn5ceVa7F8PvayMbLetXOcHgPJGFlw9sgAoWxxZAAAAAGAXzYITMMbosTFTdG1YtOr6tNCun/Zq4C3D9fSEGVe8X/uo3nor/r0KqhIAUJ7IAgDOiNOQ/kJqaqqee+45ffXVVzpy5Ijq1q2rNm3aaOzYserZs2eZ7GPFihVa9OFSfbZ8vsIj6yswsLYSFs6Wm5tbmcwPALg6ZAGA6opm4QoOHTqkLl26yN/fXy+++KKuu+465efna+XKlRo9erR++eWXMtnPwYMHFRwSpI43tLWO1Q7wL5O5gZKYt729o0uo9DhfuuohC1DdkAVXryplAachXcHDDz8si8WizZs3a9CgQbr22mvVokULjR8/Xps2bZIkHT58WAMGDJCPj498fX115513Ki0tzTrH1KlT1aZNG73//vuKiIiQn5+fhg4dqrNnz0qShg8frri4OP2eckx1fVqofVRvSSpy6Pl4+kn935CH1aBOO13foo8+/XhZkXozzmRq3OjJah5+oxqGdtQd/e7T7l3/C7GZz8Wre/Qd+uSjL9Q+qrca1eukUbGPK+tstnWbwsJCvTbrHXVsdYvqB7RR22Y9NWvmW9b1R34/pvvvGa/G19yga8Oide9dY3T4tyNl9IwDgPMhC8gCoDqjWbiMU6dOacWKFRo9erS8vb2LrPf391dhYaEGDBigU6dOae3atVq1apV+/fVX3XXXXTbbHjx4UEuXLtWyZcu0bNkyrV27Vi+88IIk6ZVXXtH06dNV75oQ7Tq4RivXfmy3nkcefEpHfk/Vf75O0DsfzFLCvI904vgpm21G3DNeJ46f1Eefvalv1y3WdW2iNPjWETp96ox1m0PJKVr+ZaI+WPyGPlz8hjb8sEWv/vtt6/pnp8zSay+/rfFPPqh1W7/QnHdnKqjuHz9SlJ+fr7sGjJJPLW99sfI9LVv1gWp619TQgQ8oLy+vVM8zADgzsoAsAKo7TkO6jAMHDsgYo2bNml12m8TERO3atUvJyckKCwuTJL333ntq0aKFtmzZog4dOkj64xOa+fPnq1atWpKke+65R4mJiXruuefk5+enWrVqycWlhoKDg+zu5+D+Q0r8Zp1Wrl2ktu2vkyTNjn9GXdr3t26zacM2/XfbLu1JXicPD3dJ0rTnn9DyZYn6cuk3uvfvd0qSTKHRa289L59af4TekLv7a93aTZIeVdbZbM174wPN+PdTGjpsoCQpsmED3dD5j8ORS5esUGGh0az46dZrZr/65rNqck201q/bou49u5T4eQYAZ0YWDJREFgDVGc3CZRTnh6337t2rsLAwazhIUlRUlPz9/bV3715rQERERFjDQZJCQ0OVnp5e7Fr2Jf0qV1dXtW7bwjrWpGlD+fn/7xf3ft6VpOysc2raoLPNfXPO5+pQcor1dliDetZwkKTgkCAd//8/ldqXdFC5uXm6qdsNduv4eVeSkn89rMiQDrb7yMnVoV9TpLL5jh8AOA2yoCiyAKheaBYuo0mTJrJYLGXyxbU/X8nCYrGosLDwque9VHb2OQWHBOmz5QlF1vn5/S9IXN1s/+QWi0Xm/6/F09PzyvvIOqfWbaP0xjv/KrKuTp2A0pQNAE6NLLCzD7IAqFb4zsJlBAQEKCYmRvHx8crOzi6y/syZM2revLlSUlKUkvK/T2v27NmjM2fOKCoqqsxqaXJtpC5cuKCd//3ZOnZgX7IyzmRab7dqE6X0tBNydXVVw0bhNktgndrF2k/DxuHy8vLUujWb7K5v1aa5fj34m4KCAovsw9evlt37AEBlRhYURRYA1QvNwhXEx8eroKBAHTt21JIlS7R//37t3btXr776qqKjo9WrVy9dd911GjZsmLZv367Nmzfr3nvvVdeuXXX99deXWR2Nr41Uj9436vFHpmnblp+0878/a9zoyfLy+t+nP127R+v6jq0VOzROqxPX6/BvR7R503/1/NRXtGP77mLtx9PTQ2PGjdD0Sf/Wxws/V/Kvh7V18059uGCJJGnQXbcpILC27r1rjDat36bfDv2u9d9v1j8ff15Hj6SW2eMFAGdCFpAFQHVGs3AFDRs21Pbt29W9e3c99thjatmypXr37q3ExETNmTNHFotFn3/+uWrXrq2bb75ZvXr1UsOGDfXxx/avYnE1XnnzWYWE1tXAW2J13/97VPf8fYjqBP3vcK/FYtFH/3lT0V2u16MPPq3oNv30wPDHlZJy1HoFi+J47B8P6qG44Zr57Ou6sX1/jYp9zHqljZo1vfT5ygW6JixU9/2/R3Vj+/4aO3qycnNyVauWT5k/ZgBwBmQBWQBUZxZTnG9vVWKZmZny8/NTRkaGfH19bdbl5OQoOTlZkZGRf3mOZnk7nr3HofsvL/m5hfo9JV2/XPi3ci3l+4lTVfoBFEfhh3iunrO/Dq/0b2JVRhY4FllQuZAFV8/ZX4clyQKOLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALDL1dEFVEUHg26qsH35HnqrwvYFACg+sgBAVcCRhWpo4w9b9X9DHtZ1jbuprk8Lff1loqNLAgBUMLIAQHHQLFRD586dV4uWTfXCy087uhQAgIOQBQCKg9OQqqGefW5Szz4Vd3gcAOB8yAIAxcGRBQAAAAB2ObRZmDp1qiwWi83SrFkz6/qcnByNHj1agYGB8vHx0aBBg5SWlubAigEAZY0sAADn5fAjCy1atNCxY8esyw8//GBdN27cOH355ZdavHix1q5dq6NHj+qOO+5wYLUAgPJAFgCAc3L4dxZcXV0VEhJSZDwjI0PvvPOOFi5cqB49ekiSEhIS1Lx5c23atEk33HCD3flyc3OVm5trvZ2ZmVk+hQMAygxZAADOyeFHFvbv36969eqpYcOGGjZsmA4fPixJ2rZtm/Lz89WrVy/rts2aNVODBg20cePGy843Y8YM+fn5WZewsLByfwwAgKtDFgCAc3Jos9CpUyfNnz9fK1as0Jw5c5ScnKybbrpJZ8+eVWpqqtzd3eXv729zn+DgYKWmpl52zokTJyojI8O6pKSklPOjqHyysrK166e92vXTXknS4d9+166f9ur3lKMOrgxAdUQWOAZZAKA4HHoaUt++fa3/3apVK3Xq1Enh4eH65JNP5OXlVao5PTw85OHhUVYllkqj4+tKfJ/j2XvKoRL7dm7/WX/rd5/19uR/zJQk3TVsgF576/kKqwMAJLLgUmQBAGfj8O8sXMrf31/XXnutDhw4oN69eysvL09nzpyx+UQpLS3N7nmtKL4uN3dUetbPji4DAOwiCyoGWQCgOBz+nYVLZWVl6eDBgwoNDVX79u3l5uamxMT//fx8UlKSDh8+rOjoaAdWCQAoT2QBADgPhx5ZePzxx9W/f3+Fh4fr6NGjmjJlilxcXHT33XfLz89PI0aM0Pjx4xUQECBfX1/FxcUpOjr6sle/AABUPmQBADgvhzYLv//+u+6++26dPHlSQUFBuvHGG7Vp0yYFBQVJkmbNmqUaNWpo0KBBys3NVUxMjN544w1HlgwAKGNkAQA4L4c2C4sWLbriek9PT8XHxys+Pr6CKgIAVDSyAACcl1N9ZwEAAACA86BZAAAAAGAXzQIAAAAAu2gWAAAAANhFswAAAADALqf6BecqY+rfSnyXoFLu6vgTz5XyngCAckUWAKgCOLJQzbzy0jz1uflORYZ0UFTETbp3aJwO7Et2dFkAgApEFgAoLpqFambDD1v091F3a/l3H+mTL+fpQv4F3TlgpLKzzzm6NABABSELABQXpyFVMx8vnWtz+9U3n1NU5E366b97FH3j9Q6qCgBQkcgCAMXFkYVqLjPzrCTJv7afgysBADgKWQDgcmgWqrHCwkJNevJf6hjdVs1bNHF0OQAAByALAFwJpyFVY0+Oe1a/7NmvL1e97+hSAAAOQhYAuBKahWrqH+Of1aoVa/X5ygWqd02Io8sBADgAWQB7Rn7RwNElVH7tHF1A2aFZqGaMMZr42HP6+stELV0+X+ER9R1dEgCggpEFAIqLZqGaeXLcM/rP4q/13qLX5F2rptLSjkuSfH1rycvL08HVAQAqAlkAoLhoFsrD1M9KfJfj2XvKoZCi5r/9sSRpYN/hNuOvvvmshv5fyX9tFABwGWQBgCqAZqGaSc/62dElAAAcjCwAUFxcOhUAAACAXTQLAAAAAOyiWQAAAABgF99ZAGDFtbXLQBW6tjYAABxZAAAAAGAXzQIAAAAAu2gWAAAAANhFswAAAADALpoFAAAAAHZxNaRyMG97+wrb18Cm71fYvgAAxUcWAKgKOLJQDSXMW6Sunf6mhqEd1TC0o/r2+H9K/Gado8sCAFQgsgBAcThNs/DCCy/IYrFo7Nix1rGcnByNHj1agYGB8vHx0aBBg5SWlua4IquIetcEa9L0cfp23WKt+v4T3XRzJ9171xj9sueAo0sDUM2RBRWHLABQHE7RLGzZskVvvfWWWrVqZTM+btw4ffnll1q8eLHWrl2ro0eP6o477nBQlVVHTL/u6hVzsxo2DlejJhH659RH5e1TU9u27HR0aQCqMbKgYpEFAIrD4c1CVlaWhg0bpnnz5ql27drW8YyMDL3zzjt6+eWX1aNHD7Vv314JCQnasGGDNm3adNn5cnNzlZmZabPg8goKCvTZ4q91Lvu8ru/Y2tHlAKimyALHIgsAXI7Dm4XRo0fr1ltvVa9evWzGt23bpvz8fJvxZs2aqUGDBtq4ceNl55sxY4b8/PysS1hYWLnVXpnt2b1PEcHXq35AWz0xdrrmf/SqmjZv7OiyAFRTZIFjkAUA/opDm4VFixZp+/btmjFjRpF1qampcnd3l7+/v814cHCwUlNTLzvnxIkTlZGRYV1SUlLKuuwqofG1EfpuwxKtWPORht9/l+JG/VNJezlPFUDFIwschywA8FccdunUlJQUPfroo1q1apU8PT3LbF4PDw95eHiU2XxVlbu7uxo2CpcktW7bQv/dtltz3/hA/35tqmMLA1CtkAWORRYA+CsOO7Kwbds2paenq127dnJ1dZWrq6vWrl2rV199Va6urgoODlZeXp7OnDljc7+0tDSFhIQ4pugqzBQWKi8vz9FlAKhmyALnQhYA+DOHHVno2bOndu3aZTN23333qVmzZnryyScVFhYmNzc3JSYmatCgQZKkpKQkHT58WNHR0Y4oucp4dsos9ex9k64JC1XW2Wz9Z/FXWr9uiz7+fK6jSwNQzZAFjkMWACgOhzULtWrVUsuWLW3GvL29FRgYaB0fMWKExo8fr4CAAPn6+iouLk7R0dG64YYbHFFysY1st63E9zmevaccKrHvxPFTGjNqotJSj8vXt5aat7xWH38+V916dK6wGgBAIgv+jCwA4Gwc1iwUx6xZs1SjRg0NGjRIubm5iomJ0RtvvOHosiq92W884+gSAKDYyILyQRYAKA6nahbWrFljc9vT01Px8fGKj493TEEAgApHFgCA83D47ywAAAAAcE40CwAAAADsolmQZIxxdAlV1h9PrZFU6OBKAODKyILyQxYAlVe1bhbc3NwkSefOnXNwJVVXXm6hCgsvKF8Zji4FAOwiC8ofWQBUXk71BeeK5uLiIn9/f6Wnp0uSatasKYvF4pBa8nOr1qctxvwRDidPnNHxwh9UaMl1dEkAYBdZUH7IAqDyq9bNgiTrL4BeDAlHOZvn2P2XPaPCwgs6XviDUrXc0cUAwBWRBeWFLAAqu2rfLFgsFoWGhqpu3brKz893WB2f/Py4w/ZdPgqVrww+RQJQKZAF5YUsACq7at8sXOTi4iIXFxeH7T/XkuqwfQMA/kAWAICtav0FZwAAAACXR7MAAAAAwC6aBQAAAAB20SwAAAAAsItmAQAAAIBdpWoWGjZsqJMnTxYZP3PmjBo2bHjVRQEAnB9ZAABVX6mahUOHDqmgoKDIeG5uro4cOXLVRQEAnB9ZAABVX4l+Z+GLL76w/vfKlSvl5+dnvV1QUKDExERFRESUWXEAAOdDFgBA9VGiZmHgwIGS/vily9jYWJt1bm5uioiI0L///e8yKw4A4HzIAgCoPkrULBQWFkqSIiMjtWXLFtWpU6dcigIAOC+yAACqjxI1CxclJyeXdR0AgEqGLACAqq9UzYIkJSYmKjExUenp6dZPmS569913r7owAIDzIwsAoGorVbMwbdo0TZ8+Xddff71CQ0NlsVjKuq5qZ+QXDRxdQuXXztEFANULWQAAVV+pmoU333xT8+fP1z333FPW9QAAKgmyAACqvlL9zkJeXp46d+5c1rUAACoRsgAAqr5SNQv333+/Fi5cWNa1AAAqEbIAAKq+Up2GlJOTo7lz5+rbb79Vq1at5ObmZrP+5ZdfLpPiAADOiywAgKqvVM3CTz/9pDZt2kiSdu/ebbOOL7gBQPVAFgBA1VeqZmH16tVlXQcAoJIhCwCg6ivVdxYAAAAAVH2lOrLQvXv3Kx5i/u6774o1z5w5czRnzhwdOnRIktSiRQtNnjxZffv2lfTH+bCPPfaYFi1apNzcXMXExOiNN95QcHBwacoGAJQhsgAAqr5SHVlo06aNWrdubV2ioqKUl5en7du367rrriv2PPXr19cLL7ygbdu2aevWrerRo4cGDBign3/+WZI0btw4ffnll1q8eLHWrl2ro0eP6o477ihNyQCAMkYWAEDVV6ojC7NmzbI7PnXqVGVlZRV7nv79+9vcfu655zRnzhxt2rRJ9evX1zvvvKOFCxeqR48ekqSEhAQ1b95cmzZt0g033FCa0gEAZYQsAICqr0y/s/B///d/evfdd0t134KCAi1atEjZ2dmKjo7Wtm3blJ+fr169elm3adasmRo0aKCNGzdedp7c3FxlZmbaLACAikMWAEDVUabNwsaNG+Xp6Vmi++zatUs+Pj7y8PDQgw8+qM8++0xRUVFKTU2Vu7u7/P39bbYPDg5WamrqZeebMWOG/Pz8rEtYWFhpHgoAoJTIAgCoOkp1GtKfzxU1xujYsWPaunWrJk2aVKK5mjZtqh07digjI0OffvqpYmNjtXbt2tKUJUmaOHGixo8fb72dmZlJSABAOSALAKDqK1Wz4OfnZ3O7Ro0aatq0qaZPn64+ffqUaC53d3c1btxYktS+fXtt2bJFr7zyiu666y7l5eXpzJkzNp8opaWlKSQk5LLzeXh4yMPDo0Q1AABKjiwAgKqvVM1CQkJCWddhVVhYqNzcXLVv315ubm5KTEzUoEGDJElJSUk6fPiwoqOjy23/AIDiIQsAoOorVbNw0bZt27R3715Jf1wXu23btiW6/8SJE9W3b181aNBAZ8+e1cKFC7VmzRqtXLlSfn5+GjFihMaPH6+AgAD5+voqLi5O0dHRXP0CAJwIWQAAVVepmoX09HQNHTpUa9assR4WPnPmjLp3765FixYpKCio2PPce++9OnbsmPz8/NSqVSutXLlSvXv3lvTHZflq1KihQYMG2fwQDwDA8cgCAKj6StUsxMXF6ezZs/r555/VvHlzSdKePXsUGxurRx55RB999FGx5nnnnXeuuN7T01Px8fGKj48vTZkAgHJEFgBA1VeqZmHFihX69ttvreEgSVFRUYqPjy/xl9oAAJUTWQAAVV+pfmehsLBQbm5uRcbd3NxUWFh41UUBAJwfWQAAVV+pmoUePXro0Ucf1dGjR61jR44c0bhx49SzZ88yKw4A4LzIAgCo+krVLLz++uvKzMxURESEGjVqpEaNGikyMlKZmZl67bXXyrpGAIATIgsAoOor1XcWwsLCtH37dn377bf65ZdfJEnNmzdXr169yrQ4AIDzIgsAoOor0ZGF7777TlFRUcrMzJTFYlHv3r0VFxenuLg4dejQQS1atNC6devKq1YAgBMgCwCg+ihRszB79myNHDlSvr6+Rdb5+fnpgQce0Msvv1xmxQEAnA9ZAADVR4mahZ07d+qWW2657Po+ffpo27ZtV10UAMB5kQUAUH2UqFlIS0uze5m8i1xdXXX8+PGrLgoA4LzIAgCoPkrULFxzzTXavXv3Zdf/9NNPCg0NveqiAADOiywAgOqjRM1Cv379NGnSJOXk5BRZd/78eU2ZMkW33XZbmRUHAHA+ZAEAVB8lunTq008/rf/85z+69tprNWbMGDVt2lSS9Msvvyg+Pl4FBQV66qmnyqVQAIBzIAsAoPooUbMQHBysDRs26KGHHtLEiRNljJEkWSwWxcTEKD4+XsHBweVSKADAOZAFAFB9lPhH2cLDw/X111/r9OnTOnDggIwxatKkiWrXrl0e9QEAnBBZAADVQ6l+wVmSateurQ4dOpRlLQCASoYsAICqrURfcAYAAABQfdAsAAAAALCLZgEAAACAXTQLAAAAAOyiWQAAAABgV6mvhgQAAMrWyC8aOLqEyq+dowsAqhaOLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdjm0WZgxY4Y6dOigWrVqqW7duho4cKCSkpJstsnJydHo0aMVGBgoHx8fDRo0SGlpaQ6qGABQ1sgCAHBeDm0W1q5dq9GjR2vTpk1atWqV8vPz1adPH2VnZ1u3GTdunL788kstXrxYa9eu1dGjR3XHHXc4sGoAQFkiCwDAebk6cucrVqywuT1//nzVrVtX27Zt080336yMjAy98847WrhwoXr06CFJSkhIUPPmzbVp0ybdcMMNRebMzc1Vbm6u9XZmZmb5PggAwFUhCwDAeTm0WfizjIwMSVJAQIAkadu2bcrPz1evXr2s2zRr1kwNGjTQxo0b7QbEjBkzNG3atIopGABQ5sgCwLEOxp9wdAmVXqOpjq6g7DjNF5wLCws1duxYdenSRS1btpQkpaamyt3dXf7+/jbbBgcHKzU11e48EydOVEZGhnVJSUkp79IBAGWELAAA5+I0RxZGjx6t3bt364cffriqeTw8POTh4VFGVQEAKhJZAADOxSmOLIwZM0bLli3T6tWrVb9+fet4SEiI8vLydObMGZvt09LSFBISUsFVAgDKE1kAAM7Hoc2CMUZjxozRZ599pu+++06RkZE269u3by83NzclJiZax5KSknT48GFFR0dXdLkAgHJAFgCA83LoaUijR4/WwoUL9fnnn6tWrVrWc0/9/Pzk5eUlPz8/jRgxQuPHj1dAQIB8fX0VFxen6Ohou19oAwBUPmQBADgvhzYLc+bMkSR169bNZjwhIUHDhw+XJM2aNUs1atTQoEGDlJubq5iYGL3xxhsVXCkAoLyQBQDgvBzaLBhj/nIbT09PxcfHKz4+vgIqAgBUNLIAAJyXU3zBGQAAAIDzoVkAAAAAYBfNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALCLZgEAAACAXTQLAAAAAOyiWQAAAABgF80CAAAAALtoFgAAAADYRbMAAAAAwC6aBQAAAAB20SwAAAAAsItmAQAAAIBdNAsAAAAA7KJZAAAAAGAXzQIAAAAAu2gWAAAAANhFswAAAADALpoFAAAAAHbRLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2OXQZuH7779X//79Va9ePVksFi1dutRmvTFGkydPVmhoqLy8vNSrVy/t37/fMcUCAMoFWQAAzsuhzUJ2drZat26t+Ph4u+tnzpypV199VW+++aZ+/PFHeXt7KyYmRjk5ORVcKQCgvJAFAOC8XB258759+6pv37521xljNHv2bD399NMaMGCAJOm9995TcHCwli5dqqFDh9q9X25urnJzc623MzMzy75wAECZIQsAwHk57XcWkpOTlZqaql69elnH/Pz81KlTJ23cuPGy95sxY4b8/PysS1hYWEWUCwAoB2QBADiW0zYLqampkqTg4GCb8eDgYOs6eyZOnKiMjAzrkpKSUq51AgDKD1kAAI7l0NOQyoOHh4c8PDwcXQYAwIHIAgAoG057ZCEkJESSlJaWZjOelpZmXQcAqNrIAgBwLKdtFiIjIxUSEqLExETrWGZmpn788UdFR0c7sDIAQEUhCwDAsRx6GlJWVpYOHDhgvZ2cnKwdO3YoICBADRo00NixY/Xss8+qSZMmioyM1KRJk1SvXj0NHDjQcUUDAMoUWQAAzsuhzcLWrVvVvXt36+3x48dLkmJjYzV//nxNmDBB2dnZGjVqlM6cOaMbb7xRK1askKenp6NKBgCUMbIAAJyXQ5uFbt26yRhz2fUWi0XTp0/X9OnTK7AqAEBFIgsAwHk57XcWAAAAADgWzQIAAAAAu2gWAAAAANhFswAAAADALpoFAAAAAHbRLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALCLZgEAAACAXTQLAAAAAOxydXQBAADgDwfjTzi6hEqv0VRHVwBULRxZAAAAAGAXzQIAAAAAu2gWAAAAANjFdxacBOepXj3OU716vA6vHq9DAEBVwpEFAAAAAHbRLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2FUpmoX4+HhFRETI09NTnTp10ubNmx1dEgCggpEFAFDxnL5Z+PjjjzV+/HhNmTJF27dvV+vWrRUTE6P09HRHlwYAqCBkAQA4htM3Cy+//LJGjhyp++67T1FRUXrzzTdVs2ZNvfvuu44uDQBQQcgCAHAMV0cXcCV5eXnatm2bJk6caB2rUaOGevXqpY0bN9q9T25urnJzc623MzIyJEmZmZnlW+xVOlt4wdElVHrO/jeuDHgdXj1nfx1erM8Y4+BKio8sQEk4+9+4MuB1ePWc/XVYkixw6mbhxIkTKigoUHBwsM14cHCwfvnlF7v3mTFjhqZNm1ZkPCwsrFxqhBPx83N0BUCleR2ePXtWfpWkVrIAJVJJXteo4irJ67A4WeDUzUJpTJw4UePHj7feLiws1KlTpxQYGCiLxeLAyiqvzMxMhYWFKSUlRb6+vo4uB9UYr8WrZ4zR2bNnVa9ePUeXUq7IgrLH+w/Ogtfi1StJFjh1s1CnTh25uLgoLS3NZjwtLU0hISF27+Ph4SEPDw+bMX9///IqsVrx9fXlTQmnwGvx6lSWIwoXkQXOhfcfnAWvxatT3Cxw6i84u7u7q3379kpMTLSOFRYWKjExUdHR0Q6sDABQUcgCAHAcpz6yIEnjx49XbGysrr/+enXs2FGzZ89Wdna27rvvPkeXBgCoIGQBADiG0zcLd911l44fP67JkycrNTVVbdq00YoVK4p80Q3lx8PDQ1OmTClySB+oaLwWqy+ywPF4/8FZ8FqsWBZTma6fBwAAAKDCOPV3FgAAAAA4Ds0CAAAAALtoFgAAAADYRbOAK4qIiNDs2bPLZW6LxaKlS5c6fA78tfnz51f6a9R369ZNY8eOdfgcQGVEFkAiC8pyjkrFoEzExsYaSUaScXV1NREREeaJJ54w58+fd3RpxZKQkGD8/PyKjKenp5vs7OxSzxseHm59Xi4u11xzjTHGmGPHjpmcnJxizTNlyhTTunXrIuMlmaM6S09PNw8++KAJCwsz7u7uJjg42PTp08f88MMPxbr/uXPnTFpaWjlXefUufR9euuzfv9+cPHnSZGZmFmue1atXG0nm9OnTNuMlmQPVE1lgH1ngHMgCsqA0nP7SqZXJLbfcooSEBOXn52vbtm2KjY2VxWLRv/71L0eXVmpBQUFXPcf06dM1cuRI620XFxdJuuwvr5ZEWcxRHQwaNEh5eXlasGCBGjZsqLS0NCUmJurkyZPFur+Xl5e8vLzKucqycfF9eKmgoCDr6+5qBAQEXPUcqPrIAvvIAscjC8iCUnF0t1JVxMbGmgEDBtiM3XHHHaZt27bW2zk5OSYuLs4EBQUZDw8P06VLF7N582br+osd7IoVK0ybNm2Mp6en6d69u0lLSzNff/21adasmalVq5a5++67bT7hWb58uenSpYvx8/MzAQEB5tZbbzUHDhywrk9OTjaSzJIlS0y3bt2Ml5eXadWqldmwYYPNfi9dpkyZYoz549OgWbNmWec6ffq0GTVqlKlbt67x8PAwLVq0MF9++eVln5c/3/9Sksxnn31mvZ2SkmKGDh1qateubWrWrGnat29vNm3aZBISEorUl5CQYHeOn376yXTv3t14enqagIAAM3LkSHP27Nkif6cXX3zRhISEmICAAPPwww+bvLy8yz6Gyu706dNGklmzZs1fbne5v629TxuXLl1q2rZtazw8PExkZKSZOnWqyc/Pt66XZObNm2cGDhxovLy8TOPGjc3nn39uM8fu3bvNrbfeamrVqmV8fHzMjTfeaPPanTdvnmnWrJnx8PAwTZs2NfHx8Vd8DPbehxd17drVPProo9bbOTk5ZsKECaZ+/frG3d3dNGrUyLz99tvW98ulS2xsrN05Tp06Ze655x7j7+9vvLy8zC233GL27dtnXX/xeVuxYoVp1qyZ8fb2NjExMebo0aNXfByovMgC+8gCxyML/kAWlBzfWSgnu3fv1oYNG+Tu7m4dmzBhgpYsWaIFCxZo+/btaty4sWJiYnTq1Cmb+06dOlWvv/66NmzYoJSUFN15552aPXu2Fi5cqK+++krffPONXnvtNev22dnZGj9+vLZu3arExETVqFFDf/vb31RYWGgz71NPPaXHH39cO3bs0LXXXqu7775bFy5cUOfOnTV79mz5+vrq2LFjOnbsmB5//PEij6mwsFB9+/bV+vXr9cEHH2jPnj164YUXyqRLz8rKUteuXXXkyBF98cUX2rlzpyZMmKDCwkLdddddeuyxx9SiRQtrfXfddVeRObKzsxUTE6PatWtry5YtWrx4sb799luNGTPGZrvVq1fr4MGDWr16tRYsWKD58+dr/vz5V/0YnJWPj498fHy0dOlS5ebm2t2mpH/bdevW6d5779Wjjz6qPXv26K233tL8+fP13HPP2Ww3bdo03Xnnnfrpp5/Ur18/DRs2zPp6P3LkiG6++WZ5eHjou+++07Zt2/T3v/9dFy5ckCR9+OGHmjx5sp577jnt3btXzz//vCZNmqQFCxaUyfNy77336qOPPtKrr76qvXv36q233pKPj4/CwsK0ZMkSSVJSUpKOHTumV155xe4cw4cP19atW/XFF19o48aNMsaoX79+ys/Pt25z7tw5vfTSS3r//ff1/fff6/Dhw3bfX6iayIKSIQvKD1lgH1lQDI7uVqqK2NhY4+LiYry9vY2Hh4eRZGrUqGE+/fRTY4wxWVlZxs3NzXz44YfW++Tl5Zl69eqZmTNnGmP+96nOt99+a91mxowZRpI5ePCgdeyBBx4wMTExl63l+PHjRpLZtWuXMeZ/nya9/fbb1m1+/vlnI8ns3bvXGHP581Qv/TRo5cqVpkaNGiYpKanYz0t4eLhxd3c33t7e1uWVV14xxth+EvTWW2+ZWrVqmZMnT9qd53LnqV46x9y5c03t2rVNVlaWdf1XX31latSoYVJTU40xf/ydwsPDzYULF6zbDBkyxNx1113FfkyV0aeffmpq165tPD09TefOnc3EiRPNzp07rev/6m/759dHz549zfPPP2+zzfvvv29CQ0OttyWZp59+2no7KyvLSDLLly83xhgzceJEExkZedlP8ho1amQWLlxoM/bMM8+Y6Ojoyz7OS9+HF5fBgwcbY2w/CUpKSjKSzKpVq+zOc7nzVC+dY9++fUaSWb9+vXX9iRMnjJeXl/nkk0+MMcb6Seiln5DFx8eb4ODgyz4GVG5kgX1kgXMgC8iC0uA7C2Woe/fumjNnjrKzszVr1iy5urpq0KBBkqSDBw8qPz9fXbp0sW7v5uamjh07au/evTbztGrVyvrfwcHBqlmzpho2bGgztnnzZuvt/fv3a/Lkyfrxxx914sQJ66dIhw8fVsuWLe3OGxoaKklKT09Xs2bNivX4duzYofr16+vaa68t1vYXPfHEExo+fLj1dp06dezO3bZt26s6D3Dv3r1q3bq1vL29rWNdunRRYWGhkpKSFBwcLElq0aKFzackoaGh2rVrV6n3WxkMGjRIt956q9atW6dNmzZp+fLlmjlzpt5++20NHz68xH/bnTt3av369TafHhUUFCgnJ0fnzp1TzZo1Jdm+5ry9veXr66v09HRJf/zNb7rpJrm5uRWZPzs7WwcPHtSIESNsznG+cOGC/Pz8rljbxffhpfv9sx07dsjFxUVdu3Yt1uO1Z+/evXJ1dVWnTp2sY4GBgWratKnNe7pmzZpq1KiR9XZoaKj1OUDVRBbYRxY4HllgiywoHpqFMuTt7a3GjRtLkt599121bt1a77zzjkaMGFGieS59w1gsliJvIIvFYnNYuX///goPD9e8efNUr149FRYWqmXLlsrLy7vivJKKHJ6+ktJ+qalOnTrW56Ws5y6Nv3o+qypPT0/17t1bvXv31qRJk3T//fdrypQpGj58eImf/6ysLE2bNk133HGH3f1cdKXn+kr7zMrKkiTNmzfP5h9gSX95qsOl78PLcfTrzRhTYftHxSML7CMLnANZ8D+Ofr1VlizgOwvlpEaNGvrnP/+pp59+WufPn1ejRo3k7u6u9evXW7fJz8/Xli1bFBUVVer9nDx5UklJSXr66afVs2dPNW/eXKdPny7xPO7u7iooKLjiNq1atdLvv/+uffv2lbbcK869Y8eOIufslqS+5s2ba+fOncrOzraOrV+/XjVq1FDTpk3LtN6qICoqyvpclfRv265dOyUlJalx48ZFlho1ivfPSqtWrbRu3TqbczovCg4OVr169fTrr78WmT8yMrL4D/IyrrvuOhUWFmrt2rV21188v/xKr7nmzZvrwoUL+vHHH61jF9+PV/OeRtVCFpQMWVDxyAKy4K/QLJSjIUOGyMXFRfHx8fL29tZDDz2kJ554QitWrNCePXs0cuRInTt3rsSfNl2qdu3aCgwM1Ny5c3XgwAF99913Gj9+fInniYiIUFZWlhITE3XixAmdO3euyDZdu3bVzTffrEGDBmnVqlVKTk7W8uXLtWLFilLXf9Hdd9+tkJAQDRw4UOvXr9evv/6qJUuWaOPGjdb6kpOTtWPHDp04ccLul7OGDRsmT09PxcbGavfu3Vq9erXi4uJ0zz33WA87V0cnT55Ujx499MEHH+inn35ScnKyFi9erJkzZ2rAgAGSSv63nTx5st577z1NmzZNP//8s/bu3atFixbp6aefLnZdY8aMUWZmpoYOHaqtW7dq//79ev/995WUlCTpjy/EzZgxQ6+++qr27dunXbt2KSEhQS+//PJVPycRERGKjY3V3//+dy1dulTJyclas2aNPvnkE0lSeHi4LBaLli1bpuPHj1s/3bpUkyZNNGDAAI0cOVI//PCDdu7cqf/7v//TNddcY31eAYksKAmyoPyQBUWRBcVDs1COXF1dNWbMGM2cOVPZ2dl64YUXNGjQIN1zzz1q166dDhw4oJUrV6p27dql3keNGjW0aNEibdu2TS1bttS4ceP04osvlniezp0768EHH9Rdd92loKAgzZw50+52S5YsUYcOHXT33XcrKipKEyZM+MtPeYrD3d1d33zzjerWrat+/frpuuuus7kCw6BBg3TLLbeoe/fuCgoK0kcffVRkjpo1a2rlypU6deqUOnTooMGDB6tnz556/fXXr7q+yszHx0edOnXSrFmzdPPNN6tly5aaNGmSRo4cafPclORvGxMTo2XLlumbb75Rhw4ddMMNN2jWrFkKDw8vdl2BgYH67rvvrFc/ad++vebNm2c9VHv//ffr7bffVkJCgq677jp17dpV8+fPL5NPkyRpzpw5Gjx4sB5++GE1a9ZMI0eOtH66ds0112jatGn6xz/+oeDg4CJXUbkoISFB7du312233abo6GgZY/T111/bPfcW1RdZUHxkQfkhC+wjC/6axVSWE6YAAAAAVCiOLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALCLZgEAAACAXTQLAAAAAOyiWQAAAABgF80CAAAAALtoFgAAAADYRbMAAAAAwC6aBQAAAAB20SwAAAAAsItmAfgLqampevTRR9W4cWN5enoqODhYXbp00Zw5c3Tu3DlHlwcAqABkAaorV0cXADizX3/9VV26dJG/v7+ef/55XXfddfLw8NCuXbs0d+5cXXPNNbr99tvLZd95eXlyd3cvl7kBAMVHFqA648gCcAUPP/ywXF1dtXXrVt15551q3ry5GjZsqAEDBuirr75S//79JUlnzpzR/fffr6CgIPn6+qpHjx7auXOndZ6pU6eqTZs2ev/99xURESE/Pz8NHTpUZ8+etW7TrVs3jRkzRmPHjlWdOnUUExMjSdq9e7f69u0rHx8fBQcH65577tGJEycq9okAgGqMLEB1RrMAXMbJkyf1zTffaPTo0fL29ra7jcVikSQNGTJE6enpWr58ubZt26Z27dqpZ8+eOnXqlHXbgwcPaunSpVq2bJmWLVumtWvX6oUXXrCZb8GCBXJ3d9f69ev15ptv6syZM+rRo4fatm2rrVu3asWKFUpLS9Odd95Zfg8cAGBFFqDaMwDs2rRpk5Fk/vOf/9iMBwYGGm9vb+Pt7W0mTJhg1q1bZ3x9fU1OTo7Ndo0aNTJvvfWWMcaYKVOmmJo1a5rMzEzr+ieeeMJ06tTJertr166mbdu2NnM888wzpk+fPjZjKSkpRpJJSkoqk8cJALg8sgDVHd9ZAEpo8+bNKiws1LBhw5Sbm6udO3cqKytLgYGBNtudP39eBw8etN6OiIhQrVq1rLdDQ0OVnp5uc5/27dvb3N65c6dWr14tHx+fInUcPHhQ1157bVk8JABACZEFqC5oFoDLaNy4sSwWi5KSkmzGGzZsKEny8vKSJGVlZSk0NFRr1qwpMoe/v7/1v93c3GzWWSwWFRYW2oz9+RB3VlaW+vfvr3/9619F5g4NDS32YwEAlA5ZgOqOZgG4jMDAQPXu3Vuvv/664uLiLnuuart27ZSamipXV1dFRESUaQ3t2rXTkiVLFBERIVdX3q4AUNHIAlR3fMEZuII33nhDFy5c0PXXX6+PP/5Ye/fuVVJSkj744AP98ssvcnFxUa9evRQdHa2BAwfqm2++0aFDh7RhwwY99dRT2rp161Xtf/To0Tp16pTuvvtubdmyRQcPHtTKlSt13333qaCgoIweJQDgSsgCVGe0p8AVNGrUSP/973/1/PPPa+LEifr999/l4eGhqKgoPf7443r44YdlsVj09ddf66mnntJ9992n48ePKyQkRDfffLOCg4Ovav/16tXT+vXr9eSTT6pPnz7Kzc1VeHi4brnlFtWoQa8PABWBLEB1ZjHGGEcXAQAAAMD50I4CAAAAsItmAQAAAIBdNAsAAAAA7KJZAAAAAGAXzQIAAAAAu2gWAAAAANhFswAAAADALpoFAAAAAHbRLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALCLZgEAAACAXTQLQBk7evSopk6dqh07dpR6jq+//lpTp04ts5r+yvnz5zVixAi1bNlSfn5+8vHxUevWrfXKK68oPz+/wuoAgKqiMmbBn/3www+yWCyyWCw6ceKEw+qAY9EsAGXs6NGjmjZt2lUHxLRp08quqL9w/vx5/fzzz+rXr59mzJihl156Sa1bt9a4ceMUGxtbYXUAQFVRGbPgUoWFhYqLi5O3t7dD9g/n4eroAgBUjMLCQuXl5cnT07PIuoCAAG3atMlm7MEHH5Sfn59ef/11vfzyywoJCamoUgEA5eRKWXCpuXPnKiUlRffff79eeeWVCqoOzogjC6hUfvvtNz388MNq2rSpvLy8FBgYqCFDhujQoUM2282fP18Wi0Xr16/X+PHjFRQUJG9vb/3tb3/T8ePHbbaNiIjQbbfdph9++EEdO3aUp6enGjZsqPfee6/I/n/99VcNGTJEAQEBqlmzpm644QZ99dVX1vVr1qxRhw4dJEn33Xef9fDt/PnzJUnr1q3TkCFD1KBBA3l4eCgsLEzjxo3T+fPnrXMMHz5c8fHxkmS9v8Visa7Pzs7WY489prCwMHl4eKhp06Z66aWXZIyxqdVisWjMmDH68MMP1aJFC3l4eGjFihUler4jIiIkSWfOnCnR/QCgPJEF5ZsFp06d0tNPP63p06fL39//itui6uPIAiqVLVu2aMOGDRo6dKjq16+vQ4cOac6cOerWrZv27NmjmjVr2mwfFxen2rVra8qUKTp06JBmz56tMWPG6OOPP7bZ7sCBAxo8eLBGjBih2NhYvfvuuxo+fLjat2+vFi1aSJLS0tLUuXNnnTt3To888ogCAwO1YMEC3X777fr000/1t7/9Tc2bN9f06dM1efJkjRo1SjfddJMkqXPnzpKkxYsX69y5c3rooYcUGBiozZs367XXXtPvv/+uxYsXS5IeeOABHT16VKtWrdL7779vU6cxRrfffrtWr16tESNGqE2bNlq5cqWeeOIJHTlyRLNmzbLZ/rvvvtMnn3yiMWPGqE6dOtb/+b+cvLw8ZWZm6vz589q6dateeuklhYeHq3HjxiX7QwFAOSILyjcLJk2apJCQED3wwAN65plnSvbHQdVjgErk3LlzRcY2btxoJJn33nvPOpaQkGAkmV69epnCwkLr+Lhx44yLi4s5c+aMdSw8PNxIMt9//711LD093Xh4eJjHHnvMOjZ27Fgjyaxbt846dvbsWRMZGWkiIiJMQUGBMcaYLVu2GEkmISGhWPXPmDHDWCwW89tvv1nHRo8ebey9PZcuXWokmWeffdZmfPDgwcZisZgDBw5YxySZGjVqmJ9//rnIPJfz0UcfGUnW5frrrzc//fRTse8PABWBLCi/LNi5c6dxcXExK1euNMYYM2XKFCPJHD9+vFj3R9XDaUioVLy8vKz/nZ+fr5MnT6px48by9/fX9u3bi2w/atQom8O2N910kwoKCvTbb7/ZbBcVFWX95EeSgoKC1LRpU/3666/Wsa+//lodO3bUjTfeaB3z8fHRqFGjdOjQIe3Zs6dE9WdnZ+vEiRPq3LmzjDH673//+5f3//rrr+Xi4qJHHnnEZvyxxx6TMUbLly+3Ge/atauioqL+ct6LunfvrlWrVmnx4sV68MEH5ebmpuzs7GLfHwAqAllQflnwyCOPqG/fvurTp0+xtkfVx2lIqFTOnz+vGTNmKCEhQUeOHLE5NzMjI6PI9g0aNLC5Xbt2bUnS6dOnr7jdxW0v3e63335Tp06dimzXvHlz6/qWLVtesf7Dhw9r8uTJ+uKLL4rUYK/+P/vtt99Ur1491apV67I1XCoyMtLm9vHjx1VQUGC97ePjIx8fH+vt4OBgBQcHS5IGDx6s559/Xr1799b+/fv5gjMAp0EWlE8WfPzxx9qwYYN27979lzWg+qBZQKUSFxenhIQEjR07VtHR0fLz85PFYtHQoUNVWFhYZHsXFxe785g/fQGsuNtdjYKCAvXu3VunTp3Sk08+qWbNmsnb21tHjhzR8OHD7dZ/tS799EqSOnToYBMiU6ZMueI1vAcPHqynnnpKn3/+uR544IEyrw8ASoMsKJniZsETTzyhIUOGyN3d3fpl8YsXuEhJSVFeXp7q1atX5vXBudEsoFL59NNPFRsbq3//+9/WsZycnAq5Wk94eLiSkpKKjP/yyy/W9ZJsDnVfateuXdq3b58WLFige++91zq+atWqIttebo7w8HB9++23Onv2rM0nSn+u4XI+/PBDm6ttNGzY8IrbX9y2OJ90AUBFIQvKJwtSUlK0cOFCLVy4sMh92rVrp9atW1/V70agcqJZQKXi4uJS5BOe1157zeZwannp16+fZs+erY0bNyo6OlrSH+eazp07VxEREdbzQS/+gM2fQ+viJ1aX1m+MsXv96kvnuPSydf369dPcuXP1+uuva+LEidbxWbNmyWKxqG/fvld8DF26dLE7fuLECQUGBhYJprfffluSdP31119xXgCoSGRB+WTBZ599VmRs0aJF+vjjj/Xee++pfv36V5wXVRPNAiqV2267Te+//778/PwUFRWljRs36ttvv1VgYGC57/sf//iHPvroI/Xt21ePPPKIAgICtGDBAiUnJ2vJkiWqUeOP6wU0atRI/v7+evPNN1WrVi15e3urU6dOatasmRo1aqTHH39cR44cka+vr5YsWVLkfFVJat++vaQ/vmgWExMjFxcXDR06VP3791f37t311FNP6dChQ2rdurW++eYbff755xo7dqwaNWpUqsf2wQcf6M0339TAgQPVsGFDnT17VitXrtSqVavUv39/9ejRo/RPHACUMbKgfLJg4MCBRcYuHkno27ev6tSpU6p5UclV/AWYgNI7ffq0ue+++0ydOnWMj4+PiYmJMb/88osJDw83sbGx1u0uXi5vy5YtNvdfvXq1kWRWr15tHQsPDze33nprkX117drVdO3a1Wbs4MGDZvDgwcbf3994enqajh07mmXLlhW57+eff26ioqKMq6urzaXz9uzZY3r16mV8fHxMnTp1zMiRI83OnTuLXF7vwoULJi4uzgQFBRmLxWJz6byzZ8+acePGmXr16hk3NzfTpEkT8+KLL9pcFtCYPy6XN3r06L94Rv+wZcsWM2TIENOgQQPj4eFhvL29Tbt27czLL79s8vPzizUHAFQUsqB8ssAeLp0KizFl+K0dAAAAAFUGv7MAAAAAwC6aBQAAAAB20SwAAAAAsItmAQAAAIBdNAsAAAAA7KJZAAAAAGBXlf9RtsLCQh09elS1atW67M+mA0B1YYzR2bNnVa9ePeuPR1UHZAEA/E+JssCxP/NgzO+//26GDRtmAgICjKenp2nZsqXNj6cUFhaaSZMmmZCQEOPp6Wl69uxp9u3bV+z5U1JSjCQWFhYWlkuWlJSU8vgnvdTIAhYWFpaKX4qTBQ49snD69Gl16dJF3bt31/LlyxUUFKT9+/erdu3a1m1mzpypV199VQsWLFBkZKQmTZqkmJgY7dmzR56enn+5j1q1akmSUlJS5OvrW26PBQAqg8zMTIWFhVn/bXQGZAEAVKySZIFDf8H5H//4h9avX69169bZXW+MUb169fTYY4/p8ccflyRlZGQoODhY8+fP19ChQ4vcJzc3V7m5udbbF5+MjIwMAgJAtZeZmSk/Pz+n+jeRLACAilWSLHDoCatffPGFrr/+eg0ZMkR169ZV27ZtNW/ePOv65ORkpaamqlevXtYxPz8/derUSRs3brQ754wZM+Tn52ddwsLCyv1xAABKjywAAOfl0Gbh119/1Zw5c9SkSROtXLlSDz30kB555BEtWLBAkpSamipJCg4OtrlfcHCwdd2fTZw4URkZGdYlJSWlfB8EAOCqkAUA4Lwc+p2FwsJCXX/99Xr++eclSW3bttXu3bv15ptvKjY2tlRzenh4yMPDoyzLBACUI7IAAJyXQ5uF0NBQRUVF2Yw1b95cS5YskSSFhIRIktLS0hQaGmrdJi0tTW3atKmwOgGUPWOMLly4oIKCAkeXUqW4uLjI1dW1Ul0elCwAqi+yoHyUZRY4tFno0qWLkpKSbMb27dun8PBwSVJkZKRCQkKUmJhoDYTMzEz9+OOPeuihhyq6XABlJC8vT8eOHdO5c+ccXUqVVLNmTYWGhsrd3d3RpRQLWQBUT2RB+SqrLHBoszBu3Dh17txZzz//vO68805t3rxZc+fO1dy5cyVJFotFY8eO1bPPPqsmTZpYL5dXr149DRw40JGlAyilwsJCJScny8XFRfXq1ZO7u3ul+hTcmRljlJeXp+PHjys5OVlNmjSpFD+8RhYA1Q9ZUH7KOgsc2ix06NBBn332mSZOnKjp06crMjJSs2fP1rBhw6zbTJgwQdnZ2Ro1apTOnDmjG2+8UStWrCjWdbUBOJ+8vDwVFhYqLCxMNWvWdHQ5VY6Xl5fc3Nz022+/KS8vr1L8W0kWANUPWVC+yjILHPo7CxXBGa8pDlRnOTk5Sk5OVmRkJP+jV06u9BxX138Tq+vjBpwVWVD+yioLHHpkAf8zb3t7R5dQ6Y1st83RJQDAVSELrh5ZAJQt5z+ZFQAAAIBD0CwAqJaMMRo1apQCAgJksVi0Y8cOdevWTWPHjr3i/SIiIjR79uwKqREAUL7Igr9GswDA6aSmpiouLk4NGzaUh4eHwsLC1L9/fyUmJpbZPlasWKH58+dr2bJlOnbsmFq2bKn//Oc/euaZZ8psHwCA0iMLnAPfWQBgVRHnS3uYEDVzfUynzhXIraDo5xWHfzui23r9n/z8aunpZx5RVItrlZ9/QasTf9CDD43Uhv8uK5M6du75QcEhddSktb+kUzqde0rykAok5WRf/n6FJl9Zeak6nr3H7vog7yi74wBQWZAFZMGlOLIAwKk8Oe4ZWSwWrVi7SP0H9lGjJhFqFtVYD8UN1/LVCyVJv6cc1b13jVFE8PVqGNpR998zXulpJ6xzzHwuXt2j79AnH32h9lG91aheJ42KfVxZZ//4lz/ugX9q4uPP6/eUY6rr00Lto3pLkgbeMlxPT5hhned4+kn935CH1aBOO13foo8+/bhoOGWcydS40ZPVPPxGNQztqB49emjnzp3W9VOnTlWbNm30/vvvKyIiQn5+fho6dKjOnj1r3aawsFAzZ85U48aN5eHhoQYNGui5556zrk9JSdGdd94pf39/BQQEaMCAATp06FDZPOEA4ITIAufJApoFAE7j9Kkz+m7VD/r7qLvl7V30utt+/r4qLCzUvXfF6fSpDH2+YoEWf/G2fjv0u0bFPm6z7aHkFC3/MlEfLH5DHy5+Qxt+2KJX//22JOm5mRP15NNjVO+aEO06uEYr135st55HHnxKR35P1X++TtA7H8xSwryPdOL4KZttRtwzXieOn9RHn72pb9ctVrt27dSzZ0+dOvW/7Q4ePKilS5dq2bJlWrZsmdauXasXXnjBun7ixIl64YUXNGnSJO3Zs0cLFy5UcHCwJCk/P18xMTGqVauW1q1bp/Xr18vHx0e33HKL8vLySvdEA4ATIwucKws4DQmA00j+9bCMMWp8beRlt/l+zSbt/Xm/tv68UtfUD5UkvT73ed3UYYD+u22X2ra/TpJkCo1ee+t5+dTyliQNubu/1q3dJOlR+frVkk8tb7m41FBwcJDd/Rzcf0iJ36zTyrWLrHPOjn9GXdr3t26zacM2/XfbLu1JXicPD3dJ0ksvvaSlS5fq008/1ahRoyT98WnR/PnzVatWLUnSPffco8TERD333HM6e/asXnnlFb3++uuKjY2VJDVq1Eg33nijJOnjjz9WYWGh3n77beuvmyYkJMjf319r1qxRnz59Sv5EA4ATIwucKwtoFgA4jeL8ROT+X37VNfVDrOEgSU2bN5afv6/2Jf1q/cc8rEE9azhIUnBIkI7/6ZOgK9mX9KtcXV3Vum0L61iTpg3l5/+/H6/5eVeSsrPOqWmDztYxi2ro/PnzOnjwoHUsIiLCGg6SFBoaqvT0dEnS3r17lZubq549e9qtY+fOnTpw4IDN/aU/fmzn0n0AQFVBFhTlyCygWQDgNBo2aiCLxaID+5Kvei5XN9t/3iwWi0xh4VXPe6ns7HMKDgnSZ8sTrGOBNZtIkvz9/a1jbm5uRWop/P9r8fLyuuI+srKy1L59e3344YdF1gUF2f8kDAAqM7KgKEdmAd9ZAOA0agf4q3uvLnp37kfKzj5XZH3GmUw1adZQR35P1ZHfj1nHk/YeUMaZTDVt1qjMamlybaQuXLignf/92Tp2YF+yMs5kWm+3ahOl9LQTcnV1VcNG4WrYKFyNGzdW48aNVadOneLtp0kTeXl5XfZSgO3atdP+/ftVt25d69wXFz8/v6t7kADghMiCohyZBTQLAJzKCy8/rYKCAt3Sdai+XPqNfj3wm/b9clDz3vhA/XoMU9fu0Wreooke+vuT+mnHHm3f+pPGjPqnOt/YQW3atSyzOhpfG6kevW/U449M07YtP2nnf3/WuNGT5eXlad2ma/doXd+xtWKHxml14nod/u2INmzYoKeeekpbt24t1n48PT315JNPasKECXrvvfd08OBBbdq0Se+8844kadiwYapTp44GDBigdevWKTk5WWvWrNEjjzyi33//vcweLwA4E7LAebKAZgGAU4mIDFPi+k/V5eaOmvrPF3VzxwEacvtIrVu7STNnT5LFYtF7H78m/9q+uj3mXg3uf7/CI+pr7oKXyryWV958ViGhdTXwlljd9/8e1T1/H6I6QQHW9RaLRR/9501Fd7lejz74tKLb9NPQoUP122+/Wa9gURyTJk3SY489psmTJ6t58+a66667rOex1qxZU99//70aNGigO+64Q82bN9eIESOUk5MjX1/fv5gZAConssB5ssBiTHG+RlJ5ZWZmys/PTxkZGU4drBXxAyhV3ch22xxdQqVXkT/EUz+srtw8qt7nFc7wQzw5OTlKTk5WZGSkPD09bdZVln8Ty1pledxkwdUjC64eWXD1qlIWVL2/DgAAAIAyQbMAAAAAwC6aBQAAAAB20SwAAAAAsItmAQAAAIBdNAsAAAAA7KJZAAAAAGAXzQIAAAAAu2gWAAAAANjl6ugCAKAsZEY8UGH78j30VoXtCwBQfGRB2ePIAgBUgO+//179+/dXvXr1ZLFYtHTpUkeXBACoYJUxC2gWAKACZGdnq3Xr1oqPj3d0KQAAB6mMWcBpSABQAfr27au+ffs6ugwAgANVxizgyAIAAAAAuxzaLEydOlUWi8VmadasmXV9Tk6ORo8ercDAQPn4+GjQoEFKS0tzYMUAgLJGFgCA83L4kYUWLVro2LFj1uWHH36wrhs3bpy+/PJLLV68WGvXrtXRo0d1xx13OLBaAEB5IAsAwDk5/DsLrq6uCgkJKTKekZGhd955RwsXLlSPHj0kSQkJCWrevLk2bdqkG264oaJLBQCUE7IAAJyTw48s7N+/X/Xq1VPDhg01bNgwHT58WJK0bds25efnq1evXtZtmzVrpgYNGmjjxo2XnS83N1eZmZk2CwDAuZEFAOCcHNosdOrUSfPnz9eKFSs0Z84cJScn66abbtLZs2eVmpoqd3d3+fv729wnODhYqampl51zxowZ8vPzsy5hYWHl/CgA4K9lZWVpx44d2rFjhyQpOTlZO3bssP5PcXVGFgCoLipjFjj0NKRLLx3VqlUrderUSeHh4frkk0/k5eVVqjknTpyo8ePHW29nZmYSEkA14Oy/pLl161Z1797devviv1OxsbGaP3++g6pyDmQBgLJCFpQ9h39n4VL+/v669tprdeDAAfXu3Vt5eXk6c+aMzSdKaWlpds9rvcjDw0MeHh4VUC0AFF+3bt1kjHF0GZUCWQCgqqqMWeDw7yxcKisrSwcPHlRoaKjat28vNzc3JSYmWtcnJSXp8OHDio6OdmCVAIDyRBYAgPNw6JGFxx9/XP3791d4eLiOHj2qKVOmyMXFRXfffbf8/Pw0YsQIjR8/XgEBAfL19VVcXJyio6O5+gUAVCFkAQA4L4c2C7///rvuvvtunTx5UkFBQbrxxhu1adMmBQUFSZJmzZqlGjVqaNCgQcrNzVVMTIzeeOMNR5YMAChjZAEAOC+HNguLFi264npPT0/Fx8crPj6+gioCAFQ0sgAAnJdTfWcBAAAAgPOgWQAAAABgF80CAAAAALtoFgAAAADYRbMAAAAAwC6n+gVnACitoBefqrB9HX/iuQrbFwCg+MiCsseRBQAoZzNmzFCHDh1Uq1Yt1a1bVwMHDlRSUpKjywIAVKDKmgU0CwBQztauXavRo0dr06ZNWrVqlfLz89WnTx9lZ2c7ujQAQAWprFnAaUgAUM5WrFhhc3v+/PmqW7eutm3bpptvvtlBVQEAKlJlzQKOLABABcvIyJAkBQQEOLgSAICjVJYsoFkAgApUWFiosWPHqkuXLmrZsqWjywEAOEBlygJOQwKACjR69Gjt3r1bP/zwg6NLAQA4SGXKApoFAKggY8aM0bJly/T999+rfv36ji4HAOAAlS0LaBYAoJwZYxQXF6fPPvtMa9asUWRkpKNLAgBUsMqaBTQLAFDORo8erYULF+rzzz9XrVq1lJqaKkny8/OTl5eXg6sDAFSEypoFNAsAqgRn/iXNOXPmSJK6detmM56QkKDhw4dXfEEAUEWRBWWPZgEAypkxxtElAAAcrLJmAZdOBQAAAGAXzQIAAAAAu2gWAAAAANhFswAAAADALpoFAAAAAHbRLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBe/4AygSliadE+F7Wtg0/crbF8AgOIjC8oeRxYAoALMmTNHrVq1kq+vr3x9fRUdHa3ly5c7uiwAQAWqjFngNM3CCy+8IIvForFjx1rHcnJyNHr0aAUGBsrHx0eDBg1SWlqa44oEgFKqX7++XnjhBW3btk1bt25Vjx49NGDAAP3888+OLs2pkAUAqrLKmAVO0Sxs2bJFb731llq1amUzPm7cOH355ZdavHix1q5dq6NHj+qOO+5wUJUAUHr9+/dXv3791KRJE1177bV67rnn5OPjo02bNjm6NKdBFgCo6ipjFji8WcjKytKwYcM0b9481a5d2zqekZGhd955Ry+//LJ69Oih9u3bKyEhQRs2bHDqJxQA/kpBQYEWLVqk7OxsRUdHO7ocp0AWAKhuKksWOLxZGD16tG699Vb16tXLZnzbtm3Kz8+3GW/WrJkaNGigjRs3Xna+3NxcZWZm2iwA4Ax27dolHx8feXh46MEHH9Rnn32mqKgoR5flFMgCANVFZcsCh14NadGiRdq+fbu2bNlSZF1qaqrc3d3l7+9vMx4cHKzU1NTLzjljxgxNmzatrEsFgKvWtGlT7dixQxkZGfr0008VGxurtWvXOnVIVASyAEB1UtmywGFHFlJSUvToo4/qww8/lKenZ5nNO3HiRGVkZFiXlJSUMpsbAK6Gu7u7GjdurPbt22vGjBlq3bq1XnnlFUeX5VBkAYDqprJlgcOahW3btik9PV3t2rWTq6urXF1dtXbtWr366qtydXVVcHCw8vLydObMGZv7paWlKSQk5LLzenh4WC9HdXEBAGdUWFio3NxcR5fhUGQBgOrO2bPAYach9ezZU7t27bIZu++++9SsWTM9+eSTCgsLk5ubmxITEzVo0CBJUlJSkg4fPuzUXwIBAHsmTpyovn37qkGDBjp79qwWLlyoNWvWaOXKlY4uzaHIAgDVSWXMAoc1C7Vq1VLLli1txry9vRUYGGgdHzFihMaPH6+AgAD5+voqLi5O0dHRuuGGGxxRMgAn5uy/pJmenq57771Xx44dk5+fn1q1aqWVK1eqd+/eji7NocgCAGWJLCh7Dv2C81+ZNWuWatSooUGDBik3N1cxMTF64403HF0WAJTYO++84+gSKi2yAEBVURmzwKmahTVr1tjc9vT0VHx8vOLj4x1TEACgwpEFAOA8HP47CwAAAACcE80CAAAAALtoFgAAAADYRbMAoIIVSjIyxtF1VF2GJxeA0yMLyltZZQHNAoAKla9MFRZeUF5uoaNLqbLOnTsnSXJzc3NwJQBgH1lQ/soqC5zqakgAqr5CS46OF/4g1xO9JfnL3aOGLBZHV1V2clxyHLZvY4zOnTun9PR0+fv7y8XFxWG1AMCVkAXlp6yzgGYBQIVL1XIpX7qQdqNq1HCVVHUSIsPd8f+D7u/vr5CQEEeXAQBXRBaUr7LKApoFABXPYpSqr5VemCi3Qj9VpTMi72y6xKH7d3Nz44gCgMqBLCg3ZZkFNAsAHKbQkqtcpTu6jDLl6enp6BIAoFIhC5wbzQIAAACsRn7RwNElVH7tHF1A2ak6x3sAAAAAlCmaBQAAAAB20SwAAAAAsKtUzULDhg118uTJIuNnzpxRw4YNr7ooAIDzIwsAoOorVbNw6NAhFRQUFBnPzc3VkSNHrrooAIDzIwsAoOor0dWQvvjiC+t/r1y5Un5+ftbbBQUFSkxMVERERJkVBwBwPmQBAFQfJWoWBg4cKEmyWCyKjY21Wefm5qaIiAj9+9//LrPiAADOhywAgOqjRM1CYWGhJCkyMlJbtmxRnTp1yqUoAIDzIgsAoPoo1Y+yJScnl3UdAIBKhiwAgKqv1L/gnJiYqMTERKWnp1s/Zbro3XffverCAADOjywAgKqtVM3CtGnTNH36dF1//fUKDQ2VxWIp67oAAE6OLACAqq9UzcKbb76p+fPn65577inregAAlQRZAABVX6l+ZyEvL0+dO3cu61oAAJUIWQAAVV+pmoX7779fCxcuLOtaAACVCFkAAFVfqU5DysnJ0dy5c/Xtt9+qVatWcnNzs1n/8ssvl0lxAADnRRYAQNVXqmbhp59+Ups2bSRJu3fvtlnHF9wAoHogCwCg6itVs7B69eqyrgMAUMmQBQBQ9ZXqOwsAAAAAqr5SHVno3r37FQ8xf/fdd6UuCABQOZAFAFD1lerIQps2bdS6dWvrEhUVpby8PG3fvl3XXXddseeZM2eOWrVqJV9fX/n6+io6OlrLly+3rs/JydHo0aMVGBgoHx8fDRo0SGlpaaUpGQBQxsgCAKj6SnVkYdasWXbHp06dqqysrGLPU79+fb3wwgtq0qSJjDFasGCBBgwYoP/+979q0aKFxo0bp6+++kqLFy+Wn5+fxowZozvuuEPr168vTdkAgDJEFgBA1WcxxpiymuzAgQPq2LGjTp06Veo5AgIC9OKLL2rw4MEKCgrSwoULNXjwYEnSL7/8oubNm2vjxo264YYb7N4/NzdXubm51tuZmZkKCwtTRkaGfH19S11XeZu3vb2jS6j0Rrbb5ugSKj1eh1fP2V+HmZmZ8vPzK9d/E8mC0uM9ePWc/T1YKUz9m6MrqPymfuboCq6oJFlQpl9w3rhxozw9PUt134KCAi1atEjZ2dmKjo7Wtm3blJ+fr169elm3adasmRo0aKCNGzdedp4ZM2bIz8/PuoSFhZWqHgBA6ZAFAFB1lOo0pDvuuMPmtjFGx44d09atWzVp0qQSzbVr1y5FR0crJydHPj4++uyzzxQVFaUdO3bI3d1d/v7+NtsHBwcrNTX1svNNnDhR48ePt96++GkSAKBskQUAUPWVqlnw8/OzuV2jRg01bdpU06dPV58+fUo0V9OmTbVjxw5lZGTo008/VWxsrNauXVuasiRJHh4e8vDwKPX9AQDFQxYAQNVXqmYhISGhzApwd3dX48aNJUnt27fXli1b9Morr+iuu+5SXl6ezpw5Y/OJUlpamkJCQsps/wCA0iELAKDqK1WzcNG2bdu0d+9eSVKLFi3Utm3bqy6osLBQubm5at++vdzc3JSYmKhBgwZJkpKSknT48GFFR0df9X4AFDXyiwaOLqHya+foAioeWQAAVVepmoX09HQNHTpUa9assX7Sc+bMGXXv3l2LFi1SUFBQseaZOHGi+vbtqwYNGujs2bNauHCh1qxZo5UrV8rPz08jRozQ+PHjFRAQIF9fX8XFxSk6OvqyV78AAFQcsgAAqr5SXQ0pLi5OZ8+e1c8//6xTp07p1KlT2r17tzIzM/XII48Ue5709HTde++9atq0qXr27KktW7Zo5cqV6t27t6Q/ruF92223adCgQbr55psVEhKi//znP6UpGQBQxsgCAKj6SvU7C35+fvr222/VoUMHm/HNmzerT58+OnPmTFnVd9Uq4priZYFra189rq1dBri29tWrQtfW/itkQdkjC64eWVAGyIKrV4WyoFRHFgoLC+Xm5lZk3M3NTYWFhaWZEgBQyZAFAFD1lapZ6NGjhx599FEdPXrUOnbkyBGNGzdOPXv2LLPiAADOiywAgKqvVM3C66+/rszMTEVERKhRo0Zq1KiRIiMjlZmZqddee62sawQAOCGyAACqvlJdDSksLEzbt2/Xt99+q19++UWS1Lx5c/Xq1atMiwMAOC+yAACqvhIdWfjuu+8UFRWlzMxMWSwW9e7dW3FxcYqLi1OHDh3UokULrVu3rrxqBQA4AbIAAKqPEjULs2fP1siRI+1+a9rPz08PPPCAXn755TIrDgDgfMgCAKg+StQs7Ny5U7fccstl1/fp00fbtnHJMgCoysgCAKg+StQspKWl2b1M3kWurq46fvz4VRcFAHBeZAEAVB8lahauueYa7d69+7Lrf/rpJ4WGhl51UQAA50UWAED1UaJmoV+/fpo0aZJycnKKrDt//rymTJmi2267rcyKAwA4H7IAAKqPEl069emnn9Z//vMfXXvttRozZoyaNm0qSfrll18UHx+vgoICPfXUU+VSKADAOZAFAFB9lKhZCA4O1oYNG/TQQw9p4sSJMsZIkiwWi2JiYhQfH6/g4OByKRQA4BzIAgCoPkr8o2zh4eH6+uuvdfr0aR04cEDGGDVp0kS1a9cuj/oAAE6ILACA6qFUv+AsSbVr11aHDh3KshYAQCVDFgBA1VaiLzgDAAAAqD5oFgAAAADYRbMAAAAAwC6aBQAAAAB20SwAAAAAsItmAQAAAIBdNAsAAAAA7KJZAAAAAGAXzQIAAAAAu2gWAAAAANjl6ugCAADAH0Z+0cDRJVR+7RxdAFC1cGQBAAAAgF00CwAAAADsolkAAAAAYJdDm4UZM2aoQ4cOqlWrlurWrauBAwcqKSnJZpucnByNHj1agYGB8vHx0aBBg5SWluagigEAZY0sAADn5dBmYe3atRo9erQ2bdqkVatWKT8/X3369FF2drZ1m3HjxunLL7/U4sWLtXbtWh09elR33HGHA6sGAJQlsgAAnJdDr4a0YsUKm9vz589X3bp1tW3bNt18883KyMjQO++8o4ULF6pHjx6SpISEBDVv3lybNm3SDTfcUGTO3Nxc5ebmWm9nZmaW74MAAFwVsgAAnJdTfWchIyNDkhQQECBJ2rZtm/Lz89WrVy/rNs2aNVODBg20ceNGu3PMmDFDfn5+1iUsLKz8CwcAlBmyAACch9M0C4WFhRo7dqy6dOmili1bSpJSU1Pl7u4uf39/m22Dg4OVmppqd56JEycqIyPDuqSkpJR36QCAMkIWAIBzcZofZRs9erR2796tH3744arm8fDwkIeHRxlVBQCoSGQBADgXpziyMGbMGC1btkyrV69W/fr1reMhISHKy8vTmTNnbLZPS0tTSEhIBVcJAChPZAEAOB+HNgvGGI0ZM0afffaZvvvuO0VGRtqsb9++vdzc3JSYmGgdS0pK0uHDhxUdHV3R5QIAygFZAADOy6GnIY0ePVoLFy7U559/rlq1alnPPfXz85OXl5f8/Pw0YsQIjR8/XgEBAfL19VVcXJyio6PtXv0CAFD5kAUA4Lwc2izMmTNHktStWzeb8YSEBA0fPlySNGvWLNWoUUODBg1Sbm6uYmJi9MYbb1RwpQCA8kIWAIDzcmizYIz5y208PT0VHx+v+Pj4CqgIAFDRyAIAcF5O8QVnAAAAAM6HZgEAAACAXTQLAAAAAOyiWQAAAABgF80CAAAAALtoFgAAAADYRbMAAAAAwC6aBQAAAAB20SwAAAAAsItmAQAAAIBdNAsAAAAA7KJZAAAAAGAXzQIAAAAAu2gWAAAAANhFswAAAADALpoFAAAAAHbRLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALCLZgEAAACAXTQLAAAAAOyiWQAAAABgl0Obhe+//179+/dXvXr1ZLFYtHTpUpv1xhhNnjxZoaGh8vLyUq9evbR//37HFAsAKBdkAQA4L4c2C9nZ2WrdurXi4+Ptrp85c6ZeffVVvfnmm/rxxx/l7e2tmJgY5eTkVHClAIDyQhYAgPNydeTO+/btq759+9pdZ4zR7Nmz9fTTT2vAgAGSpPfee0/BwcFaunSphg4dWpGlAgDKCVkAAM7Lab+zkJycrNTUVPXq1cs65ufnp06dOmnjxo2XvV9ubq4yMzNtFgBA5UQWAIBjOfTIwpWkpqZKkoKDg23Gg4ODrevsmTFjhqZNm1autQEAKgZZAFS8g/EnHF1CpddoqqMrKDtOe2ShtCZOnKiMjAzrkpKS4uiSAAAVjCwAgLLhtM1CSEiIJCktLc1mPC0tzbrOHg8PD/n6+tosAIDKiSwAAMdy2mYhMjJSISEhSkxMtI5lZmbqxx9/VHR0tAMrAwBUFLIAABzLod9ZyMrK0oEDB6y3k5OTtWPHDgUEBKhBgwYaO3asnn32WTVp0kSRkZGaNGmS6tWrp4EDBzqu6HIy8osGji6h8mvn6AIAlAZZAADOy6HNwtatW9W9e3fr7fHjx0uSYmNjNX/+fE2YMEHZ2dkaNWqUzpw5oxtvvFErVqyQp6eno0oGAJQxsgAAnJdDm4Vu3brJGHPZ9RaLRdOnT9f06dMrsCoAQEUiCwDAeTntdxYAAAAAOBbNAgAAAAC7aBYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALCLZgEAAACAXTQLAAAAAOyiWQAAAABgF80CAAAAALtoFgAAAADYRbMAAAAAwC5XRxcAAAD+cDD+hKNLqPQaTXV0BUDVwpEFAAAAAHbRLAAAAACwi2YBAAAAgF00CwAAAADsolkAAAAAYBfNAgAAAAC7aBYAAAAA2MXvLACw4hrvV49rvAMAqhKOLAAAAACwi2YBAAAAgF00CwAAAADs4jsLToJzxa8e54oDAACULY4sAAAAALCLZgEAAACAXTQLAAAAAOyiWQAAAABgV6VoFuLj4xURESFPT0916tRJmzdvdnRJAIAKRhYAQMVz+mbh448/1vjx4zVlyhRt375drVu3VkxMjNLT0x1dGgCggpAFAOAYTn/p1JdfflkjR47UfffdJ0l688039dVXX+ndd9/VP/7xjyLb5+bmKjc313o7IyNDkpSZmVkxBZfS2cILji6h0nP2v3FlwOvw6jn76/BifcYYB1dSMmQBisvZ/8aVAa/Dq+fsr8MSZYFxYrm5ucbFxcV89tlnNuP33nuvuf322+3eZ8qUKUYSCwsLC8sVlpSUlAr4V7xskAUsLCws5bMUJwuc+sjCiRMnVFBQoODgYJvx4OBg/fLLL3bvM3HiRI0fP956u7CwUKdOnVJgYKAsFku51ltVZWZmKiwsTCkpKfL19XV0OajGeC1ePWOMzp49q3r16jm6lGIjC5wD7z84C16LV68kWeDUzUJpeHh4yMPDw2bM39/fMcVUMb6+vrwp4RR4LV4dPz8/R5dQ7siC8sP7D86C1+LVKW4WOPUXnOvUqSMXFxelpaXZjKelpSkkJMRBVQEAKhJZAACO49TNgru7u9q3b6/ExETrWGFhoRITExUdHe3AygAAFYUsAADHcfrTkMaPH6/Y2Fhdf/316tixo2bPnq3s7GzrFTFQ/jw8PDRlypQih/SBisZrsfoiCxyP9x+cBa/FimUxxvmvn/f666/rxRdfVGpqqtq0aaNXX31VnTp1cnRZAIAKRBYAQMWrFM0CAAAAgIrn1N9ZAAAAAOA4NAsAAAAA7KJZAAAAAGAXzQKuKCIiQrNnzy6XuS0Wi5YuXerwOfDX5s+fX+l/0Kpbt24aO3asw+cAKiOyABJZUJZzVCoGZSI2NtZIMpKMq6uriYiIME888YQ5f/68o0srloSEBOPn51dkPD093WRnZ5d63vDwcOvzcnG55pprjDHGHDt2zOTk5BRrnilTppjWrVsXGS/JHNVZenq6efDBB01YWJhxd3c3wcHBpk+fPuaHH34o1v3PnTtn0tLSyrnKq3fp+/DSZf/+/ebkyZMmMzOzWPOsXr3aSDKnT5+2GS/JHKieyAL7yALnQBaQBaXh9L+zUJnccsstSkhIUH5+vrZt26bY2FhZLBb961//cnRppRYUFHTVc0yfPl0jR4603nZxcZGkMvnlVX69tXgGDRqkvLw8LViwQA0bNlRaWpoSExN18uTJYt3fy8tLXl5e5Vxl2bj4PrxUUFCQ9XV3NQICAq56DlR9ZIF9ZIHjkQVkQak4ulupKmJjY82AAQNsxu644w7Ttm1b6+2cnBwTFxdngoKCjIeHh+nSpYvZvHmzdf3FDnbFihWmTZs2xtPT03Tv3t2kpaWZr7/+2jRr1szUqlXL3H333Taf8Cxfvtx06dLF+Pn5mYCAAHPrrbeaAwcOWNcnJycbSWbJkiWmW7duxsvLy7Rq1cps2LDBZr+XLlOmTDHG/PFp0KxZs6xznT592owaNcrUrVvXeHh4mBYtWpgvv/zyss/Ln+9/KUnms88+s95OSUkxQ4cONbVr1zY1a9Y07du3N5s2bTIJCQlF6ktISLA7x08//WS6d+9uPD09TUBAgBk5cqQ5e/Zskb/Tiy++aEJCQkxAQIB5+OGHTV5e3mUfQ2V3+vRpI8msWbPmL7e73N/W3qeNS5cuNW3btjUeHh4mMjLSTJ061eTn51vXSzLz5s0zAwcONF5eXqZx48bm888/t5lj9+7d5tZbbzW1atUyPj4+5sYbb7R57c6bN880a9bMeHh4mKZNm5r4+PgrPgZ778OLunbtah599FHr7ZycHDNhwgRTv3594+7ubho1amTefvtt6/vl0iU2NtbuHKdOnTL33HOP8ff3N15eXuaWW24x+/bts66/+LytWLHCNGvWzHh7e5uYmBhz9OjRKz4OVF5kgX1kgeORBX8gC0qO7yyUk927d2vDhg1yd3e3jk2YMEFLlizRggULtH37djVu3FgxMTE6deqUzX2nTp2q119/XRs2bFBKSoruvPNOzZ49WwsXLtRXX32lb775Rq+99pp1++zsbI0fP15bt25VYmKiatSoob/97W8qLCy0mfepp57S448/rh07dujaa6/V3XffrQsXLqhz586aPXu2fH19dezYMR07dkyPP/54kcdUWFiovn37av369frggw+0Z88evfDCC2XSpWdlZalr1646cuSIvvjiC+3cuVMTJkxQYWGh7rrrLj322GNq0aKFtb677rqryBzZ2dmKiYlR7dq1tWXLFi1evFjffvutxowZY7Pd6tWrdfDgQa1evVoLFizQ/PnzNX/+/Kt+DM7Kx8dHPj4+Wrp0qXJzc+1uU9K/7bp163Tvvffq0Ucf1Z49e/TWW29p/vz5eu6552y2mzZtmu6880799NNP6tevn4YNG2Z9vR85ckQ333yzPDw89N1332nbtm36+9//rgsXLkiSPvzwQ02ePFnPPfec9u7dq+eff16TJk3SggULyuR5uffee/XRRx/p1Vdf1d69e/XWW2/Jx8dHYWFhWrJkiSQpKSlJx44d0yuvvGJ3juHDh2vr1q364osvtHHjRhlj1K9fP+Xn51u3OXfunF566SW9//77+v7773X48GG77y9UTWRByZAF5YcssI8sKAZHdytVRWxsrHFxcTHe3t7Gw8PDSDI1atQwn376qTHGmKysLOPm5mY+/PBD633y8vJMvXr1zMyZM40x//tU59tvv7VuM2PGDCPJHDx40Dr2wAMPmJiYmMvWcvz4cSPJ7Nq1yxjzv0+T3n77bes2P//8s5Fk9u7da4y5/Hmql34atHLlSlOjRg2TlJRU7OclPDzcuLu7G29vb+vyyiuvGGNsPwl66623TK1atczJkyftznO581QvnWPu3Lmmdu3aJisry7r+q6++MjVq1DCpqanGmD/+TuHh4ebChQvWbYYMGWLuuuuuYj+myujTTz81tWvXNp6enqZz585m4sSJZufOndb1f/W3/fPro2fPnub555+32eb99983oaGh1tuSzNNPP229nZWVZSSZ5cuXG2OMmThxoomMjLzsJ3mNGjUyCxcutBl75plnTHR09GUf56Xvw4vL4MGDjTG2nwQlJSUZSWbVqlV257nceaqXzrFv3z4jyaxfv966/sSJE8bLy8t88sknxhhj/ST00k/I4uPjTXBw8GUfAyo3ssA+ssA5kAVkQWnwnYUy1L17d82ZM0fZ2dmaNWuWXF1dNWjQIEnSwYMHlZ+fry5duli3d3NzU8eOHbV3716beVq1amX97+DgYNWsWVMNGza0Gdu8ebP19v79+zV58mT9+OOPOnHihPVTpMOHD6tly5Z25w0NDZUkpaenq1mzZsV6fDt27FD9+vV17bXXFmv7i5544gkNHz7certOnTp2527btu1VnQe4d+9etW7dWt7e3taxLl26qLCwUElJSQoODpYktWjRwuZTktDQUO3atavU+60MBg0apFtvvVXr1q3Tpk2btHz5cs2cOVNvv/22hg8fXuK/7c6dO7V+/XqbT48KCgqUk5Ojc+fOqWbNmpJsX3Pe3t7y9fVVenq6pD/+5jfddJPc3NyKzJ+dna2DBw9qxIgRNuc4X7hwQX5+fles7eL78NL9/tmOHTvk4uKirl27Fuvx2rN37165urqqU6dO1rHAwEA1bdrU5j1ds2ZNNWrUyHo7NDTU+hygaiIL7CMLHI8ssEUWFA/NQhny9vZW48aNJUnvvvuuWrdurXfeeUcjRowo0TyXvmEsFkuRN5DFYrE5rNy/f3+Fh4dr3rx5qlevngoLC9WyZUvl5eVdcV5JRQ5PX0lpv9RUp04d6/NS1nOXxl89n1WVp6enevfurd69e2vSpEm6//77NWXKFA0fPrzEz39WVpamTZumO+64w+5+LrrSc32lfWZlZUmS5s2bZ/MPsKS/PNXh0vfh5Tj69WaMqbD9o+KRBfaRBc6BLPgfR7/eKksW8J2FclKjRg3985//1NNPP63z58+rUaNGcnd31/r1663b5Ofna8uWLYqKiir1fk6ePKmkpCQ9/fTT6tmzp5o3b67Tp0+XeB53d3cVFBRccZtWrVrp999/1759+0pb7hXn3rFjR5FzdktSX/PmzbVz505lZ2dbx9avX68aNWqoadOmZVpvVRAVFWV9rkr6t23Xrp2SkpLUuHHjIkuNGsX7Z6VVq1Zat26dzTmdFwUHB6tevXr69ddfi8wfGRlZ/Ad5Gdddd50KCwu1du1au+svnl9+pddc8+bNdeHCBf3444/WsYvvx6t5T6NqIQtKhiyoeGQBWfBXaBbK0ZAhQ+Ti4qL4+Hh5e3vroYce0hNPPKEVK1Zoz549GjlypM6dO1fiT5suVbt2bQUGBmru3Lk6cOCAvvvuO40fP77E80RERCgrK0uJiYk6ceKEzp07V2Sbrl276uabb9agQYO0atUqJScna/ny5VqxYkWp67/o7rvvVkhIiAYOHKj169fr119/1ZIlS7Rx40ZrfcnJydqxY4dOnDhh98tZw4YNk6enp2JjY7V7926tXr1acXFxuueee6yHnaujkydPqkePHvrggw/0008/KTk5WYsXL9bMmTM1YMAASSX/206ePFnvvfeepk2bpp9//ll79+7VokWL9PTTTxe7rjFjxigzM1NDhw7V1q1btX//fr3//vtKSkqS9McX4mbMmKFXX31V+/bt065du5SQkKCXX375qp+TiIgIxcbG6u9//7uWLl2q5ORkrVmzRp988okkKTw8XBaLRcuWLdPx48etn25dqkmTJhowYIBGjhypH374QTt37tT//d//6ZprrrE+r4BEFpQEWVB+yIKiyILioVkoR66urhozZoxmzpyp7OxsvfDCCxo0aJDuuecetWvXTgcOHNDKlStVu3btUu+jRo0aWrRokbZt26aWLVtq3LhxevHFF0s8T+fOnfXggw/qrrvuUlBQkGbOnGl3uyVLlqhDhw66++67FRUVpQkTJvzlpzzF4e7urm+++UZ169ZVv379dN1119lcgWHQoEG65ZZb1L17dwUFBemjjz4qMkfNmjW1cuVKnTp1Sh06dNDgwYPVs2dPvf7661ddX2Xm4+OjTp06adasWbr55pvVsmVLTZo0SSNHjrR5bkryt42JidGyZcv0zTffqEOHDrrhhhs0a9YshYeHF7uuwMBAfffdd9arn7Rv317z5s2zHqq9//779fbbbyshIUHXXXedunbtqvnz55fJp0mSNGfOHA0ePFgPP/ywmjVrppEjR1o/Xbvmmms0bdo0/eMf/1BwcHCRq6hclJCQoPbt2+u2225TdHS0jDH6+uuv7Z57i+qLLCg+sqD8kAX2kQV/zWIqywlTAAAAACoURxYAAAAA2EWzAAAAAMAumgUAAAAAdtEsAAAAALCLZgEAAACAXTQLAAAAAOyiWQAAAABgF80CAAAAALtoFgAAAADYRbMAAAAAwC6aBQAAAAB20SwAAAAAsItmAQAAAIBdNAsAAAAA7KJZAAAAAGAXzQIAAAAAu2gWAAAAANhFswD8hdTUVD366KNq3LixPD09FRwcrC5dumjOnDk6d+6co8sDAFQAsgDVlaujCwCc2a+//qouXbrI399fzz//vK677jp5eHho165dmjt3rq655hrdfvvt5bLvvLw8ubu7l8vcAIDiIwtQnXFkAbiChx9+WK6urtq6davuvPNONW/eXA0bNtSAAQP01VdfqX///pKkM2fO6P7771dQUJB8fX3Vo0cP7dy50zrP1KlT1aZNG73//vuKiIiQn5+fhg4dqrNnz1q36datm8aMGaOxY8eqTp06iomJkSTt3r1bffv2lY+Pj4KDg3XPPffoxIkTFftEAEA1RhagOqNZAC7j5MmT+uabbzR69Gh5e3vb3cZisUiShgwZovT0dC1fvlzbtm1Tu3bt1LNnT506dcq67cGDB7V06VItW7ZMy5Yt09q1a/XCCy/YzLdgwQK5u7tr/fr1evPNN3XmzBn16NFDbdu21datW7VixQqlpaXpzjvvLL8HDgCwIgtQ7RkAdm3atMlIMv/5z39sxgMDA423t7fx9vY2EyZMMOvWrTO+vr4mJyfHZrtGjRqZt956yxhjzJQpU0zNmjVNZmamdf0TTzxhOnXqZL3dtWtX07ZtW5s5nnnmGdOnTx+bsZSUFCPJJCUllcnjBABcHlmA6o7vLAAltHnzZhUWFmrYsGHKzc3Vzp07lZWVpcDAQJvtzp8/r4MHD1pvR0REqFatWtbboaGhSk9Pt7lP+/btbW7v3LlTq1evlo+PT5E6Dh48qGuvvbYsHhIAoITIAlQXNAvAZTRu3FgWi0VJSUk24w0bNpQkeXl5SZKysrIUGhqqNWvWFJnD39/f+t9ubm426ywWiwoLC23G/nyIOysrS/3799e//vWvInOHhoYW+7EAAEqHLEB1R7MAXEZgYKB69+6t119/XXFxcZc9V7Vdu3ZKTU2Vq6urIiIiyrSGdu3aacmSJYqIiJCrK29XAKhoZAGqO77gDFzBG2+8oQsXLuj666/Xxx9/rL179yopKUkffPCBfvnlF7m4uKhXr16Kjo7+/9q787Co6v7/469hGxBkEETARMAlBU1NM0PNfe3WNMnSb1+3TLsNNTVvvbnNrUXNutMWs2wBW8xSy8xSU1wyl1RK0zRSw1wBN0BQUOH8/ujnfJs4miAyqM/HdZ3r4nzOZz7nPWdmgNecTd27d9c333yjAwcOaOPGjRo3bpy2bdt2TeuPjY3VqVOn1Lt3b23dulX79+/XihUrNGDAAOXn55fQswQAXAl/C3ArI54CV1C9enX9+OOPmjJliuLi4nT48GFZrVZFRUVp9OjReuKJJ2SxWPT1119r3LhxGjBggI4fP67g4GC1aNFCQUFB17T+ypUra8OGDRo7dqw6dOigvLw8hYWFqVOnTnJxIesDQGngbwFuZRbDMAxnFwEAAACg7CGOAgAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWABuEQcOHJDFYlFCQoK9bdKkSbJYLM4rCgAAlGmEBaAYjh49qkmTJmn79u3FHuPrr7/WpEmTSqwmAACAkkZYAIrh6NGjmjx58jWHhcmTJ5dcUQAAACWMsADcRAoKCpSbm+vsMgAAwE2CsIAy5/fff9cTTzyhWrVqycvLSwEBAerZs6cOHDjg0C8hIUEWi0UbNmzQqFGjFBgYKG9vbz3wwAM6fvy4Q9/w8HB16dJF3333ne6++255enqqWrVqev/99wut/7ffflPPnj3l7++vcuXK6Z577tFXX31lX7527Vo1btxYkjRgwABZLBaHcwHWr1+vnj17qmrVqrJarQoNDdXIkSN17tw5+xj9+/fXrFmzJMn++D+fO5CTk6OnnnpKoaGhslqtqlWrll566SUZhuFQq8Vi0dChQ/XRRx+pTp06slqtWr58edE3usmYCxYsUFRUlLy8vBQdHa2dO3dKkt566y3VqFFDnp6eatWqVaHXBQAA3DzcnF0A8Fdbt27Vxo0b1atXL1WpUkUHDhzQ7Nmz1apVK+3evVvlypVz6D9s2DBVqFBBEydO1IEDBzRz5kwNHTpUn3zyiUO/ffv26cEHH9TAgQPVr18/vffee+rfv78aNWqkOnXqSJLS0tLUtGlTnT17VsOHD1dAQIDmzp2r+++/XwsXLtQDDzygyMhIPfPMM5owYYIGDx6se++9V5LUtGlTSdKCBQt09uxZDRkyRAEBAdqyZYtee+01HT58WAsWLJAkPf744zp69KhWrlypDz74wKFOwzB0//33a82aNRo4cKAaNGigFStW6F//+peOHDmiGTNmOPRfvXq1Pv30Uw0dOlQVK1ZUeHj4Nb8G69ev15IlSxQbGytJmjp1qrp06aIxY8bojTfe0BNPPKHTp09r+vTpevTRR7V69eprXicAACiDDKCMOXv2bKG2TZs2GZKM999/394WHx9vSDLatWtnFBQU2NtHjhxpuLq6GhkZGfa2sLAwQ5Lx7bff2tvS09MNq9VqPPXUU/a2ESNGGJKM9evX29vOnDljREREGOHh4UZ+fr5hGIaxdetWQ5IRHx9/VfVPnTrVsFgsxu+//25vi42NNcw+gosXLzYkGc8995xD+4MPPmhYLBZj37599jZJhouLi/Hzzz8XGuevUlJSCtU8ceLEQjVIMqxWq5GSkmJve+uttwxJRnBwsJGVlWVvj4uLMyQ59AUAADcPDkNCmePl5WX/+cKFCzp58qRq1KghPz8//fDDD4X6Dx482OEQnnvvvVf5+fn6/fffHfpFRUXZ9wJIUmBgoGrVqqXffvvN3vb111/r7rvvVvPmze1tPj4+Gjx4sA4cOKDdu3cXqf6cnBydOHFCTZs2lWEY+vHHH//28V9//bVcXV01fPhwh/annnpKhmFo2bJlDu0tW7ZUVFTU345bFG3btnXYQ9GkSRNJUkxMjMqXL1+o/c/bEAAA3DwICyhzzp07pwkTJtiP169YsaICAwOVkZGhzMzMQv2rVq3qMF+hQgVJ0unTp6/Y71LfP/f7/fffVatWrUL9IiMj7cv/zsGDB9W/f3/5+/vLx8dHgYGBatmypSSZ1v9Xv//+uypXruzwT/mVaoiIiHCYP378uFJTU+1Tdnb2367zr/66rWw2myQpNDTUtP2v2xoAANwcOGcBZc6wYcMUHx+vESNGKDo6WjabTRaLRb169VJBQUGh/q6urqbjGH85Gfhq+12L/Px8tW/fXqdOndLYsWNVu3ZteXt768iRI+rfv79p/dfqz3syJKlx48YOgWLixIlFvp/D5bZVaWxDAABQdhAWUOYsXLhQ/fr103//+197W25urjIyMq77usPCwpScnFyo/ZdffrEvl3TZux7v3LlTv/76q+bOnau+ffva21euXFmo7+XGCAsL06pVq3TmzBmHvQt/reFyPvroI4crL1WrVu2K/QEAAC6Hw5BQ5ri6uhb6pvq1115Tfn7+dV/3fffdpy1btmjTpk32tpycHM2ZM0fh4eH2cwO8vb0lqVCAufTN+5/rNwxDr7zySqF1XW6M++67T/n5+Xr99dcd2mfMmCGLxaLOnTtf8Tk0a9ZM7dq1s0+EBQAAUFzsWUCZ06VLF33wwQey2WyKiorSpk2btGrVKgUEBFz3df/73//Wxx9/rM6dO2v48OHy9/fX3LlzlZKSokWLFsnF5Y98Xb16dfn5+enNN99U+fLl5e3trSZNmqh27dqqXr26Ro8erSNHjsjX11eLFi0yPaa/UaNGkqThw4erY8eOcnV1Va9evdS1a1e1bt1a48aN04EDB1S/fn198803+uKLLzRixAhVr179um8HAAAAiT0LKINeeeUV9e3bVx999JGeeuopHTt2TKtWrZKPj891X3dQUJA2btyo9u3b67XXXlNcXJw8PDz05Zdf6oEHHrD3c3d319y5c+Xq6qp//vOf6t27t9atWyd3d3d9+eWXatCggaZOnarJkyerZs2apjd/69Gjh4YNG6bly5erT58+6t27tyTJxcVFS5Ys0YgRI7R06VKNGDFCu3fv1osvvqiXX375um8DAACASywGZyYCAAAAMMGeBQAAAACmCAsAAAAATBEWAAAAAJgiLAAAAAAwRVgAAAAAYOqmv89CQUGBjh49qvLly1/2jrkAcKswDENnzpxR5cqV7fcNAQDgcpweFo4cOaKxY8dq2bJlOnv2rGrUqKH4+Hjdddddkv74wzZx4kS9/fbbysjIULNmzTR79mzVrFnzqsY/evSoQkNDr+dTAIAbzqFDh1SlShVnlwEAKOOcGhZOnz6tZs2aqXXr1lq2bJkCAwO1d+9eVahQwd5n+vTpevXVVzV37lxFRERo/Pjx6tixo3bv3i1PT8+/XUf58uUl/fGH0dfX97o9FwC4EWRlZSk0NNT+uxEAgCtx6k3Z/v3vf2vDhg1av3696XLDMFS5cmU99dRTGj16tCQpMzNTQUFBSkhIUK9evf52HVlZWbLZbMrMzCQsALjl8TsRAFAUTj1gdcmSJbrrrrvUs2dPVapUSXfeeafefvtt+/KUlBSlpqaqXbt29jabzaYmTZpo06ZNpmPm5eUpKyvLYQIAAABQdE4NC7/99pv9/IMVK1ZoyJAhGj58uObOnStJSk1NlSQFBQU5PC4oKMi+7K+mTp0qm81mnzhfAQAAACgep4aFgoICNWzYUFOmTNGdd96pwYMHa9CgQXrzzTeLPWZcXJwyMzPt06FDh0qwYgAAAODW4dQTnENCQhQVFeXQFhkZqUWLFkmSgoODJUlpaWkKCQmx90lLS1ODBg1Mx7RarbJardenYAAlKj8/XxcuXHB2GTcVd3d3ubq6OrsMAMBNwqlhoVmzZkpOTnZo+/XXXxUWFiZJioiIUHBwsBITE+3hICsrS99//72GDBlS2uUCKCGGYSg1NVUZGRnOLuWm5Ofnp+DgYO4tAwC4Zk4NCyNHjlTTpk01ZcoUPfTQQ9qyZYvmzJmjOXPmSJIsFotGjBih5557TjVr1rRfOrVy5crq3r27M0sHcA0uBYVKlSqpXLly/FNbQgzD0NmzZ5Weni5JDntkAQAoDqeGhcaNG+vzzz9XXFycnnnmGUVERGjmzJl65JFH7H3GjBmjnJwcDR48WBkZGWrevLmWL19+VfdYAFD25Ofn24NCQECAs8u56Xh5eUmS0tPTValSJQ5JAgBcE6feZ6E0cE1xoGzJzc1VSkqKwsPD7f/YomSdO3dOBw4cUERERKEvVvidCAAoCqfuWcD/efuHRs4u4YY3qGGSs0tAEXDo0fXDtgUAlBSnXjoVAAAAQNlFWABwSzIMQ4MHD5a/v78sFou2b9+uVq1aacSIEVd8XHh4uGbOnFkqNQIA4GyEBQBlTmpqqoYNG6Zq1arJarUqNDRUXbt2VWJiYomtY/ny5UpISNDSpUt17Ngx1a1bV5999pmeffbZElsHAAA3Os5ZAGBXGufOWI1g1XZ7SqfO5ss9v/D3FQd/P6Iu7f5XNlt5Pf3scEXVuV0XLlzUmsTv9M8hg7Txx6UlUseO3d8pKLiiatb3k3RKp/NOSVYpX1JuzuUfV2BcUPb5VB3P2W26PNA7yrQdAIAbEXsWAJQpY0c+K4vFouXr5qtr9w6qXjNctaNqaMiw/lq2Zp4k6fCho+r78FCFB92laiF367E+o5SedsI+xvTnZ6l1dA99+vESNYpqr+qVm2hwv9HKPvNHChj2+H8UN3qKDh86pko+ddQoqr0kqXun/np6zFT7OMfTT+p/ez6hqhUb6q46HbTwk8JBJTMjSyNjJygyrLmqhdytNm3aaMeOHfblkyZNUoMGDfTBBx8oPDxcNptNvXr10pkzZ+x9CgoKNH36dNWoUUNWq1VVq1bV888/b19+6NAhPfTQQ/Lz85O/v7+6deumAwcOlMwGBwDgCggLAMqM06cytHrld3p0cG95e5crtNzm56uCggL1fXiYTp/K1BfL52rBknf0+4HDGtxvtEPfAymHtOzLRH244A19tOANbfxuq1797zuSpOenx2ns00NV+bZg7dy/VivWfWJaz/B/jtORw6n67Ot4vfvhDMW//bFOHD/l0Gdgn1E6cfykPv78Ta1av0ANGzZU27ZtderU//Xbv3+/Fi9erKVLl2rp0qVat26dpk2bZl8eFxenadOmafz48dq9e7fmzZunoKAgSdKFCxfUsWNHlS9fXuvXr9eGDRvk4+OjTp066fz588Xb0AAAXCUOQwJQZqT8dlCGYajG7RGX7fPt2s3a8/Nebft5hW6r8scdil+fM0X3Nu6mH5N26s5Gd0iSjAJDr701RT7lvSVJPXt31fp1myU9KV9befmU95arq4uCggJN17N/7wElfrNeK9bNt485c9azataoq73P5o1J+jFpp3anrJfV6iFJeumll7R48WItXLhQgwcPlvTHnoOEhASVL19ektSnTx8lJibq+eef15kzZ/TKK6/o9ddfV79+/SRJ1atXV/PmzSVJn3zyiQoKCvTOO+/YL4kaHx8vPz8/rV27Vh06dCj6hgYA4CoRFgCUGVdzi8i9v/ym26oE24OCJNWKrCGbn69+Tf7N/o99aNXK9qAgSUHBgTr+l70CV/Jr8m9yc3NT/Tvr2Ntq1qomm9//3cjs553Jysk+q1pVm9rbLHLRuXPntH//fntbeHi4PShIUkhIiNLT0yVJe/bsUV5entq2bWtax44dO7Rv3z6Hx0t/3Nzuz+sAAOB6ICwAKDOqVa8qi8Wifb+mXPNYbu6Ov94sFouMgoJrHvfPcnLOKig4UJ8vi7e3BZSrKUny8/Ozt7m7uxeqpeD/1/J3d7HOzs5Wo0aN9NFHHxVaFhhovlcEAICSwjkLAMqMCv5+at2umd6b87Fycs4WWp6ZkaWatavpyOFUHTl8zN6evGefMjOyVKt29RKrpebtEbp48aJ2/PizvW3frynKzMiyz9drEKX0tBNyc3NTtephqlY9TDVq1FCNGjVUsWLFq1tPzZry8vK67GVhGzZsqL1796pSpUr2sS9NNpvt2p4kAAB/g7AAoEyZ9vLTys/PV6eWvfTl4m/0277f9esv+/X2Gx/qvjaPqGXraEXWqakhj47VT9t364dtP2no4P+oafPGatCwbonVUeP2CLVp31yjh09W0taftOPHnzUydoK8vDztfVq2jtZdd9dXv17DtCZxgw7+fkQbN27UuHHjtG3btqtaj6enp8aOHasxY8bo/fff1/79+7V582a9++67kqRHHnlEFStWVLdu3bR+/XqlpKRo7dq1Gj58uA4fPlxizxcAADOEBQBlSnhEqBI3LFSzFndr0n9eVIu7u6nn/YO0ft1mTZ85XhaLRe9/8pr8Kvjq/o599WDXxxQWXkVz5r5U4rW88uZzCg6ppO6d+mnA/zypPo/2VMVAf/tyi8Wijz97U9HN7tKT/3xa0Q3uU69evfT777/br2Z0NcaPH6+nnnpKEyZMUGRkpB5++GH7OQ3lypXTt99+q6pVq6pHjx6KjIzUwIEDlZubK19f378ZGQCAa2MxjKs5pfDGlZWVJZvNpszMzDL9h7U0boZ1sxvUMMnZJdzwSvOmbFVCK8ndevN9X1EWbsqWm5urlJQURUREyNPT02HZjfI7EQBQNtx8f6kBAAAAlAjCAgAAAABThAUAAAAApggLAAAAAEwRFgAAAACYIiwAAAAAMEVYAAAAAGCKsAAAAADAFGEBAAAAgCnCAgAAAABTbs4uAABKQuCL40ptXcf/9XyRHzN16lR99tln+uWXX+Tl5aWmTZvqhRdeUK1ata5DhQAAlAz2LABAKVi3bp1iY2O1efNmrVy5UhcuXFCHDh2Uk5Pj7NIAALgs9iwAQClYvny5w3xCQoIqVaqkpKQktWjRwklVAQBwZexZAAAnyMzMlCT5+/s7uRIAAC7PqWFh0qRJslgsDlPt2rXty3NzcxUbG6uAgAD5+PgoJiZGaWlpTqwYAK5dQUGBRowYoWbNmqlu3brOLgcAgMty+p6FOnXq6NixY/bpu+++sy8bOXKkvvzySy1YsEDr1q3T0aNH1aNHDydWCwDXLjY2Vrt27dL8+fOdXQoAAFfk9HMW3NzcFBwcXKg9MzNT7777rubNm6c2bdpIkuLj4xUZGanNmzfrnnvuKe1SAeCaDR06VEuXLtW3336rKlWqOLscAACuyOl7Fvbu3avKlSurWrVqeuSRR3Tw4EFJUlJSki5cuKB27drZ+9auXVtVq1bVpk2bLjteXl6esrKyHCYAcDbDMDR06FB9/vnnWr16tSIiIpxdEgAAf8upYaFJkyZKSEjQ8uXLNXv2bKWkpOjee+/VmTNnlJqaKg8PD/n5+Tk8JigoSKmpqZcdc+rUqbLZbPYpNDT0Oj8LAPh7sbGx+vDDDzVv3jyVL19eqampSk1N1blz55xdGgAAl+XUw5A6d+5s/7levXpq0qSJwsLC9Omnn8rLy6tYY8bFxWnUqFH2+aysLAIDcAsozo3SStPs2bMlSa1atXJoj4+PV//+/Uu/IAAAroLTz1n4Mz8/P91+++3at2+f2rdvr/PnzysjI8Nh70JaWprpOQ6XWK1WWa3WUqgWAK6eYRjOLgEAgCJz+jkLf5adna39+/crJCREjRo1kru7uxITE+3Lk5OTdfDgQUVHRzuxSgAAAODW4NQ9C6NHj1bXrl0VFhamo0ePauLEiXJ1dVXv3r1ls9k0cOBAjRo1Sv7+/vL19dWwYcMUHR3NlZAAAACAUuDUsHD48GH17t1bJ0+eVGBgoJo3b67NmzcrMDBQkjRjxgy5uLgoJiZGeXl56tixo9544w1nlgwAAADcMpwaFv7uhkSenp6aNWuWZs2aVUoVAQAAALikTJ2zAAAAAKDsICwAAAAAMEVYAAAAAGCKsAAAAADAFGEBAAAAgCnCAgAAAABTTr10KgCUlMXJfUptXd1rfVDkx8yePVuzZ8/WgQMHJEl16tTRhAkT1Llz5xKuDgCAksOeBQAoBVWqVNG0adOUlJSkbdu2qU2bNurWrZt+/vlnZ5cGAMBlsWcBAEpB165dHeaff/55zZ49W5s3b1adOnWcVBUAAFdGWACAUpafn68FCxYoJydH0dHRzi4HAIDLIiwAQCnZuXOnoqOjlZubKx8fH33++eeKiopydlkAAFwW5ywAQCmpVauWtm/fru+//15DhgxRv379tHv3bmeXBQDAZbFnAQBKiYeHh2rUqCFJatSokbZu3apXXnlFb731lpMrAwDAHHsWAMBJCgoKlJeX5+wyAAC4LPYsAEApiIuLU+fOnVW1alWdOXNG8+bN09q1a7VixQpnlwYAwGURFgDcFIpzo7TSlJ6err59++rYsWOy2WyqV6+eVqxYofbt2zu7NAAALouwAACl4N1333V2CQAAFBnnLAAAAAAwRVgAAAAAYIqwAAAAAMAUYQEAAACAKcICgFJWIMmQYTi7jpuXwcYFAJQQwgKAUnVBWSoouKjzeQXOLuWmdfbsWUmSu7u7kysBANzouHQqgFJVYMnV8YLv5HaivSQ/eVhdZLE4u6qSk+ua67R1G4ahs2fPKj09XX5+fnJ1dXVaLQCAmwNhAUCpS9Uy6YJ0Ma25XFzcJN08aSHTw/n/oPv5+Sk4ONjZZQAAbgKEBQClz2IoVV8rvSBR7gU23UxHRD5Ua5FT1+/u7s4eBQBAiSEsAHCaAkue8pTu7DJKlKenp7NLAACgxNw8X+cBAAAAKFGEBQAAAACmykxYmDZtmiwWi0aMGGFvy83NVWxsrAICAuTj46OYmBilpaU5r0gAAADgFlImwsLWrVv11ltvqV69eg7tI0eO1JdffqkFCxZo3bp1Onr0qHr06OGkKgEAAIBbi9PDQnZ2th555BG9/fbbqlChgr09MzNT7777rl5++WW1adNGjRo1Unx8vDZu3KjNmzdfdry8vDxlZWU5TAAAAACKzulhITY2Vv/4xz/Url07h/akpCRduHDBob127dqqWrWqNm3adNnxpk6dKpvNZp9CQ0OvW+0AAADAzcypYWH+/Pn64YcfNHXq1ELLUlNT5eHhIT8/P4f2oKAgpaamXnbMuLg4ZWZm2qdDhw6VdNkAAADALcFp91k4dOiQnnzySa1cubJEr0tutVpltVpLbDwAAADgVuW0PQtJSUlKT09Xw4YN5ebmJjc3N61bt06vvvqq3NzcFBQUpPPnzysjI8PhcWlpaQoODnZO0QAAAMAtxGl7Ftq2baudO3c6tA0YMEC1a9fW2LFjFRoaKnd3dyUmJiomJkaSlJycrIMHDyo6OtoZJQMAAAC3FKeFhfLly6tu3boObd7e3goICLC3Dxw4UKNGjZK/v798fX01bNgwRUdH65577nFGyQAAAMAtxWlh4WrMmDFDLi4uiomJUV5enjp27Kg33njD2WUBAAAAt4QyFRbWrl3rMO/p6alZs2Zp1qxZzikIAAAAuIU5/T4LAAAAAMomwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMFSssVKtWTSdPnizUnpGRoWrVql1zUQAAAACcr1hh4cCBA8rPzy/UnpeXpyNHjlxzUQAAAACcz60onZcsWWL/ecWKFbLZbPb5/Px8JSYmKjw8vMSKAwAAAOA8RQoL3bt3lyRZLBb169fPYZm7u7vCw8P13//+t8SKAwAAAOA8RQoLBQUFkqSIiAht3bpVFStWvC5FAQAAAHC+IoWFS1JSUkq6DgAAAABlTLHCgiQlJiYqMTFR6enp9j0Ol7z33nvXXBgAAAAA5ypWWJg8ebKeeeYZ3XXXXQoJCZHFYinpugAAAAA4WbHCwptvvqmEhAT16dOnpOsBAAAAUEYU6z4L58+fV9OmTUu6FgAAAABlSLHCwmOPPaZ58+aVdC0AAAAAypBiHYaUm5urOXPmaNWqVapXr57c3d0dlr/88sslUhwAAAAA5ylWWPjpp5/UoEEDSdKuXbsclnGyMwAAAHBzKFZYWLNmTUnXAQAAAKCMKdY5CwAAAABufsXas9C6desrHm60evXqYhcEAAAAoGwo1p6FBg0aqH79+vYpKipK58+f1w8//KA77rjjqseZPXu26tWrJ19fX/n6+io6OlrLli2zL8/NzVVsbKwCAgLk4+OjmJgYpaWlFadkAAAAAEVUrD0LM2bMMG2fNGmSsrOzr3qcKlWqaNq0aapZs6YMw9DcuXPVrVs3/fjjj6pTp45Gjhypr776SgsWLJDNZtPQoUPVo0cPbdiwoThlAwAAACgCi2EYRkkNtm/fPt199906depUscfw9/fXiy++qAcffFCBgYGaN2+eHnzwQUnSL7/8osjISG3atEn33HOP6ePz8vKUl5dnn8/KylJoaKgyMzPl6+tb7Lqut7d/aOTsEm54gxomObuEGx7vw2tX1t+HWVlZstlsZf53IgCgbCjRE5w3bdokT0/PYj02Pz9f8+fPV05OjqKjo5WUlKQLFy6oXbt29j61a9dW1apVtWnTpsuOM3XqVNlsNvsUGhparHoAAACAW12xDkPq0aOHw7xhGDp27Ji2bdum8ePHF2msnTt3Kjo6Wrm5ufLx8dHnn3+uqKgobd++XR4eHvLz83PoHxQUpNTU1MuOFxcXp1GjRtnnL+1ZAAAAAFA0xQoLNpvNYd7FxUW1atXSM888ow4dOhRprFq1amn79u3KzMzUwoUL1a9fP61bt644ZUmSrFarrFZrsR8PAAAA4A/FCgvx8fElVoCHh4dq1KghSWrUqJG2bt2qV155RQ8//LDOnz+vjIwMh70LaWlpCg4OLrH1AwAAADBXrLBwSVJSkvbs2SNJqlOnju68885rLqigoEB5eXlq1KiR3N3dlZiYqJiYGElScnKyDh48qOjo6GteDwAAAIArK1ZYSE9PV69evbR27Vr7t/4ZGRlq3bq15s+fr8DAwKsaJy4uTp07d1bVqlV15swZzZs3T2vXrtWKFStks9k0cOBAjRo1Sv7+/vL19dWwYcMUHR192SshAQAAACg5xboa0rBhw3TmzBn9/PPPOnXqlE6dOqVdu3YpKytLw4cPv+px0tPT1bdvX9WqVUtt27bV1q1btWLFCrVv317SH/dz6NKli2JiYtSiRQsFBwfrs88+K07JAAAAAIqoWPdZsNlsWrVqlRo3buzQvmXLFnXo0EEZGRklVd81u1GuKc717a9dWb++/Y2A9+G1K+vvwxvldyIAoGwo1p6FgoICubu7F2p3d3dXQUHBNRcFAAAAwPmKFRbatGmjJ598UkePHrW3HTlyRCNHjlTbtm1LrDgAAAAAzlOssPD6668rKytL4eHhql69uqpXr66IiAhlZWXptddeK+kaAQAAADhBsa6GFBoaqh9++EGrVq3SL7/8IkmKjIxUu3btSrQ4AAAAAM5TpD0Lq1evVlRUlLKysmSxWNS+fXsNGzZMw4YNU+PGjVWnTh2tX7/+etUKAAAAoBQVKSzMnDlTgwYNMr2Chs1m0+OPP66XX365xIoDAAAA4DxFCgs7duxQp06dLru8Q4cOSkoq25cNBAAAAHB1ihQW0tLSTC+Zeombm5uOHz9+zUUBAAAAcL4ihYXbbrtNu3btuuzyn376SSEhIddcFAAAAADnK1JYuO+++zR+/Hjl5uYWWnbu3DlNnDhRXbp0KbHiAAAAADhPkS6d+vTTT+uzzz7T7bffrqFDh6pWrVqSpF9++UWzZs1Sfn6+xo0bd10KBQAAAFC6ihQWgoKCtHHjRg0ZMkRxcXEyDEOSZLFY1LFjR82aNUtBQUHXpVAAAAAApavIN2ULCwvT119/rdOnT2vfvn0yDEM1a9ZUhQoVrkd9AAAAAJykWHdwlqQKFSqocePGJVkLAAAAgDKkSCc4AwAAALh1EBYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApp4aFqVOnqnHjxipfvrwqVaqk7t27Kzk52aFPbm6uYmNjFRAQIB8fH8XExCgtLc1JFQMAAAC3DqeGhXXr1ik2NlabN2/WypUrdeHCBXXo0EE5OTn2PiNHjtSXX36pBQsWaN26dTp69Kh69OjhxKoBAACAW4ObM1e+fPlyh/mEhARVqlRJSUlJatGihTIzM/Xuu+9q3rx5atOmjSQpPj5ekZGR2rx5s+655x5nlA0AAADcEsrUOQuZmZmSJH9/f0lSUlKSLly4oHbt2tn71K5dW1WrVtWmTZtMx8jLy1NWVpbDBAAAAKDoykxYKCgo0IgRI9SsWTPVrVtXkpSamioPDw/5+fk59A0KClJqaqrpOFOnTpXNZrNPoaGh17t0AAAA4KZUZsJCbGysdu3apfnz51/TOHFxccrMzLRPhw4dKqEKAQAAgFuLU89ZuGTo0KFaunSpvv32W1WpUsXeHhwcrPPnzysjI8Nh70JaWpqCg4NNx7JarbJarde7ZAAAAOCm59Q9C4ZhaOjQofr888+1evVqRUREOCxv1KiR3N3dlZiYaG9LTk7WwYMHFR0dXdrlAgAAALcUp+5ZiI2N1bx58/TFF1+ofPny9vMQbDabvLy8ZLPZNHDgQI0aNUr+/v7y9fXVsGHDFB0dzZWQAAAAgOvMqWFh9uzZkqRWrVo5tMfHx6t///6SpBkzZsjFxUUxMTHKy8tTx44d9cYbb5RypQAAAMCtx6lhwTCMv+3j6empWbNmadasWaVQEQAAAIBLyszVkAAAAACULYQFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADDl1LDw7bffqmvXrqpcubIsFosWL17ssNwwDE2YMEEhISHy8vJSu3bttHfvXucUCwAAANxinBoWcnJyVL9+fc2aNct0+fTp0/Xqq6/qzTff1Pfffy9vb2917NhRubm5pVwpAAAAcOtxc+bKO3furM6dO5suMwxDM2fO1NNPP61u3bpJkt5//30FBQVp8eLF6tWrl+nj8vLylJeXZ5/Pysoq+cIBAACAW0CZPWchJSVFqampateunb3NZrOpSZMm2rRp02UfN3XqVNlsNvsUGhpaGuUCAAAAN50yGxZSU1MlSUFBQQ7tQUFB9mVm4uLilJmZaZ8OHTp0XesEAAAAblZOPQzperBarbJarc4uAwAAALjhldk9C8HBwZKktLQ0h/a0tDT7MgAAAADXT5kNCxEREQoODlZiYqK9LSsrS99//72io6OdWBkAAABwa3DqYUjZ2dnat2+ffT4lJUXbt2+Xv7+/qlatqhEjRui5555TzZo1FRERofHjx6ty5crq3r2784oGAAAAbhFODQvbtm1T69at7fOjRo2SJPXr108JCQkaM2aMcnJyNHjwYGVkZKh58+Zavny5PD09nVUyAAAAcMtwalho1aqVDMO47HKLxaJnnnlGzzzzTClWBQAAAEAqw+csAAAAAHAuwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMERYAAAAAmCIsAAAAADBFWAAAAABgirAAAAAAwBRhAQAAAIApwgIAAAAAU4QFAAAAAKYICwAAAABMuTm7APxh0JKqzi7hxtfQ2QXc+HgflgDehwCAmwh7FgAAAACYIiwAAAAAMEVYAAAAAGCKsAAAAADAFGEBAAAAgCnCAgAAAABThAUAAAAApm6IsDBr1iyFh4fL09NTTZo00ZYtW5xdEgAAAHDTK/Nh4ZNPPtGoUaM0ceJE/fDDD6pfv746duyo9PR0Z5cGAAAA3NTK/B2cX375ZQ0aNEgDBgyQJL355pv66quv9N577+nf//53of55eXnKy8uzz2dmZkqSsrKySqfg4sq74OwKbnxl/TW+EfA+vHZl/H146XehYRhOrgQAcCOwGGX4L8b58+dVrlw5LVy4UN27d7e39+vXTxkZGfriiy8KPWbSpEmaPHlyKVYJADeeQ4cOqUqVKs4uAwBQxpXpPQsnTpxQfn6+goKCHNqDgoL0yy+/mD4mLi5Oo0aNss8XFBTo1KlTCggIkMViua713qyysrIUGhqqQ4cOydfX19nl4BbGe/HaGYahM2fOqHLlys4uBQBwAyjTYaE4rFarrFarQ5ufn59zirnJ+Pr68g8aygTei9fGZrM5uwQAwA2iTJ/gXLFiRbm6uiotLc2hPS0tTcHBwU6qCgAAALg1lOmw4OHhoUaNGikxMdHeVlBQoMTEREVHRzuxMgAAAODmV+YPQxo1apT69eunu+66S3fffbdmzpypnJwc+9WRcP1ZrVZNnDix0OFdQGnjvQgAQOkq01dDuuT111/Xiy++qNTUVDVo0ECvvvqqmjRp4uyyAAAAgJvaDREWAAAAAJS+Mn3OAgAAAADnISwAAAAAMEVYAAAAAGCKsIArCg8P18yZM6/L2BaLRYsXL3b6GPh7CQkJN/zNDVu1aqURI0Y4fQwAAG4khIUS0r9/f1ksFlksFrm7uysiIkJjxoxRbm6us0u7Kpf7Z3Dr1q0aPHhwsccNDw+3b5dLU5UqVSRJx44dU+fOna9qnEmTJqlBgwaF2osyxq3s+PHjGjJkiKpWrSqr1arg4GB17NhRGzZsuKrHP/zww/r111+vc5XX7s+fwz9P+/bt02effaZnn332qsZZu3atLBaLMjIyHNqLMgYAADeDMn+fhRtJp06dFB8frwsXLigpKUn9+vWTxWLRCy+84OzSii0wMPCax3jmmWc0aNAg+7yrq6sklchduLmT99WJiYnR+fPnNXfuXFWrVk1paWlKTEzUyZMnr+rxXl5e8vLyus5VloxLn8M/CwwMtL/vroW/v/81jwEAwI2EPQsl6NI3tqGhoerevbvatWunlStX2pfn5eVp+PDhqlSpkjw9PdW8eXNt3brVvvzSt5krVqzQnXfeKS8vL7Vp00bp6elatmyZIiMj5evrq//5n//R2bNn7Y9bvny5mjdvLj8/PwUEBKhLly7av3+/ffmBAwdksVj02WefqXXr1ipXrpzq16+vTZs22dc7YMAAZWZm2r+JnTRpkqTChyFlZGTo8ccfV1BQkDw9PVW3bl0tXbr0itulfPnyCg4Otk+XAshfDyE6fPiwevfuLX9/f3l7e+uuu+7S999/r4SEBE2ePFk7duyw15eQkGA6xs6dO9WmTRt5eXkpICBAgwcPVnZ2tn15//791b17d7300ksKCQlRQECAYmNjdeHChSu/uDewjIwMrV+/Xi+88IJat26tsLAw3X333YqLi9P999/v0O9yr63ZnqcvvvhCDRs2lKenp6pVq6bJkyfr4sWL9uUWi0XvvPOOHnjgAZUrV041a9bUkiVLHMb4+eef1aVLF/n6+qp8+fK69957Hd6777zzjiIjI+Xp6anatWvrjTfe+Nvne+lz+OfJ1dW10CFEeXl5Gjt2rEJDQ2W1WlWjRg29++67OnDggFq3bi1JqlChgiwWi/r37y+p8GFIp0+fVt++fVWhQgWVK1dOnTt31t69e+3LL223FStWKDIyUj4+PurUqZOOHTv2t88DAICygLBwnezatUsbN26Uh4eHvW3MmDFatGiR5s6dqx9++EE1atRQx44dderUKYfHTpo0Sa+//ro2btyoQ4cO6aGHHtLMmTM1b948ffXVV/rmm2/02muv2fvn5ORo1KhR2rZtmxITE+Xi4qIHHnhABQUFDuOOGzdOo0eP1vbt23X77berd+/eunjxopo2baqZM2fK19dXx44d07FjxzR69OhCz6mgoECdO3fWhg0b9OGHH2r37t2aNm1aiXxjm52drZYtW+rIkSNasmSJduzYoTFjxqigoEAPP/ywnnrqKdWpU8de38MPP1xojJycHHXs2FEVKlTQ1q1btWDBAq1atUpDhw516LdmzRrt379fa9as0dy5c5WQkGAPHzcjHx8f+fj4aPHixcrLyzPtU9TXdv369erbt6+efPJJ7d69W2+99ZYSEhL0/PPPO/SbPHmyHnroIf3000+677779Mgjj9jf70eOHFGLFi1ktVq1evVqJSUl6dFHH7UHjo8++kgTJkzQ888/rz179mjKlCkaP3685s6dWyLbpW/fvvr444/16quvas+ePXrrrbfk4+Oj0NBQLVq0SJKUnJysY8eO6ZVXXjEdo3///tq2bZuWLFmiTZs2yTAM3XfffQ7h8+zZs3rppZf0wQcf6Ntvv9XBgwdNP18AAJRJBkpEv379DFdXV8Pb29uwWq2GJMPFxcVYuHChYRiGkZ2dbbi7uxsfffSR/THnz583KleubEyfPt0wDMNYs2aNIclYtWqVvc/UqVMNScb+/fvtbY8//rjRsWPHy9Zy/PhxQ5Kxc+dOwzAMIyUlxZBkvPPOO/Y+P//8syHJ2LNnj2EYhhEfH2/YbLZCY4WFhRkzZswwDMMwVqxYYbi4uBjJyclXvV3CwsIMDw8Pw9vb2z698sorhmEYhiTj888/NwzDMN566y2jfPnyxsmTJ03HmThxolG/fv1C7X8eY86cOUaFChWM7Oxs+/KvvvrKcHFxMVJTUw3D+ON1CgsLMy5evGjv07NnT+Phhx++6ud0I1q4cKFRoUIFw9PT02jatKkRFxdn7Nixw778717bv74/2rZta0yZMsWhzwcffGCEhITY5yUZTz/9tH0+OzvbkGQsW7bMMAzDiIuLMyIiIozz58+brrN69erGvHnzHNqeffZZIzo6+rLP88+fw0vTgw8+aBiGYbRs2dJ48sknDcMwjOTkZEOSsXLlStNxLn0WT58+7dD+5zF+/fVXQ5KxYcMG+/ITJ04YXl5exqeffmoYxh/bTZKxb98+e59Zs2YZQUFBl30OAACUJZyzUIJat26t2bNnKycnRzNmzJCbm5tiYmIkSfv379eFCxfUrFkze393d3fdfffd2rNnj8M49erVs/8cFBSkcuXKqVq1ag5tW7Zssc/v3btXEyZM0Pfff68TJ07Y9ygcPHhQdevWNR03JCREkpSenq7atWtf1fPbvn27qlSpottvv/2q+l/yr3/9y34YhyRVrFjRdOw777zzmo4J37Nnj+rXry9vb297W7NmzVRQUKDk5GQFBQVJkurUqePwjXlISIh27txZ7PXeCGJiYvSPf/xD69ev1+bNm7Vs2TJNnz5d77zzjvr371/k13bHjh3asGGDw56E/Px85ebm6uzZsypXrpwkx/ect7e3fH19lZ6eLumP1/zee++Vu7t7ofFzcnK0f/9+DRw40OF8l4sXL8pms12xtkufwz+v96+2b98uV1dXtWzZ8qqer5k9e/bIzc1NTZo0sbcFBASoVq1aDp/pcuXKqXr16vb5kJAQ+zYAAKCsIyyUIG9vb9WoUUOS9N5776l+/fp69913NXDgwCKN8+d/ni5dXenPLBaLwyFGXbt2VVhYmN5++21VrlxZBQUFqlu3rs6fP3/FcSUVOlTpSop7gmvFihXt26Wkxy6Ov9ueNytPT0+1b99e7du31/jx4/XYY49p4sSJ6t+/f5G3f3Z2tiZPnqwePXqYrueSK23rK63z0nkmb7/9tsM/45L+9rC3P38OL8fZ7zfDMEpt/QAAXAvOWbhOXFxc9J///EdPP/20zp07p+rVq8vDw8PhUpUXLlzQ1q1bFRUVVez1nDx5UsnJyXr66afVtm1bRUZG6vTp00Uex8PDQ/n5+VfsU69ePR0+fPi6XEKzXr162r59e6HzN4pSX2RkpHbs2KGcnBx724YNG+Ti4qJatWqVaL03g6ioKPu2Kupr27BhQyUnJ6tGjRqFJheXq/u1Uq9ePa1fv9705PKgoCBVrlxZv/32W6HxIyIirv5JXsYdd9yhgoICrVu3znT5pXONrvSei4yM1MWLF/X999/b2y59Hq/lMw0AQFlCWLiOevbsKVdXV82aNUve3t4aMmSI/vWvf2n58uXavXu3Bg0apLNnzxZ5z8OfVahQQQEBAZozZ4727dun1atXa9SoUUUeJzw8XNnZ2UpMTNSJEyccrrZ0ScuWLdWiRQvFxMRo5cqVSklJ0bJly7R8+fJi139J7969FRwcrO7du2vDhg367bfftGjRIvsVm8LDw5WSkqLt27frxIkTpifqPvLII/L09FS/fv20a9curVmzRsOGDVOfPn3shyDdik6ePKk2bdroww8/1E8//aSUlBQtWLBA06dPV7du3SQV/bWdMGGC3n//fU2ePFk///yz9uzZo/nz5+vpp5++6rqGDh2qrKws9erVS9u2bdPevXv1wQcfKDk5WdIfJ0dPnTpVr776qn799Vft3LlT8fHxevnll695m4SHh6tfv3569NFHtXjxYqWkpGjt2rX69NNPJUlhYWGyWCxaunSpjh8/7nBFrUtq1qypbt26adCgQfruu++0Y8cO/e///q9uu+02+3YFAOBGR1i4jtzc3DR06FBNnz5dOTk5mjZtmmJiYtSnTx81bNhQ+/bt04oVK1ShQoVir8PFxUXz589XUlKS6tatq5EjR+rFF18s8jhNmzbVP//5Tz388MMKDAzU9OnTTfstWrRIjRs3Vu/evRUVFaUxY8b87Tf+V8PDw0PffPONKlWqpPvuu0933HGHw9V4YmJi1KlTJ7Vu3VqBgYH6+OOPC41Rrlw5rVixQqdOnVLjxo314IMPqm3btnr99devub4bmY+Pj5o0aaIZM2aoRYsWqlu3rsaPH69BgwY5bJuivLYdO3bU0qVL9c0336hx48a65557NGPGDIWFhV11XQEBAVq9erX9SliNGjXS22+/bT9s57HHHtM777yj+Ph43XHHHWrZsqUSEhJKZM+CJM2ePVsPPvignnjiCdWuXVuDBg2y72m57bbbNHnyZP373/9WUFBQoStqXRIfH69GjRqpS5cuio6OlmEY+vrrr03PwwAA4EZkMTh4FgAAAIAJ9iwAAAAAMEVYAAAAAGCKsAAAAADAFGEBAAAAgCnCAgAAAABThAUAAAAApggLAAAAAEwRFgAAAACYIiwAAAAAMEVYAAAAAGCKsAAAAADAFGEBAAAAgCnCAgAAAABThAUAAAAApggLAAAAAEwRFgAAAACYIiwAAAAAMEVYAP5GamqqnnzySdWoUUOenp4KCgpSs2bNNHv2bJ09e9bZ5QEAAFw3bs4uACjLfvvtNzVr1kx+fn6aMmWK7rjjDlmtVu3cuVNz5szRbbfdpvvvv/+6rPv8+fPy8PC4LmMDAABcDfYsAFfwxBNPyM3NTdu2bdNDDz2kyMhIVatWTd26ddNXX32lrl27SpIyMjL02GOPKTAwUL6+vmrTpo127NhhH2fSpElq0KCBPvjgA4WHh8tms6lXr146c+aMvU+rVq00dOhQjRgxQhUrVlTHjh0lSbt27VLnzp3l4+OjoKAg9enTRydOnCjdDQEAAG5JhAXgMk6ePKlvvvlGsbGx8vb2Nu1jsVgkST179lR6erqWLVumpKQkNWzYUG3bttWpU6fsfffv36/Fixdr6dKlWrp0qdatW6dp06Y5jDd37lx5eHhow4YNevPNN5WRkaE2bdrozjvv1LZt27R8+XKlpaXpoYceun5PHAAA4P/jMCTgMvbt2yfDMFSrVi2H9ooVKyo3N1eSFBsbq65du2rLli1KT0+X1WqVJL300ktavHixFi5cqMGDB0uSCgoKlJCQoPLly0uS+vTpo8TERD3//PP2sWvWrKnp06fb55977jndeeedmjJlir3tvffeU2hoqH799Vfdfvvt1+fJAwAAiLAAFNmWLVtUUFCgRx55RHl5edqxY4eys7MVEBDg0O/cuXPav3+/fT48PNweFCQpJCRE6enpDo9p1KiRw/yOHTu0Zs0a+fj4FKpj//79hAUAAHBdERaAy6hRo4YsFouSk5Md2qtVqyZJ8vLykiRlZ2crJCREa9euLTSGn5+f/Wd3d3eHZRaLRQUFBQ5tfz3cKTs7W127dtULL7xQaOyQkJCrfi4AAADFQVgALiMgIEDt27fX66+/rmHDhl32vIWGDRsqNTVVbm5uCg8PL9EaGjZsqEWLFik8PFxubnxcAQBA6eIEZ+AK3njjDV28eFF33XWXPvnkE+3Zs0fJycn68MMP9csvv8jV1VXt2rVTdHS0unfvrm+++UYHDhzQxo0bNW7cOG3btu2a1h8bG6tTp06pd+/e2rp1q/bv368VK1ZowIABys/PL6FnCQAAYI6vKoErqF69un788UdNmTJFcXFxOnz4sKxWq6KiojR69Gg98cQTslgs+vrrrzVu3DgNGDBAx48fV3BwsFq0aKGgoKBrWn/lypW1YcMGjR07Vh06dFBeXp7CwsLUqVMnubiQ9QEAwPVlMQzDcHYRAAAAAMoevpoEAAAAYIqwAAAAAMAUYQEAAACAKcICAAAAAFOEBQAAAACmCAsAAAAATBEWAAAAAJgiLAAAAAAwRVgAAAAAYIqwAAAAAMAUYQEAAACAqf8HvO5tPiPrZGAAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x1000 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "# Map confidence levels to ordinal values\n",
    "confidence_map = {\n",
    "    1: 'Not confident',\n",
    "    2: 'Somewhat confident',\n",
    "    3: 'Confident'\n",
    "}\n",
    "\n",
    "# Per annotator\n",
    "genre_conf_per_annotator = {}\n",
    "for annotator in ['annotator-1', 'annotator-2', 'annotator-3', 'annotator-4', 'annotator-llm']:\n",
    "    scifi_confidences = []\n",
    "    romantic_confidences = []\n",
    "    for sample in dataset['samples']:\n",
    "        if sample['genre'] == 'SCI':\n",
    "            scifi_confidences.append(sample['annotations']['confidence'][annotator])\n",
    "        if sample['genre'] == 'ROM':\n",
    "            romantic_confidences.append(sample['annotations']['confidence'][annotator])\n",
    "\n",
    "    genre_df = pd.DataFrame({\n",
    "        'Genre': ['Science Fiction'] * len(scifi_confidences) + ['Romantic Fiction'] * len(romantic_confidences),\n",
    "        'Confidence': scifi_confidences + romantic_confidences\n",
    "    })\n",
    "\n",
    "    key = genre_df['Confidence'].map(confidence_map)\n",
    "\n",
    "    genre_df.iloc[key.argsort()]\n",
    "    \n",
    "    genre_conf_per_annotator[annotator] = genre_df\n",
    "\n",
    "# Define colors for each confidence level\n",
    "confidence_colors = {\n",
    "    1: 'crimson',\n",
    "    2: 'coral',\n",
    "    3: 'yellowgreen'\n",
    "}\n",
    "\n",
    "# Plot the confidence levels per annotator in a bar chart with specified colors\n",
    "fig, axs = plt.subplots(3, 2, figsize=(8, 10))\n",
    "\n",
    "for i, annotator in enumerate(genre_conf_per_annotator.keys()):\n",
    "    ax = axs[i // 2, i % 2]\n",
    "    # Extract the genre and confidence data for plotting\n",
    "    data_to_plot = genre_conf_per_annotator[annotator].groupby(['Genre', 'Confidence']).size().unstack()\n",
    "    # Map the confidence to the predefined colors\n",
    "    colors = [confidence_colors[conf] for conf in data_to_plot.columns]\n",
    "    data_to_plot.plot(kind='bar', ax=ax, stacked=True, color=colors)\n",
    "    ax.set_title(annotator)\n",
    "    ax.set_ylabel('Count')\n",
    "    ax.set_xlabel('Genre')\n",
    "    ax.legend(title='Confidence')\n",
    "    # Make the x-labels horizontal\n",
    "    for tick in ax.get_xticklabels():\n",
    "        tick.set_rotation(0)\n",
    "    # Increase padding between x-axis and labels\n",
    "    ax.xaxis.labelpad = 10\n",
    "\n",
    "# Increase space between plots\n",
    "plt.tight_layout()\n",
    "\n",
    "# Remove final empty plot if number of annotators is odd\n",
    "if len(genre_conf_per_annotator.keys()) % 2 != 0:\n",
    "    fig.delaxes(axs[-1, -1])\n",
    "\n",
    "plt.show()\n",
    "\n",
    "# Save the plot\n",
    "fig.savefig('confidence_per_annotator.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Huh, it seems that the LLM Annotator is generally very confident, making the actual difference in confidence levels between genre appear even smaller than it actually is between just the human annotators. Let's redraw the first graph but specify that it's just the human annotators."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 800x600 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAG9CAYAAADa0EiEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABg2UlEQVR4nO3dd1gU1/s28HtpS0dQqiIiGFCxIBrEhgXFLok9fhViS8GKxgQTC2rEqIm9xBIxltjFaCL23iHBEisGNRIQGyCoIOx5/8jL/Fx3UQYWQb0/17WXzpkzZ56Z3RmePXNmViGEECAiIiKiQtMr7QCIiIiI3jRMoIiIiIhkYgJFREREJBMTKCIiIiKZmEARERERycQEioiIiEgmJlBEREREMjGBIiIiIpKJCRQRERGRTEyg6J10584ddOvWDeXLl4dCocDs2bNx8OBBKBQKHDx48JXLN2/eHM2bNy/xOF8HlUoFLy8vfPvtt6Wy/piYGJibm+Pu3bulsn76P2fOnEGjRo1gZmYGhUKB+Pj4EltXVFQUFAoFYmNjS2wdRCWJCdRrsHDhQigUCvj6+pZ2KPT/jRw5Ert27UJ4eDhWrVqFtm3blnZIpeaXX37BP//8gyFDhkhlmZmZmDBhAtq2bQsbGxsoFApERUXJanfq1Klo2LAhbG1tYWxsjGrVqmHEiBEaiVLbtm3h7u6OyMhIXWwOFdGzZ8/QvXt3PHjwALNmzcKqVavg4uJS2mFRKZo6dSqio6OLvPzFixcxceJE3LhxQ2cxlSUGpR3Au2DNmjWoUqUKTp8+jYSEBLi7u5d2SO+8/fv3o0uXLhg9erRU9t577+HJkycwMjIqxchevxkzZqBXr16wsrKSyu7du4dJkyahcuXKqFOnTqF65V4UFxeHunXrolevXrCwsMClS5ewdOlS/Pbbb4iPj4eZmZlU95NPPsHo0aMREREBCwsLXWwWyXT9+nXcvHkTS5cuxcCBA0s7HCoDpk6dim7duiEoKKhIy1+8eBERERFo3rw5qlSpotPYygL2QJWwxMREHD9+HD/88ANsbW2xZs2a1x6DSqXC06dPX/t6y7LU1FSUK1dOrUxPTw/GxsbQ03t3Dos///wTZ8+eRY8ePdTKHR0dkZycjJs3b2LGjBlFanvz5s1YvHgxRowYgQEDBmDmzJn46aefkJCQgO3bt6vV7dq1K7Kzs7Fx48Yib4suCSHw5MmT0g6j0LKysordRmpqKgBoHBfFoYu4iF70+PHj0g4BABOoErdmzRpYW1ujQ4cO6Natm1oC9ezZM9jY2ODjjz/WWC4jIwPGxsZqPSTZ2dmYMGEC3N3doVQq4ezsjDFjxiA7O1ttWYVCgSFDhmDNmjWoWbMmlEolYmJiAAAzZ85Eo0aNUL58eZiYmMDHxwebNm3SWP+TJ08wbNgwVKhQARYWFujcuTOSkpKgUCgwceJEtbpJSUno378/7O3toVQqUbNmTfz000+F3kerV6/G+++/D1NTU1hbW6NZs2bYvXu3Wp2FCxdK2+Lk5ITQ0FCkpaWp1WnevDm8vLxw8eJFtGjRAqampqhYsSKmT58u1ckfdyGEwIIFC6BQKKBQKACgwDFQS5YsgZubG0xMTPD+++/jyJEjWrdD7vsTHR0NLy8vaZ/lv0fPS0pKwoABA+Dk5ASlUglXV1d89tlnyMnJkeqkpaVhxIgRcHZ2hlKphLu7O7777juoVKpX7vvo6GgYGRmhWbNmauVKpRIODg6vXF6u/G+hL753dnZ2qF27NrZt2/bKNkJCQmBubo6///4bgYGBMDMzg5OTEyZNmgQhhFpdlUqF2bNno2bNmjA2Noa9vT0++eQTPHz4UCOujh07YteuXahfvz5MTEzw448/vjSOBQsWoGrVqmqfC21j43T9uZg4cSIUCgUuXryIjz76CNbW1mjSpIk0f/Xq1fDx8YGJiQlsbGzQq1cv/PPPP6/cp/7+/gCA7t27Q6FQqG3H/v370bRpU5iZmaFcuXLo0qULLl26JCuugmRnZyMsLAy2trYwMzPDBx98oHGZV9t5B/jvfQsJCZGm84/vo0ePYtiwYbC1tUW5cuXwySefICcnB2lpaejXrx+sra1hbW2NMWPGaHxmCnuOlHMcvygnJwfjx4+Hj48PrKysYGZmhqZNm+LAgQNq9W7cuAGFQoGZM2dK5yGlUokGDRrgzJkzanXzj4ukpCQEBQXB3Nwctra2GD16NPLy8tTqZmVlYdSoUdI5w8PDAzNnzlTbFwqFAllZWVi5cqV0nszf1zdv3sTnn38ODw8PmJiYoHz58ujevbvapbqoqCh0794dANCiRQupjefPr3LO63FxcWjWrBlMTU0xduxYAEBsbCwCAwNRoUIFmJiYwNXVFf3793/l/tcZQSXK09NTDBgwQAghxOHDhwUAcfr0aWl+//79Rbly5UR2drbacitXrhQAxJkzZ4QQQuTl5Yk2bdoIU1NTMWLECPHjjz+KIUOGCAMDA9GlSxe1ZQGI6tWrC1tbWxERESEWLFgg/vzzTyGEEJUqVRKff/65mD9/vvjhhx/E+++/LwCIHTt2qLXRo0cPAUD07dtXLFiwQPTo0UPUqVNHABATJkyQ6qWkpIhKlSoJZ2dnMWnSJLFo0SLRuXNnAUDMmjXrlftn4sSJAoBo1KiRmDFjhpgzZ4746KOPxJdffinVmTBhggAgAgICxLx588SQIUOEvr6+aNCggcjJyZHq+fv7CycnJ+Hs7CyGDx8uFi5cKFq2bCkAiN9//10IIcT169fFqlWrBADRunVrsWrVKrFq1SohhBAHDhwQAMSBAwekNpctWybFN3fuXDFixAhRrlw5UbVqVeHv7y/Vk/v+1KlTRzg6OorJkyeL2bNni6pVqwpTU1Nx7949qV5SUpJwcnKS2ly8eLEYN26cqF69unj48KEQQoisrCxRu3ZtUb58eTF27FixePFi0a9fP6FQKMTw4cNfuf8DAgJEvXr1XlrnzJkzAoBYsWLFK9t7kUqlEnfv3hXJycni8OHDolGjRkJfX19cunRJo+7AgQNFhQoVXtlmcHCwMDY2FtWqVRN9+/YV8+fPFx07dhQAxLhx4zTaNDAwEIMGDRKLFy8WX375pTAzM9P47Li4uAh3d3dhbW0tvvrqK7F48WK1z8GLFi5cKACIpk2birlz54qwsDBhY2Mj3NzcSvxzkX881KhRQ3Tp0kUsXLhQLFiwQAghxJQpU4RCoRA9e/YUCxcuFBEREaJChQqiSpUq0mdGm+PHj4uxY8cKAGLYsGFi1apVYvfu3UIIIfbs2SMMDAzEe++9J6ZPny61aW1tLRITEwsVlzYrVqwQAIS3t7do2bKlmDdvnhg1apTQ19cXPXr00Ng3z5938rm4uIjg4GCNNuvWrSvatm0rFixYIPr27SsAiDFjxogmTZqIjz76SCxcuFD6zKxcuVKtzcKeIwv7fmlz9+5d4ejoKMLCwsSiRYvE9OnThYeHhzA0NJTO1UIIkZiYKO0jd3d38d1334np06eLChUqiEqVKql9hvOPi5o1a4r+/fuLRYsWia5duwoAYuHChVI9lUolWrZsKRQKhRg4cKCYP3++6NSpkwAgRowYIdVbtWqVUCqVomnTptJ58vjx40IIITZu3Cjq1Kkjxo8fL5YsWSLGjh0rrK2thYuLi8jKyhJC/HeuHTZsmAAgxo4dK7WRkpIihJB3XndwcBC2trZi6NCh4scffxTR0dHizp07wtraWrz33ntixowZYunSpeLrr78W1atXf+m+1yUmUCUoNjZWABB79uwRQvz3wa1UqZLaH7Zdu3YJAGL79u1qy7Zv315UrVpVml61apXQ09MTR44cUau3ePFiAUAcO3ZMKgMg9PT0xF9//aUR0+PHj9Wmc3JyhJeXl2jZsqVUFhcXp3EwCSFESEiIxolswIABwtHRUeOE0atXL2FlZaWxvuddu3ZN6OnpiQ8++EDk5eWpzVOpVEIIIVJTU4WRkZFo06aNWp358+cLAOKnn36Syvz9/QUA8fPPP0tl2dnZwsHBQXTt2lWtfQAiNDRUrezFBConJ0fY2dmJunXrqiW4S5YsEQDU/lDKfX+MjIxEQkKCVHb27FkBQMybN08q69evn9DT05OSaG37Z/LkycLMzExcvXpVbf5XX30l9PX1xa1btzSWfV6lSpU09s2LipNAJScnCwDSq1KlSmL9+vVa606dOlUAEHfu3Hlpm8HBwQKAGDp0qFSmUqlEhw4dhJGRkbh7964QQogjR44IAGLNmjVqy8fExGiUu7i4CAAiJibmlduUnZ0typcvLxo0aCCePXsmlUdFRb2Wz0X+H57evXurtXnjxg2hr68vvv32W7Xy8+fPCwMDA43yF+V//jdu3KhWXrduXWFnZyfu37+vFpeenp7o16/fK+MqSH6yExAQIH2ehRBi5MiRQl9fX6SlpUllchOowMBAtTb9/PyEQqEQn376qVSWm5srKlWqpPZ+CVG4c2R+TIV5v7TJzc3V+NL88OFDYW9vL/r37y+V5SdQ5cuXFw8ePJDKt23bpvF3I/+4mDRpklq73t7ewsfHR5qOjo4WAMSUKVPU6nXr1k0oFAq17TEzM1Pbv/m0nddPnDihcf7duHGjxpdSIYp2Xl+8eLFaG1u3blXrZCgNvIRXgtasWQN7e3u0aNECwH9doj179sS6deukLtWWLVuiQoUKWL9+vbTcw4cPsWfPHvTs2VMq27hxI6pXrw5PT0/cu3dPerVs2RIANLp+/f39UaNGDY2YTExM1NaTnp6Opk2b4o8//pDK87ugP//8c7Vlhw4dqjYthMDmzZvRqVMnCCHU4goMDER6erpauy+Kjo6GSqXC+PHjNcYd5V9W27t3L3JycjBixAi1OoMGDYKlpSV+++03teXMzc3xv//9T5o2MjLC+++/j7///rvAOAoSGxuL1NRUfPrpp2oDy0NCQtQGXAPy35+AgAC4ublJ07Vr14alpaUUp0qlQnR0NDp16oT69etrxJa/fzZu3IimTZvC2tpabb0BAQHIy8vD4cOHX7qN9+/fh7W1tYy9Io+NjQ327NmD7du3Y9KkSahQoQIyMzO11s2P4969e4Vq+/m7BvMvp+Tk5GDv3r0A/ts3VlZWaN26tdq+8fHxgbm5ucZ74urqisDAwFeuNzY2Fvfv38egQYNgYPB/9+H06dNHY1/q+nPxvE8//VRtesuWLVCpVOjRo4fauhwcHFCtWjWNdRVGcnIy4uPjERISAhsbG7W4Wrdujd9///2Vcb3K4MGDpc8zADRt2hR5eXm4efOm7HjzDRgwQK1NX19fCCEwYMAAqUxfXx/169fX2LeFOUfmk/N+PU9fX186p6hUKjx48AC5ubmoX7++1vX07NlT7bPVtGlTACjU56Jp06Zq9X7//Xfo6+tj2LBhavVGjRoFIQR27tz50tgB9X307Nkz3L9/H+7u7ihXrtxLz/n55J7XlUqlxlCX/LF6O3bswLNnz165zpLAu/BKSF5eHtatW4cWLVogMTFRKvf19cX333+Pffv2oU2bNjAwMEDXrl2xdu1aZGdnQ6lUYsuWLXj27JlaAnXt2jVcunQJtra2WteXPwA0n6urq9Z6O3bswJQpUxAfH682BuP5k83Nmzehp6en0caLdw/evXsXaWlpWLJkCZYsWVKouJ53/fp16OnpaU30no8FADw8PNTKjYyMULVqVY2TbKVKldS2BfjvD/O5c+cKXMer1l2tWjW1ckNDQ1StWlWtTO77U7lyZY061tbW0ticu3fvIiMjA15eXi+N8dq1azh37lyh16uNeGEMiBzp6elqg62NjIzU/tAaGRkhICAAANCxY0e0atUKjRs3hp2dHTp27Kg1jhffP2309PQ03oP33nsPAKRxGNeuXUN6ejrs7Oy0tlHYY+ZF+Z+LF48HAwMDjTuNdP25eFm8165dgxBC4/Oaz9DQUGv5yxR0/AFA9erVsWvXLmRlZandUVnY/ZjvxW3OTxS0bXNR28z/wuPs7KxR/uJ6CnOOLGg9QMHv14tWrlyJ77//HpcvX1ZLALTtv8LuI2NjY43P2ovx3Lx5E05OThp3u1avXl2a/ypPnjxBZGQkVqxYgaSkJLVzSHp6+iuXl3ter1ixosbd0f7+/ujatSsiIiIwa9YsNG/eHEFBQfjoo4+gVCpfGYMuMIEqIfv370dycjLWrVuHdevWacxfs2YN2rRpAwDo1asXfvzxR+zcuRNBQUHYsGEDPD09UadOHam+SqVCrVq18MMPP2hd34snhue/IeQ7cuQIOnfujGbNmmHhwoVwdHSEoaEhVqxYgbVr18rexvxByv/73/8QHBystU7t2rVlt1sc+vr6WsuLkyQUhtz3R1dxqlQqtG7dGmPGjNE6Pz+pKEj58uWL9Ydq+PDhWLlypTTt7+//0kceNGrUCI6OjlizZo1GApUfR4UKFYocz/NUKhXs7OwKvPP1xT802o4ZXcRQUp+LF+NVqVRQKBTYuXOn1nbMzc0LG3axyN2PxTkWXhwc/ao2tZU/vx6558iixr569WqEhIQgKCgIX3zxBezs7KCvr4/IyEhcv369yOspqJ6uDR06FCtWrMCIESPg5+cHKysrKBQK9OrVq1A3r8il7TOlUCiwadMmnDx5Etu3b8euXbvQv39/fP/99zh58uRr+bwzgSoha9asgZ2dHRYsWKAxb8uWLdi6dSsWL14MExMTNGvWDI6Ojli/fj2aNGmC/fv34+uvv1Zbxs3NDWfPnkWrVq0K9Q1dm82bN8PY2Bi7du1Sy9BXrFihVs/FxQUqlQqJiYlq32YTEhLU6tna2sLCwgJ5eXlSL4Mcbm5uUKlUuHjxIurWrau1Tv6D/K5cuaLW45CTk4PExMQirbew8td97do16ZIL8F+XdWJiolqCq4v353m2trawtLTEhQsXXlrPzc0NmZmZRd4Pnp6eaj2kco0ZM0btkmlhLgc+ffpU67fUxMREVKhQocDemuepVCr8/fffagni1atXAfzfnX5ubm7Yu3cvGjdurNPkKP9zkZCQIF2eB4Dc3FzcuHFD7UuDrj8XL+Pm5gYhBFxdXV+ZOBfW88ffiy5fvowKFSqo9T6VFGtra427s3JycpCcnKzT9RT2HFlcmzZtQtWqVbFlyxa1z8WECRN0uh5tXFxcsHfvXjx69EitF+ry5cvS/HwFfWY3bdqE4OBgfP/991LZ06dPNd6jgpbX5Xm9YcOGaNiwIb799lusXbsWffr0wbp1617Ls8w4BqoEPHnyBFu2bEHHjh3RrVs3jdeQIUPw6NEj/PrrrwD+uxzRrVs3bN++HatWrUJubq7a5TsA6NGjB5KSkrB06VKt6yvM81b09fWhUCjUvrXduHFD40mz+eNAFi5cqFY+b948jfa6du2KzZs3a/1D/6qf5ggKCoKenh4mTZqk8a0l/5tVQEAAjIyMMHfuXLVvW8uXL0d6ejo6dOjw0nUUR/369WFra4vFixerPTYgKipK40Shi/fneXp6eggKCsL27du1/tRF/r7o0aMHTpw4gV27dmnUSUtLQ25u7kvX4+fnhwsXLmjcUl9YNWrUQEBAgPTy8fEB8N9t0tqe1bJ582Y8fPhQ67iuuLg4+Pn5FXrd8+fPl/4vhMD8+fNhaGiIVq1aAfhv3+Tl5WHy5Mkay+bm5mq8h4VVv359lC9fHkuXLlXbv2vWrNHozdP15+JlPvzwQ+jr6yMiIkKjZ0IIgfv378tu09HREXXr1sXKlSvV9teFCxewe/dutG/fvrhhF4qbm5vGeL4lS5YU2ANVVIU9R+piPYB6D9KpU6dw4sQJna5Hm/bt2yMvL0/t+AGAWbNmQaFQoF27dlKZmZmZ1uNEX19f4zM2b948jfcjP7l+sQ1dnNcfPnyoEUP+F/Gins/kYg9UCfj111/x6NEjdO7cWev8/J+3WLNmjZQo9ezZE/PmzcOECRNQq1Yt6Xp0vr59+2LDhg349NNPceDAATRu3Bh5eXm4fPkyNmzYID2/5mU6dOiAH374AW3btsVHH32E1NRULFiwAO7u7mpjhHx8fNC1a1fMnj0b9+/fR8OGDXHo0CHpG/7z3yqmTZuGAwcOwNfXF4MGDUKNGjXw4MED/PHHH9i7dy8ePHhQYDzu7u74+uuvMXnyZDRt2hQffvghlEolzpw5AycnJ0RGRsLW1hbh4eGIiIhA27Zt0blzZ1y5cgULFy5EgwYN1Ho/dM3Q0BBTpkzBJ598gpYtW6Jnz55ITEzEihUrNMbf6OL9edHUqVOxe/du+Pv7Y/DgwahevTqSk5OxceNGHD16FOXKlcMXX3yBX3/9FR07dkRISAh8fHyQlZWF8+fPY9OmTbhx48ZLL4l16dIFkydPxqFDh6RLyvnmz5+PtLQ0/PvvvwCA7du34/bt2wD+68J/cSD9865du4aAgAD07NkTnp6e0NPTQ2xsLFavXo0qVapg+PDhavVTU1Nx7tw5hIaGFmrfGBsbIyYmBsHBwfD19cXOnTvx22+/YezYsVIPlr+/Pz755BNERkYiPj4ebdq0gaGhIa5du4aNGzdizpw56NatW6HW9zwjIyNMnDgRQ4cORcuWLdGjRw/cuHEDUVFRcHNzUzs+SuJzURA3NzdMmTIF4eHhuHHjBoKCgmBhYYHExERs3boVgwcPVnuuXGHNmDED7dq1g5+fHwYMGIAnT55g3rx5sLKy0vpsppIwcOBAfPrpp+jatStat26Ns2fPYteuXTq73JuvsOfI4urYsSO2bNmCDz74AB06dEBiYiIWL16MGjVqFHiTha506tQJLVq0wNdff40bN26gTp062L17N7Zt24YRI0aoDYr38fHB3r178cMPP8DJyQmurq7w9fVFx44dsWrVKlhZWaFGjRo4ceIE9u7di/Lly6utq27dutDX18d3332H9PR0KJVKtGzZEnZ2dsU+r69cuRILFy7EBx98ADc3Nzx69AhLly6FpaXla0vs+RiDEtCpUydhbGwsPQ9Dm5CQEGFoaCjd/q9SqYSzs7PW20vz5eTkiO+++07UrFlTKJVKYW1tLXx8fERERIRIT0+X6kHLLfr5li9fLqpVqyaUSqXw9PQUK1askG4/fl5WVpYIDQ0VNjY2wtzcXAQFBYkrV64IAGLatGlqde/cuSNCQ0OFs7OzMDQ0FA4ODqJVq1ZiyZIlhdpfP/30k/D29pa2yd/fX3r0Q7758+cLT09PYWhoKOzt7cVnn32m8Vwbf39/UbNmTY32g4ODhYuLi1qZtn2k7TlQQvz3zB9XV1ehVCpF/fr1xeHDh4W/v7/G7c/FfX9evCVbCCFu3rwp+vXrJ2xtbYVSqRRVq1YVoaGhardAP3r0SISHhwt3d3dhZGQkKlSoIBo1aiRmzpyp9jyVgtSuXVt6VtmL8eC5RxA8/3r++T/a3L17VwwePFh4enoKMzMzYWRkJKpVqyZGjBghPWbgeYsWLRKmpqYiIyPjlfEGBwcLMzMzcf36dekZS/b29mLChAkaj8MQ4r/HTvj4+AgTExNhYWEhatWqJcaMGSP+/fdftW3t0KHDK9f9vLlz5woXFxehVCrF+++/L44dOyZ8fHxE27Zt1erp+nORf7xq249CCLF582bRpEkTYWZmJszMzISnp6cIDQ0VV65ceen2FPQYAyGE2Lt3r2jcuLEwMTERlpaWolOnTuLixYtqdV4V14vyHznw4m3o2o7DvLw88eWXX4oKFSoIU1NTERgYKBISEgp8jMGLbRYUW/5n6XmFPUfKOY5fpFKpxNSpU6XPj7e3t9ixY4fGuSr/MQYzZszQaAMvPNpB27Y8v+3Pe/TokRg5cqRwcnIShoaGolq1amLGjBlqj34QQojLly+LZs2aCRMTEwFA2q6HDx+Kjz/+WFSoUEGYm5uLwMBAcfnyZa3bvnTpUlG1alWhr6+v8b4W57z+xx9/iN69e4vKlSsLpVIp7OzsRMeOHUVsbKxG3ZKiEKKER9fSWyM+Ph7e3t5YvXo1+vTpU9rhkI6sWrUKoaGhuHXrlk5/xkMOb29vNG/eHLNmzXpl3ZCQEGzatKnEv6nLpVKpYGtriw8//FDrJTsiertwDBRppe13wGbPng09PT2Nn/2gN1ufPn1QuXJlrTc8vA4xMTG4du0awsPDS2X9RfH06VON8Rc///wzHjx4oPFTLkT0duIYKNJq+vTpiIuLQ4sWLWBgYICdO3di586dGDx4sMat1/Rm09PTe+XdfiWpbdu2Za436VVOnjyJkSNHonv37ihfvjz++OMPLF++HF5eXtLvfxHR240JFGnVqFEj7NmzB5MnT0ZmZiYqV66MiRMnajxegehdVKVKFTg7O2Pu3Ll48OABbGxs0K9fP0ybNk3jgX9E9HbiGCgiIiIimTgGioiIiEgmJlBEREREMnEMlBYqlQr//vsvLCwsSvznF4iIiKhsEELg0aNHcHJygp7ey/uYmEBp8e+///JOMyIionfUP//8g0qVKr20DhMoLfJ/YPGff/6BpaVlKUdDREREr0NGRgacnZ3Vfmi5IEygtMi/bGdpackEioiI6B1TmOE7HEROREREJBMTKCIiIiKZmEARERERycQEioiIiEgmJlBEREREMjGBIiIiIpKJCRQRERGRTEygiIiIiGRiAkVEREQkU6kmUIsWLULt2rWlJ377+flh586dL11m48aN8PT0hLGxMWrVqoXff/9dbb4QAuPHj4ejoyNMTEwQEBCAa9euleRmEBER0TumVBOoSpUqYdq0aYiLi0NsbCxatmyJLl264K+//tJa//jx4+jduzcGDBiAP//8E0FBQQgKCsKFCxekOtOnT8fcuXOxePFinDp1CmZmZggMDMTTp09f12YRERHRW04hhBClHcTzbGxsMGPGDAwYMEBjXs+ePZGVlYUdO3ZIZQ0bNkTdunWxePFiCCHg5OSEUaNGYfTo0QCA9PR02NvbIyoqCr169SpUDBkZGbCyskJ6ejp/C4+IiOgdIefvf5n5MeG8vDxs3LgRWVlZ8PPz01rnxIkTCAsLUysLDAxEdHQ0ACAxMREpKSkICAiQ5ltZWcHX1xcnTpwoMIHKzs5Gdna2NJ2RkVHMraGy6PHjx7h8+fIr6z19lofbDx+jkrUpjA31X1nf09MTpqamugiRiEoIj3/StVJPoM6fPw8/Pz88ffoU5ubm2Lp1K2rUqKG1bkpKCuzt7dXK7O3tkZKSIs3PLyuojjaRkZGIiIgozmbQG+Dy5cvw8fHRebtxcXGoV6+eztslIt3h8U+6VuoJlIeHB+Lj45Geno5NmzYhODgYhw4dKjCJKgnh4eFqPVsZGRlwdnZ+beun18PT0xNxcXGvrJeQ+gjD18VjTq+6cLezKFS7RFS28fgnXSv1BMrIyAju7u4AAB8fH5w5cwZz5szBjz/+qFHXwcEBd+7cUSu7c+cOHBwcpPn5ZY6Ojmp16tatW2AMSqUSSqWyuJtCZZypqWmhvikaJaVDeTATNWrVhVdFq9cQGRGVNB7/pGtl7jlQKpVKbTzS8/z8/LBv3z61sj179khjplxdXeHg4KBWJyMjA6dOnSpwXBURERGRXKXaAxUeHo527dqhcuXKePToEdauXYuDBw9i165dAIB+/fqhYsWKiIyMBAAMHz4c/v7++P7779GhQwesW7cOsbGxWLJkCQBAoVBgxIgRmDJlCqpVqwZXV1eMGzcOTk5OCAoKKq3NJCIiordMqSZQqamp6NevH5KTk2FlZYXatWtj165daN26NQDg1q1b0NP7v06yRo0aYe3atfjmm28wduxYVKtWDdHR0fDy8pLqjBkzBllZWRg8eDDS0tLQpEkTxMTEwNjY+LVvHxEREb2dytxzoMoCPgfq3XYhKR0d5x3FjqFNOAaC6B3D4//dJufvf5kbA0VERERU1jGBIiIiIpKJCRQRERGRTEygiIiIiGRiAkVEREQkExMoIiIiIpmYQBERERHJxASKiIiISCYmUEREREQyMYEiIiIikokJFBEREZFMTKCIiIiIZGICRURERCQTEygiIiIimZhAEREREcnEBIqIiIhIJiZQRERERDIxgSIiIiKSiQkUERERkUxMoIiIiIhkYgJFREREJBMTKCIiIiKZmEARERERycQEioiIiEgmJlBEREREMjGBIiIiIpKJCRQRERGRTEygiIiIiGRiAkVEREQkExMoIiIiIplKNYGKjIxEgwYNYGFhATs7OwQFBeHKlSsvXaZ58+ZQKBQarw4dOkh1QkJCNOa3bdu2pDeHiIiI3hEGpbnyQ4cOITQ0FA0aNEBubi7Gjh2LNm3a4OLFizAzM9O6zJYtW5CTkyNN379/H3Xq1EH37t3V6rVt2xYrVqyQppVKZclsBBEREb1zSjWBiomJUZuOioqCnZ0d4uLi0KxZM63L2NjYqE2vW7cOpqamGgmUUqmEg4ODbgMmIiIiQiknUC9KT08HoJkkvczy5cvRq1cvjR6rgwcPws7ODtbW1mjZsiWmTJmC8uXLa20jOzsb2dnZ0nRGRkYRoqfSFnf7Jv59dKfY7fzz4DH0jJNw+MafSMwwLXZ7Thb28KnkUux2iIio7CgzCZRKpcKIESPQuHFjeHl5FWqZ06dP48KFC1i+fLlaedu2bfHhhx/C1dUV169fx9ixY9GuXTucOHEC+vr6Gu1ERkYiIiJCJ9tBpSPxXhZ6r5sFpe0+nbRn5gosStBJU8i+2wq7Qr6FawXtl6WJiOjNU2YSqNDQUFy4cAFHjx4t9DLLly9HrVq18P7776uV9+rVS/p/rVq1ULt2bbi5ueHgwYNo1aqVRjvh4eEICwuTpjMyMuDs7FyEraDSkpWdi2dpvhjhFwRnm+L1GmXnqpCa8RR2lsZQGhTvPot/HjzGjGvJyMrOLVY7RERUtpSJBGrIkCHYsWMHDh8+jEqVKhVqmaysLKxbtw6TJk16Zd2qVauiQoUKSEhI0JpAKZVKDjJ/C4hcSzSr4g2vilalHYrkQlI6pudmlXYYRESkY6WaQAkhMHToUGzduhUHDx6Eq6troZfduHEjsrOz8b///e+VdW/fvo379+/D0dGxOOESERERASjl50CFhoZi9erVWLt2LSwsLJCSkoKUlBQ8efJEqtOvXz+Eh4drLLt8+XIEBQVpDAzPzMzEF198gZMnT+LGjRvYt28funTpAnd3dwQGBpb4NhEREdHbr1R7oBYtWgTgv4djPm/FihUICQkBANy6dQt6eup53pUrV3D06FHs3r1bo019fX2cO3cOK1euRFpaGpycnNCmTRtMnjyZl+mIiIhIJ0r9Et6rHDx4UKPMw8OjwGVNTEywa9eu4oZGREREVCD+Fh4RERGRTEygiIiIiGRiAkVEREQkExMoIiIiIpmYQBERERHJxASKiIiISCYmUEREREQyMYEiIiIikokJFBEREZFMTKCIiIiIZGICRURERCQTEygiIiIimZhAEREREcnEBIqIiIhIJiZQRERERDIxgSIiIiKSiQkUERERkUxMoIiIiIhkYgJFREREJBMTKCIiIiKZmEARERERycQEioiIiEgmJlBEREREMjGBIiIiIpKJCRQRERGRTEygiIiIiGRiAkVEREQkExMoIiIiIpmYQBERERHJxASKiIiISKZSTaAiIyPRoEEDWFhYwM7ODkFBQbhy5cpLl4mKioJCoVB7GRsbq9URQmD8+PFwdHSEiYkJAgICcO3atZLcFCIiInqHlGoCdejQIYSGhuLkyZPYs2cPnj17hjZt2iArK+uly1laWiI5OVl63bx5U23+9OnTMXfuXCxevBinTp2CmZkZAgMD8fTp05LcHCIiInpHGJTmymNiYtSmo6KiYGdnh7i4ODRr1qzA5RQKBRwcHLTOE0Jg9uzZ+Oabb9ClSxcAwM8//wx7e3tER0ejV69eutsAIiIieieVagL1ovT0dACAjY3NS+tlZmbCxcUFKpUK9erVw9SpU1GzZk0AQGJiIlJSUhAQECDVt7Kygq+vL06cOKE1gcrOzkZ2drY0nZGRoYvNISKi1yTxXhaysnOL3U5Caqbav8VlpjSAawUznbRFZUuZSaBUKhVGjBiBxo0bw8vLq8B6Hh4e+Omnn1C7dm2kp6dj5syZaNSoEf766y9UqlQJKSkpAAB7e3u15ezt7aV5L4qMjERERITuNoaIiF6bxHtZaDHzoE7bHLE+XmdtHRjdnEnUW6jMJFChoaG4cOECjh49+tJ6fn5+8PPzk6YbNWqE6tWr48cff8TkyZOLtO7w8HCEhYVJ0xkZGXB2di5SW0RE9Hrl9zzN7lkX7nbmxWrr6bM83H74BJWsTWBsqF+sthJSMzFifbxOesao7CkTCdSQIUOwY8cOHD58GJUqVZK1rKGhIby9vZGQkAAA0tioO3fuwNHRUap3584d1K1bV2sbSqUSSqWyaMETEVGZ4G5nDq+KVsVup36V4sdCb79SvQtPCIEhQ4Zg69at2L9/P1xdXWW3kZeXh/Pnz0vJkqurKxwcHLBv3z6pTkZGBk6dOqXWc0VERERUVKXaAxUaGoq1a9di27ZtsLCwkMYoWVlZwcTEBADQr18/VKxYEZGRkQCASZMmoWHDhnB3d0daWhpmzJiBmzdvYuDAgQD+u0NvxIgRmDJlCqpVqwZXV1eMGzcOTk5OCAoKKpXtJCIiordLqSZQixYtAgA0b95crXzFihUICQkBANy6dQt6ev/XUfbw4UMMGjQIKSkpsLa2ho+PD44fP44aNWpIdcaMGYOsrCwMHjwYaWlpaNKkCWJiYjQeuElERERUFKWaQAkhXlnn4MGDatOzZs3CrFmzXrqMQqHApEmTMGnSpOKER0RERKQVfwuPiIiISCYmUEREREQyMYEiIiIikokJFBEREZFMTKCIiIiIZGICRURERCQTEygiIiIimZhAEREREclUpARq1apVaNy4MZycnHDz5k0AwOzZs7Ft2zadBkdERERUFslOoBYtWoSwsDC0b98eaWlpyMvLAwCUK1cOs2fP1nV8RERERGWO7ARq3rx5WLp0Kb7++mvo6+tL5fXr18f58+d1GhwRERFRWSQ7gUpMTIS3t7dGuVKpRFZWlk6CIiIiIirLZCdQrq6uiI+P1yiPiYlB9erVdRETERERUZlmIHeBsLAwhIaG4unTpxBC4PTp0/jll18QGRmJZcuWlUSMRERERGWK7ARq4MCBMDExwTfffIPHjx/jo48+gpOTE+bMmYNevXqVRIxEREREZYrsBAoA+vTpgz59+uDx48fIzMyEnZ2druMiIiIiKrNkJ1CJiYnIzc1FtWrVYGpqClNTUwDAtWvXYGhoiCpVqug6RiIiIqIyRfYg8pCQEBw/flyj/NSpUwgJCdFFTERERERlmuwE6s8//0Tjxo01yhs2bKj17jwiIiKit43sBEqhUODRo0ca5enp6dJTyYmIiIjeZrITqGbNmiEyMlItWcrLy0NkZCSaNGmi0+CIiIiIyiLZg8i/++47NGvWDB4eHmjatCkA4MiRI8jIyMD+/ft1HiARERFRWSO7B6pGjRo4d+4cevTogdTUVDx69Aj9+vXD5cuX4eXlVRIxEhEREZUpRXoOlJOTE6ZOnarrWIiIiIjeCEVKoNLS0nD69GmkpqZCpVKpzevXr59OAiMiIiIqq2QnUNu3b0efPn2QmZkJS0tLKBQKaZ5CoWACRURERG892WOgRo0ahf79+yMzMxNpaWl4+PCh9Hrw4EFJxEhERERUpshOoJKSkjBs2DDpJ1yIiIiI3jWyE6jAwEDExsaWRCxEREREbwTZY6A6dOiAL774AhcvXkStWrVgaGioNr9z5846C46IiIioLJLdAzVo0CD8888/mDRpErp3746goCDp9cEHH8hqKzIyEg0aNICFhQXs7OwQFBSEK1euvHSZpUuXomnTprC2toa1tTUCAgJw+vRptTohISFQKBRqr7Zt28rdVCIiIiKtZCdQKpWqwJfc38I7dOgQQkNDcfLkSezZswfPnj1DmzZtkJWVVeAyBw8eRO/evXHgwAGcOHECzs7OaNOmDZKSktTqtW3bFsnJydLrl19+kbupRERERFoV6TlQ+Z4+fQpjY+MiLx8TE6M2HRUVBTs7O8TFxaFZs2Zal1mzZo3a9LJly7B582bs27dP7REKSqUSDg4ORY6NiIiIqCCyE6i8vDxMnToVixcvxp07d3D16lVUrVoV48aNQ5UqVTBgwIAiB5Oeng4AsLGxKfQyjx8/xrNnzzSWOXjwIOzs7GBtbY2WLVtiypQpKF++vNY2srOzkZ2dLU1nZGQUIXoqTU+e/df7eSEpvdhtPX2Wh9sPn6CStQmMDfWL1VZCamax4yGiV1MYZCAx4wr0jM1LOxRJYkYmFAb8e/K2kp1Affvtt1i5ciWmT5+OQYMGSeVeXl6YPXt2kRMolUqFESNGoHHjxrJ+U+/LL7+Ek5MTAgICpLK2bdviww8/hKurK65fv46xY8eiXbt2OHHiBPT1Nf8gRkZGIiIiokhxU9lw/f8nKl9tOV/KkWhnpixWZy8RvYJhuVMYe7rs/cSYYblWANqXdhhUAmSf1X/++WcsWbIErVq1wqeffiqV16lTB5cvXy5yIKGhobhw4QKOHj1a6GWmTZuGdevW4eDBg2qXEnv16iX9v1atWqhduzbc3Nxw8OBBtGrVSqOd8PBwhIWFSdMZGRlwdnYu4pZQaWhT87/LtW525jDRQa/RiPXxmN2zLtztiv9t1kxpANcKZsVuh4gK9izNF993+AhuOjhmdeV6aiaGrble2mFQCZGdQCUlJcHd3V2jXKVS4dmzZ0UKYsiQIdixYwcOHz6MSpUqFWqZmTNnYtq0adi7dy9q16790rpVq1ZFhQoVkJCQoDWBUiqVUCqVRYqdygYbMyP0er+yTtt0tzOHV0UrnbZJRCVD5FrC1dIDNcqXnWNW9TQdIvduaYdBJUT2XXg1atTAkSNHNMo3bdoEb29vWW0JITBkyBBs3boV+/fvh6ura6GWmz59OiZPnoyYmBjUr1//lfVv376N+/fvw9HRUVZ8RERERNrI7oEaP348goODkZSUBJVKhS1btuDKlSv4+eefsWPHDllthYaGYu3atdi2bRssLCyQkpICALCysoKJiQkAoF+/fqhYsSIiIyMBAN999x3Gjx+PtWvXokqVKtIy5ubmMDc3R2ZmJiIiItC1a1c4ODjg+vXrGDNmDNzd3REYGCh3c4mIiIg0yO6B6tKlC7Zv3469e/fCzMwM48ePx6VLl7B9+3a0bt1aVluLFi1Ceno6mjdvDkdHR+m1fv16qc6tW7eQnJystkxOTg66deumtszMmTMBAPr6+jh37hw6d+6M9957DwMGDICPjw+OHDnCy3RERESkE0W6Nahp06bYs2dPsVcuhHhlnYMHD6pN37hx46X1TUxMsGvXrmJERURERPRysnugiIiIiN51heqBsra2hkKhKFSDDx48KFZARERERGVdoRKo2bNnS/+/f/8+pkyZgsDAQPj5+QEATpw4gV27dmHcuHElEiQRERFRWVKoBCo4OFj6f9euXTFp0iQMGTJEKhs2bBjmz5+PvXv3YuTIkbqPkoiIiKgMkT0GateuXWjbtq1Gedu2bbF3716dBEVERERUlslOoMqXL49t27ZplG/btq3AH+slIiIiepvIfoxBREQEBg4ciIMHD8LX1xcAcOrUKcTExGDp0qU6D5CIiIiorJGdQIWEhKB69eqYO3cutmzZAgCoXr06jh49KiVURERERG+zIj1I09fXF2vWrNF1LERERERvhEIlUBkZGbC0tJT+/zL59YiIiIjeVoV+kGZycjLs7OxQrlw5rQ/VFEJAoVAgLy9P50ESERERlSWFSqD2798PGxsbAMCBAwdKNCAiIiKisq5QCdScOXPg7e0NS0tL3Lx5Ez179oRSqSzp2IiIiIjKpEI9B2rHjh3IysoCAHz88cdIT08v0aCIiIiIyrJC9UB5enoiPDwcLVq0gBACGzZsKHCweL9+/XQaIBEREVFZU6gEavHixQgLC8Nvv/0GhUKBb775RutAcoVCwQSKiIiI3nqFSqAaNWqEkydPAgD09PRw9epV2NnZlWhgRERERGWV7N/CS0xMhK2tbUnEQkRERPRGkP0kchcXF6SlpeH06dNITU2FSqVSm89LeERERPS2k51Abd++HX369EFmZiYsLS3VxkJxDBQRERG9C2Rfwhs1ahT69++PzMxMpKWl4eHDh9LrwYMHJREjERERUZkiO4FKSkrCsGHDYGpqWhLxEBEREZV5shOowMBAxMbGlkQsRERERG8E2WOgOnTogC+++AIXL15ErVq1YGhoqDa/c+fOOguOiIiIqCySnUANGjQIADBp0iSNeQqFAnl5ecWPioiIiKgMk51AvfjYAiIiIqJ3jewxUERERETvuiIlUIcOHUKnTp3g7u4Od3d3dO7cGUeOHNF1bERERERlkuwEavXq1QgICICpqSmGDRuGYcOGwcTEBK1atcLatWtLIkYiIiKiMkX2GKhvv/0W06dPx8iRI6WyYcOG4YcffsDkyZPx0Ucf6TRAIiIiorJGdg/U33//jU6dOmmUd+7cGYmJibLaioyMRIMGDWBhYQE7OzsEBQXhypUrr1xu48aN8PT0hLGxMWrVqoXff/9dbb4QAuPHj4ejoyNMTEwQEBCAa9euyYqNiIiIqCCyEyhnZ2fs27dPo3zv3r1wdnaW1dahQ4cQGhqKkydPYs+ePXj27BnatGmDrKysApc5fvw4evfujQEDBuDPP/9EUFAQgoKCcOHCBanO9OnTMXfuXCxevBinTp2CmZkZAgMD8fTpU1nxEREREWkj+xLeqFGjMGzYMMTHx6NRo0YAgGPHjiEqKgpz5syR1VZMTIzadFRUFOzs7BAXF4dmzZppXWbOnDlo27YtvvjiCwDA5MmTsWfPHsyfPx+LFy+GEAKzZ8/GN998gy5dugAAfv75Z9jb2yM6Ohq9evWSu8lEREREamQnUJ999hkcHBzw/fffY8OGDQCA6tWrY/369VLCUlTp6ekAABsbmwLrnDhxAmFhYWplgYGBiI6OBgAkJiYiJSUFAQEB0nwrKyv4+vrixIkTWhOo7OxsZGdnS9MZGRnF2QwiInqNnjz77wHOF5LSi93W02d5uP3wCSpZm8DYUL9YbSWkZhY7Hiq7ZCdQAPDBBx/ggw8+0GkgKpUKI0aMQOPGjeHl5VVgvZSUFNjb26uV2dvbIyUlRZqfX1ZQnRdFRkYiIiKiOOETEVEpuf7/E5Wvtpwv5Ui0M1MW6U8tlXGy39UzZ85ApVLB19dXrfzUqVPQ19dH/fr1ixRIaGgoLly4gKNHjxZp+eIIDw9X69XKyMiQPZ6LiIhKR5uaDgAANztzmOig12jE+njM7lkX7nbmxY7NTGkA1wpmxW6Hyh7ZCVRoaCjGjBmjkUAlJSXhu+++w6lTp2QHMWTIEOzYsQOHDx9GpUqVXlrXwcEBd+7cUSu7c+cOHBwcpPn5ZY6Ojmp16tatq7VNpVIJpVIpO24iIip9NmZG6PV+ZZ226W5nDq+KVjptk94usu/Cu3jxIurVq6dR7u3tjYsXL8pqSwiBIUOGYOvWrdi/fz9cXV1fuYyfn5/GXYB79uyBn58fAMDV1RUODg5qdTIyMnDq1CmpDhEREVFxyO6BUiqVuHPnDqpWrapWnpycDAMDec2FhoZi7dq12LZtGywsLKQxSlZWVjAxMQEA9OvXDxUrVkRkZCQAYPjw4fD398f333+PDh06YN26dYiNjcWSJUsAAAqFAiNGjMCUKVNQrVo1uLq6Yty4cXByckJQUJDczSUiIiLSILsHqk2bNggPD5fumAOAtLQ0jB07Fq1bt5bV1qJFi5Ceno7mzZvD0dFReq1fv16qc+vWLSQnJ0vTjRo1wtq1a7FkyRLUqVMHmzZtQnR0tNrA8zFjxmDo0KEYPHgwGjRogMzMTMTExMDY2Fju5hIRERFpUAghhJwFkpKS0KxZM9y/fx/e3t4AgPj4eNjb22PPnj1vxeDrjIwMWFlZIT09HZaWlqUdDr1mF5LS0XHeUewY2oRjIIjeMTz+321y/v7LvoRXsWJFnDt3DmvWrMHZs2dhYmKCjz/+GL1794ahoWGRgyYiIiJ6UxTp4RRmZmYYPHiwrmMhIiIieiPIHgNFRERE9K5jAkVEREQkExMoIiIiIpmYQBERERHJVKQEKi0tDcuWLUN4eDgePHgAAPjjjz+QlJSk0+CIiIiIyiLZd+GdO3cOAQEBsLKywo0bNzBo0CDY2Nhgy5YtuHXrFn7++eeSiJOIiIiozJDdAxUWFoaQkBBcu3ZN7cne7du3x+HDh3UaHBEREVFZJDuBOnPmDD755BON8ooVK0q/ZUdERET0NpOdQCmVSmRkZGiUX716Fba2tjoJioiIiKgsk51Ade7cGZMmTcKzZ88AAAqFArdu3cKXX36Jrl276jxAIiIiorJGdgL1/fffIzMzE3Z2dnjy5An8/f3h7u4OCwsLfPvttyURIxEREVGZIvsuPCsrK+zZswdHjx7FuXPnkJmZiXr16iEgIKAk4iMiIiIqc4r0Y8IA0KRJEzRp0kSXsRARERG9EWQnUHPnztVarlAoYGxsDHd3dzRr1gz6+vrFDo6IiIioLJKdQM2aNQt3797F48ePYW1tDQB4+PAhTE1NYW5ujtTUVFStWhUHDhyAs7OzzgMmIiIiKm2yB5FPnToVDRo0wLVr13D//n3cv38fV69eha+vL+bMmYNbt27BwcEBI0eOLIl4iYiIiEqd7B6ob775Bps3b4abm5tU5u7ujpkzZ6Jr1674+++/MX36dD7SgIiIiN5asnugkpOTkZubq1Gem5srPYncyckJjx49Kn50RERERGWQ7ASqRYsW+OSTT/Dnn39KZX/++Sc+++wztGzZEgBw/vx5uLq66i5KIiIiojJEdgK1fPly2NjYwMfHB0qlEkqlEvXr14eNjQ2WL18OADA3N8f333+v82CJiIiIygLZY6AcHBywZ88eXL58GVevXgUAeHh4wMPDQ6rTokUL3UVIREREVMYU+UGanp6e8PT01GUsRERERG+EIiVQt2/fxq+//opbt24hJydHbd4PP/ygk8CIiIiIyirZCdS+ffvQuXNnVK1aFZcvX4aXlxdu3LgBIQTq1atXEjESERERlSmyB5GHh4dj9OjROH/+PIyNjbF582b8888/8Pf3R/fu3UsiRiIiIqIyRXYCdenSJfTr1w8AYGBggCdPnsDc3ByTJk3Cd999p/MAiYiIiMoa2QmUmZmZNO7J0dER169fl+bdu3dPd5ERERERlVGyx0A1bNgQR48eRfXq1dG+fXuMGjUK58+fx5YtW9CwYcOSiJGIiIioTJHdA/XDDz/A19cXABAREYFWrVph/fr1qFKlivQgzcI6fPgwOnXqBCcnJygUCkRHR7+0fkhICBQKhcarZs2aUp2JEydqzOfjFoiIiEiXZPVA5eXl4fbt26hduzaA/y7nLV68uMgrz8rKQp06ddC/f398+OGHr6w/Z84cTJs2TZrOzc1FnTp1NAav16xZE3v37pWmDQyK/LgrIiIiIg2yMgt9fX20adMGly5dQrly5Yq98nbt2qFdu3aFrm9lZQUrKytpOjo6Gg8fPsTHH3+sVs/AwAAODg7Fjo+IiIhIG9ldM15eXvj777/LxI8FL1++HAEBAXBxcVErv3btGpycnGBsbAw/Pz9ERkaicuXKBbaTnZ2N7OxsaTojI6PEYqbS8/jxY1y+fPmV9RJSHyE7JQEXz5sj547FK+t7enrC1NRUFyESEdEbQnYCNWXKFIwePRqTJ0+Gj48PzMzM1OZbWlrqLLiX+ffff7Fz506sXbtWrdzX1xdRUVHw8PBAcnIyIiIi0LRpU1y4cAEWFtr/GEZGRiIiIuJ1hE2l6PLly/Dx8Sl0/Z4rC1cvLi6OD5ElInrHyE6g2rdvDwDo3LkzFAqFVC6EgEKhQF5enu6ie4mVK1eiXLlyCAoKUit//pJg7dq14evrCxcXF2zYsAEDBgzQ2lZ4eDjCwsKk6YyMDDg7O5dI3FR6PD09ERcX98p6T5/l4fbDx6hkbQpjQ/1CtUtERO8W2QnUgQMHSiIOWYQQ+Omnn9C3b18YGRm9tG65cuXw3nvvISEhocA6SqUSSqVS12FSGWNqasqeIiIi0gnZCZS/v39JxCHLoUOHkJCQUGCP0vMyMzNx/fp19O3b9zVERkRERO8C2c+BAoAjR47gf//7Hxo1aoSkpCQAwKpVq3D06FFZ7WRmZiI+Ph7x8fEAgMTERMTHx+PWrVsA/ru0lv+zMc9bvnw5fH194eXlpTFv9OjROHToEG7cuIHjx4/jgw8+gL6+Pnr37i1zK4mIiIi0k51Abd68GYGBgTAxMcEff/wh3b2Wnp6OqVOnymorNjYW3t7e8Pb2BgCEhYXB29sb48ePBwAkJydLyVS+9PR0bN68ucDep9u3b6N3797w8PBAjx49UL58eZw8eRK2trZyN5WIiIhIK4UQQshZwNvbGyNHjkS/fv1gYWGBs2fPomrVqvjzzz/Rrl07pKSklFSsr01GRgasrKyQnp7+2u4qJCKi0nchKR0d5x3FjqFN4FXR6tUL0FtFzt9/2T1QV65cQbNmzTTKrayskJaWJrc5IiIiojeO7ATKwcFB6x1tR48eRdWqVXUSFBEREVFZJjuBGjRoEIYPH45Tp05BoVDg33//xZo1azB69Gh89tlnJREjERERUZki+zEGX331FVQqFVq1aoXHjx+jWbNmUCqVGD16NIYOHVoSMRIRERGVKbITKIVCga+//hpffPEFEhISkJmZiRo1asDc3Lwk4iMiIiIqc2Rfwlu9ejUeP34MIyMj1KhRA++//z6TJyIiInqnyE6gRo4cCTs7O3z00Uf4/fffX9tv3xERERGVFbITqOTkZKxbtw4KhQI9evSAo6MjQkNDcfz48ZKIj4iIiKjMkZ1AGRgYoGPHjlizZg1SU1Mxa9Ys3LhxAy1atICbm1tJxEhERERUpsgeRP48U1NTBAYG4uHDh7h58yYuXbqkq7iIiIiIyqwi/Zjw48ePsWbNGrRv3x4VK1bE7Nmz8cEHH+Cvv/7SdXxEREREZY7sHqhevXphx44dMDU1RY8ePTBu3Dj4+fmVRGxEREREZZLsBEpfXx8bNmxAYGAg9PX11eZduHABXl5eOguOiIiIqCySnUCtWbNGbfrRo0f45ZdfsGzZMsTFxfGxBkRERPTWK9IYKAA4fPgwgoOD4ejoiJkzZ6Jly5Y4efKkLmMjIiIiKpNk9UClpKQgKioKy5cvR0ZGBnr06IHs7GxER0ejRo0aJRUjERERUZlS6B6oTp06wcPDA+fOncPs2bPx77//Yt68eSUZGxEREVGZVOgeqJ07d2LYsGH47LPPUK1atZKMiYiIiKhMK3QP1NGjR/Ho0SP4+PjA19cX8+fPx71790oyNiIiIqIyqdAJVMOGDbF06VIkJyfjk08+wbp16+Dk5ASVSoU9e/bg0aNHJRknERERUZkh+y48MzMz9O/fH0ePHsX58+cxatQoTJs2DXZ2dujcuXNJxEhERERUphT5MQYA4OHhgenTp+P27dv45ZdfdBUTERERUZlWrAQqn76+PoKCgvDrr7/qojkiIiKiMk0nCRQRERHRu4QJFBEREZFMTKCIiIiIZGICRURERCQTEygiIiIimZhAEREREcnEBIqIiIhIplJNoA4fPoxOnTrByckJCoUC0dHRL61/8OBBKBQKjVdKSopavQULFqBKlSowNjaGr68vTp8+XYJbQURERO+aUk2gsrKyUKdOHSxYsEDWcleuXEFycrL0srOzk+atX78eYWFhmDBhAv744w/UqVMHgYGBSE1N1XX4RERE9I4yKM2Vt2vXDu3atZO9nJ2dHcqVK6d13g8//IBBgwbh448/BgAsXrwYv/32G3766Sd89dVXxQmXiIiICMAbOgaqbt26cHR0ROvWrXHs2DGpPCcnB3FxcQgICJDK9PT0EBAQgBMnThTYXnZ2NjIyMtReRERERAV5oxIoR0dHLF68GJs3b8bmzZvh7OyM5s2b448//gAA3Lt3D3l5ebC3t1dbzt7eXmOc1PMiIyNhZWUlvZydnUt0O4iIiOjNVqqX8OTy8PCAh4eHNN2oUSNcv34ds2bNwqpVq4rcbnh4OMLCwqTpjIwMJlFERERUoDcqgdLm/fffx9GjRwEAFSpUgL6+Pu7cuaNW586dO3BwcCiwDaVSCaVSWaJxEhER0dvjjbqEp018fDwcHR0BAEZGRvDx8cG+ffuk+SqVCvv27YOfn19phUhERERvmVLtgcrMzERCQoI0nZiYiPj4eNjY2KBy5coIDw9HUlISfv75ZwDA7Nmz4erqipo1a+Lp06dYtmwZ9u/fj927d0tthIWFITg4GPXr18f777+P2bNnIysrS7orj4iIiKi4SjWBio2NRYsWLaTp/HFIwcHBiIqKQnJyMm7duiXNz8nJwahRo5CUlARTU1PUrl0be/fuVWujZ8+euHv3LsaPH4+UlBTUrVsXMTExGgPLiYiIiIpKIYQQpR1EWZORkQErKyukp6fD0tKytMMhIqLX5EJSOjrOO4odQ5vAq6JVaYdDr5mcv/9v/BgoIiIioteNCRQRERGRTEygiIiIiGRiAkVEREQkExMoIiIiIpmYQBERERHJxASKiIiISCYmUEREREQyMYEiIiIikokJFBEREZFMTKCIiIiIZGICRURERCQTEygiIiIimZhAEREREcnEBIqIiIhIJiZQRERERDIxgSIiIiKSiQkUERERkUxMoIiIiIhkYgJFREREJBMTKCIiIiKZmEARERERycQEioiIiEgmJlBEREREMjGBIiIiIpKJCRQRERGRTEygiIiIiGRiAkVEREQkExMoIiIiIpmYQBERERHJVKoJ1OHDh9GpUyc4OTlBoVAgOjr6pfW3bNmC1q1bw9bWFpaWlvDz88OuXbvU6kycOBEKhULt5enpWYJbQURERO+aUk2gsrKyUKdOHSxYsKBQ9Q8fPozWrVvj999/R1xcHFq0aIFOnTrhzz//VKtXs2ZNJCcnS6+jR4+WRPhERET0jjIozZW3a9cO7dq1K3T92bNnq01PnToV27Ztw/bt2+Ht7S2VGxgYwMHBodDtZmdnIzs7W5rOyMgo9LJERFT2PX78GJcvX35lvYTUR8hOScDF8+bIuWPxyvqenp4wNTXVRYj0hinVBKq4VCoVHj16BBsbG7Xya9euwcnJCcbGxvDz80NkZCQqV65cYDuRkZGIiIgo6XCJiKiUXL58GT4+PoWu33Nl4erFxcWhXr16RYyK3mQKIYQo7SAAQKFQYOvWrQgKCir0MtOnT8e0adNw+fJl2NnZAQB27tyJzMxMeHh4IDk5GREREUhKSsKFCxdgYaH924S2HihnZ2ekp6fD0tKyWNtFRESlr7A9UE+f5eH2w8eoZG0KY0P9V9ZnD9TbJSMjA1ZWVoX6+//G9kCtXbsWERER2LZtm5Q8AVC7JFi7dm34+vrCxcUFGzZswIABA7S2pVQqoVQqSzxmIiIqHaampuwpIp16IxOodevWYeDAgdi4cSMCAgJeWrdcuXJ47733kJCQ8JqiIyIiorfdG/ccqF9++QUff/wxfvnlF3To0OGV9TMzM3H9+nU4Ojq+huiIiIjoXVCqPVCZmZlqPUOJiYmIj4+HjY0NKleujPDwcCQlJeHnn38G8N9lu+DgYMyZMwe+vr5ISUkBAJiYmMDKygoAMHr0aHTq1AkuLi74999/MWHCBOjr66N3796vfwOJiIjorVSqPVCxsbHw9vaWHkEQFhYGb29vjB8/HgCQnJyMW7duSfWXLFmC3NxchIaGwtHRUXoNHz5cqnP79m307t0bHh4e6NGjB8qXL4+TJ0/C1tb29W4cERERvbXKzF14ZYmcUfhERET0dpDz9/+NGwNFREREVNqYQBERERHJxASKiIiISCYmUEREREQyMYEiIiIikokJFBEREZFMTKCIiIiIZGICRURERCQTEygiIiIimZhAEREREcnEBIqIiIhIJiZQRERERDIxgSIiIiKSiQkUERERkUxMoIiIiIhkYgJFREREJBMTKCIiIiKZmEARERERycQEioiIiEgmJlBEREREMjGBIiIiIpKJCRQRERGRTEygiIiIiGRiAkVEREQkExMoIiIiIpmYQBERERHJxASKiIiISCYmUEREREQyMYEiIiIikqlUE6jDhw+jU6dOcHJygkKhQHR09CuXOXjwIOrVqwelUgl3d3dERUVp1FmwYAGqVKkCY2Nj+Pr64vTp07oPnoiIiN5ZpZpAZWVloU6dOliwYEGh6icmJqJDhw5o0aIF4uPjMWLECAwcOBC7du2S6qxfvx5hYWGYMGEC/vjjD9SpUweBgYFITU0tqc0gIiKid4xCCCFKOwgAUCgU2Lp1K4KCggqs8+WXX+K3337DhQsXpLJevXohLS0NMTExAABfX180aNAA8+fPBwCoVCo4Oztj6NCh+OqrrwoVS0ZGBqysrJCeng5LS8uibxQRERG9MeT8/X+jxkCdOHECAQEBamWBgYE4ceIEACAnJwdxcXFqdfT09BAQECDV0SY7OxsZGRlqLyIiIqKCvFEJVEpKCuzt7dXK7O3tkZGRgSdPnuDevXvIy8vTWiclJaXAdiMjI2FlZSW9nJ2dSyR+IiIieju8UQlUSQkPD0d6err0+ueff0o7JCIiIirDDEo7ADkcHBxw584dtbI7d+7A0tISJiYm0NfXh76+vtY6Dg4OBbarVCqhVCpLJGYiIiJ6+7xRPVB+fn7Yt2+fWtmePXvg5+cHADAyMoKPj49aHZVKhX379kl1iIiIiIqrVBOozMxMxMfHIz4+HsB/jymIj4/HrVu3APx3aa1fv35S/U8//RR///03xowZg8uXL2PhwoXYsGEDRo4cKdUJCwvD0qVLsXLlSly6dAmfffYZsrKy8PHHH7/WbSMiIqK3V6lewouNjUWLFi2k6bCwMABAcHAwoqKikJycLCVTAODq6orffvsNI0eOxJw5c1CpUiUsW7YMgYGBUp2ePXvi7t27GD9+PFJSUlC3bl3ExMRoDCwnIiIiKqoy8xyosoTPgSIiInr3vLXPgSIiIiIqC5hAEREREcnEBIqIiIhIJiZQRERERDIxgSIiIiKSiQkUERERkUxMoIiIiIhkYgJFREREJBMTKCIiIiKZSvWnXMqq/IezZ2RklHIkRERE9Lrk/90vzI+0MIHS4tGjRwAAZ2fnUo6EiIiIXrdHjx7BysrqpXX4W3haqFQq/Pvvv7CwsIBCoSjtcOg1y8jIgLOzM/755x/+FiLRO4bH/7tNCIFHjx7ByckJenovH+XEHigt9PT0UKlSpdIOg0qZpaUlT6BE7yge/++uV/U85eMgciIiIiKZmEARERERycQEiugFSqUSEyZMgFKpLO1QiOg14/FPhcVB5EREREQysQeKiIiISCYmUEREREQyMYEiIiIikokJFNFzqlSpgtmzZ5dI2wqFAtHR0aXeBlFZFBUVhXLlypV2GMXSvHlzjBgxotTboNeDCRQVSUhICBQKBRQKBQwNDeHq6ooxY8bg6dOnpR1aoRR0sj5z5gwGDx5c5HarVKki7Zf8V/5DWZOTk9GuXbtCtTNx4kTUrVtXo1xOG0Sv0927d/HZZ5+hcuXKUCqVcHBwQGBgII4dO1ao5Xv27ImrV6+WcJTF9/y57/lXQkICtmzZgsmTJxeqnYMHD0KhUCAtLU2tXE4bVLr4JHIqsrZt22LFihV49uwZ4uLiEBwcDIVCge+++660QysyW1vbYrcxadIkDBo0SJrW19cHADg4OBS7bV20QVQSunbtipycHKxcuRJVq1bFnTt3sG/fPty/f79Qy5uYmMDExKSEo9SN/HPf82xtbaVjvThsbGyK3Qa9JoKoCIKDg0WXLl3Uyj788EPh7e0tTT99+lQMHTpU2NraCqVSKRo3bixOnz4tzT9w4IAAIGJiYkTdunWFsbGxaNGihbhz5474/fffhaenp7CwsBC9e/cWWVlZ0nI7d+4UjRs3FlZWVsLGxkZ06NBBJCQkSPMTExMFALF582bRvHlzYWJiImrXri2OHz+utt7nXxMmTBBCCOHi4iJmzZoltfXw4UMxePBgYWdnJ5RKpahZs6bYvn17gfvlxeWfB0Bs3bpVmv7nn39Er169hLW1tTA1NRU+Pj7i5MmTYsWKFRrxrVixQmsb586dEy1atBDGxsbCxsZGDBo0SDx69EjjfZoxY4ZwcHAQNjY24vPPPxc5OTkFbgORXA8fPhQAxMGDB19Zr6DjacWKFcLKykqtfnR0tPD29hZKpVK4urqKiRMnimfPnknzAYilS5eKoKAgYWJiItzd3cW2bdvU2rhw4YLo0KGDsLCwEObm5qJJkyZq54ulS5cKT09PoVQqhYeHh1iwYMFLt0HbuS+fv7+/GD58uDT99OlTMWbMGFGpUiVhZGQk3NzcxLJly6Rz1POv4OBgrW08ePBA9O3bV5QrV06YmJiItm3biqtXr0rz8/dbTEyM8PT0FGZmZiIwMFD8+++/L90OKj5ewiOduHDhAo4fPw4jIyOpbMyYMdi8eTNWrlyJP/74A+7u7ggMDMSDBw/Ulp04cSLmz5+P48eP459//kGPHj0we/ZsrF27Fr/99ht2796NefPmSfWzsrIQFhaG2NhY7Nu3D3p6evjggw+gUqnU2v36668xevRoxMfH47333kPv3r2Rm5uLRo0aYfbs2bC0tERycjKSk5MxevRojW1SqVRo164djh07htWrV+PixYuYNm2aTr5lZmZmwt/fH0lJSfj1119x9uxZjBkzBiqVCj179sSoUaNQs2ZNKb6ePXtqtJGVlYXAwEBYW1vjzJkz2LhxI/bu3YshQ4ao1Ttw4ACuX7+OAwcOYOXKlYiKikJUVFSxt4Eon7m5OczNzREdHY3s7GytdeQeT0eOHEG/fv0wfPhwXLx4ET/++COioqLw7bffqtWLiIhAjx49cO7cObRv3x59+vSRzjFJSUlo1qwZlEol9u/fj7i4OPTv3x+5ubkAgDVr1mD8+PH49ttvcenSJUydOhXjxo3DypUrdbJf+vXrh19++QVz587FpUuX8OOPP8Lc3BzOzs7YvHkzAODKlStITk7GnDlztLYREhKC2NhY/Prrrzhx4gSEEGjfvj2ePXsm1Xn8+DFmzpyJVatW4fDhw7h165bWcxrpWGlncPRmCg4OFvr6+sLMzEwolUoBQOjp6YlNmzYJIYTIzMwUhoaGYs2aNdIyOTk5wsnJSUyfPl0I8X89QXv37pXqREZGCgDi+vXrUtknn3wiAgMDC4zl7t27AoA4f/68EOL/eqCWLVsm1fnrr78EAHHp0iUhhPZvu0Ko9yDt2rVL6OnpiStXrhR6v7i4uAgjIyNhZmYmvebMmSOEUO89+vHHH4WFhYW4f/++1nYmTJgg6tSpo1H+fBtLliwR1tbWIjMzU5r/22+/CT09PZGSkiKE+O99cnFxEbm5uVKd7t27i549exZ6m4gKY9OmTcLa2loYGxuLRo0aifDwcHH27Flp/quOpxePyVatWompU6eq1Vm1apVwdHSUpgGIb775RprOzMwUAMTOnTuFEEKEh4cLV1fXAntc3dzcxNq1a9XKJk+eLPz8/ArczufPffmvbt26CSHUe4+uXLkiAIg9e/ZobSf//Pfw4UO18ufbuHr1qgAgjh07Js2/d++eMDExERs2bBBCCKnH+vletQULFgh7e/sCt4F0g2OgqMhatGiBRYsWISsrC7NmzYKBgQG6du0KALh+/TqePXuGxo0bS/UNDQ3x/vvv49KlS2rt1K5dW/q/vb09TE1NUbVqVbWy06dPS9PXrl3D+PHjcerUKdy7d0/qebp16xa8vLy0tuvo6AgASE1NhaenZ6G2Lz4+HpUqVcJ7771XqPr5vvjiC4SEhEjTFSpU0Nq2t7d3scY7XLp0CXXq1IGZmZlU1rhxY6hUKly5cgX29vYAgJo1a6p9y3d0dMT58+eLvF4ibbp27YoOHTrgyJEjOHnyJHbu3Inp06dj2bJlCAkJkX08nT17FseOHVPrccrLy8PTp0/x+PFjmJqaAlA/zs3MzGBpaYnU1FQA/x1nTZs2haGhoUb7WVlZuH79OgYMGKA2ZjE3NxdWVlYvjS3/3Pf8el8UHx8PfX19+Pv7F2p7tbl06RIMDAzg6+srlZUvXx4eHh5q51FTU1O4ublJ046OjtI+oJLDBIqKzMzMDO7u7gCAn376CXXq1MHy5csxYMAAWe08f3LLv6vveQqFQu3yXKdOneDi4oKlS5fCyckJKpUKXl5eyMnJeWm7ADQu871MUQe0VqhQQdovum67KF61P4l0xdjYGK1bt0br1q0xbtw4DBw4EBMmTEBISIjsz3xmZiYiIiLw4Ycfal1Pvpd9vl+2zszMTADA0qVL1RIUAK+8TP/8ua8gpX2MC/5KW4njGCjSCT09PYwdOxbffPMNnjx5Ajc3NxgZGandwvzs2TOcOXMGNWrUKPJ67t+/jytXruCbb75Bq1atUL16dTx8+FB2O0ZGRsjLy3tpndq1a+P27dslcmt17dq1ER8frzEeTE581atXx9mzZ5GVlSWVHTt2DHp6evDw8NBpvERFUaNGDenzKfd4qlevHq5cuQJ3d3eNl55e4f501a5dG0eOHFEbL5TP3t4eTk5O+PvvvzXad3V1LfxGFqBWrVpQqVQ4dOiQ1vn540VfdpxXr14dubm5OHXqlFSWfw4sznmUdIMJFOlM9+7doa+vjwULFsDMzAyfffYZvvjiC8TExODixYsYNGgQHj9+LLuH6nnW1tYoX748lixZgoSEBOzfvx9hYWGy26lSpQoyMzOxb98+3Lt3D48fP9ao4+/vj2bNmqFr167Ys2cPEhMTsXPnTsTExBQ5/ny9e/eGg4MDgoKCcOzYMfz999/YvHkzTpw4IcWXmJiI+Ph43Lt3T+vA3D59+sDY2BjBwcG4cOECDhw4gKFDh6Jv377S5Tui1+H+/fto2bIlVq9ejXPnziExMREbN27E9OnT0aVLFwDyj6fx48fj559/RkREBP766y9cunQJ69atwzfffFPouIYMGYKMjAz06tULsbGxuHbtGlatWoUrV64A+G8AemRkJObOnYurV6/i/PnzWLFiBX744Ydi75MqVaogODgY/fv3R3R0NBITE3Hw4EFs2LABAODi4gKFQoEdO3bg7t27Uo/Y86pVq4YuXbpg0KBBOHr0KM6ePYv//e9/qFixorRfqfQwgSKdMTAwwJAhQzB9+nRkZWVh2rRp6Nq1K/r27Yt69eohISEBu3btgrW1dZHXoaenh3Xr1iEuLg5eXl4YOXIkZsyYIbudRo0a4dNPP0XPnj1ha2uL6dOna623efNmNGjQAL1790aNGjUwZsyYV/YMFYaRkRF2794NOzs7tG/fHrVq1VK7I6lr165o27YtWrRoAVtbW/zyyy8abZiammLXrl148OABGjRogG7duqFVq1aYP39+seMjksPc3By+vr6YNWsWmjVrBi8vL4wbNw6DBg1S+zzKOZ4CAwOxY8cO7N69Gw0aNEDDhg0xa9YsuLi4FDqu8uXLY//+/dJdrz4+Pli6dKl0yWvgwIFYtmwZVqxYgVq1asHf3x9RUVE66YECgEWLFqFbt274/PPP4enpiUGDBkk9chUrVkRERAS++uor2Nvba9w9m2/FihXw8fFBx44d4efnByEEfv/9d63juuj1UgheKCUiIiKShT1QRERERDIxgSIiIiKSiQkUERERkUxMoIiIiIhkYgJFREREJBMTKCIiIiKZmEARERERycQEioiIiEgmJlBEREREMjGBIqK3RkpKCoYPHw53d3cYGxvD3t4ejRs3xqJFi7T+3iERUVEZlHYARES68Pfff6Nx48YoV64cpk6dilq1akGpVOL8+fNYsmQJKlasiM6dO5fIunNycmBkZFQibRNR2cQeKCJ6K3z++ecwMDBAbGwsevTogerVq6Nq1aro0qULfvvtN3Tq1AkAkJaWhoEDB8LW1haWlpZo2bIlzp49K7UzceJE1K1bF6tWrUKVKlVgZWWFXr164dGjR1Kd5s2bY8iQIRgxYgQqVKiAwMBAAMCFCxfQrl07mJubw97eHn379sW9e/de744goteCCRQRvfHu37+P3bt3IzQ0FGZmZlrrKBQKAED37t2RmpqKnTt3Ii4uDvXq1UOrVq3w4MEDqe7169cRHR2NHTt2YMeOHTh06BCmTZum1t7KlSthZGSEY8eOYfHixUhLS0PLli3h7e2N2NhYxMTE4M6dO+jRo0fJbTgRlRpewiOiN15CQgKEEPDw8FArr1ChAp4+fQoACA0NRadOnXD69GmkpqZCqVQCAGbOnIno6Ghs2rQJgwcPBgCoVCpERUXBwsICANC3b1/s27cP3377rdR2tWrVMH36dGl6ypQp8Pb2xtSpU6Wyn376Cc7Ozrh69Sree++9ktl4IioVTKCI6K11+vRpqFQq9OnTB9nZ2Th79iwyMzNRvnx5tXpPnjzB9evXpekqVapIyRMAODo6IjU1VW0ZHx8ftemzZ8/iwIEDMDc314jj+vXrTKCI3jJMoIjojefu7g6FQoErV66olVetWhUAYGJiAgDIzMyEo6MjDh48qNFGuXLlpP8bGhqqzVMoFFCpVGplL14qzMzMRKdOnfDdd99ptO3o6FjobSGiNwMTKCJ645UvXx6tW7fG/PnzMXTo0ALHQdWrVw8pKSkwMDBAlSpVdBpDvXr1sHnzZlSpUgUGBjy1Er3tOIiciN4KCxcuRG5uLurXr4/169fj0qVLuHLlClavXo3Lly9DX18fAQEB8PPzQ1BQEHbv3o0bN27g+PHj+PrrrxEbG1us9YeGhuLBgwfo3bs3zpw5g+vXr2PXrl34+OOPkZeXp6OtJKKygl+TiOit4Obmhj///BNTp05FeHg4bt++DaVSiRo1amD06NH4/PPPoVAo8Pvvv+Prr7/Gxx9/jLt378LBwQHNmjWDvb19sdbv5OSEY8eO4csvv0SbNm2QnZ0NFxcXtG3bFnp6/K5K9LZRCCFEaQdBRERE9Cbh1yIiIiIimZhAEREREcnEBIqIiIhIJiZQRERERDIxgSIiIiKSiQkUERERkUxMoIiIiIhkYgJFREREJBMTKCIiIiKZmEARERERycQEioiIiEim/wfwAhIADNXsuwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Repeat but skip annotator-llm\n",
    "scifi_confidences = []\n",
    "romantic_confidences = []\n",
    "\n",
    "for sample in dataset['samples']:\n",
    "    if sample['genre'] == 'SCI':\n",
    "        sum_confidence = 0\n",
    "        count = 0\n",
    "        # Skip annotator-llm and recalculate average confidence\n",
    "        for annotator in ['annotator-1', 'annotator-2', 'annotator-3', 'annotator-4']:\n",
    "            if sample['annotations']['confidence'][annotator] is not None:\n",
    "                sum_confidence += sample['annotations']['confidence'][annotator]\n",
    "                count += 1\n",
    "        if count > 0: # Avoid division by zero\n",
    "            scifi_confidences.append(sum_confidence / count)\n",
    "    if sample['genre'] == 'ROM':\n",
    "        sum_confidence = 0\n",
    "        count = 0\n",
    "        # Skip annotator-llm and recalculate average confidence\n",
    "        for annotator in ['annotator-1', 'annotator-2', 'annotator-3', 'annotator-4']:\n",
    "            if sample['annotations']['confidence'][annotator] is not None:\n",
    "                sum_confidence += sample['annotations']['confidence'][annotator]\n",
    "                count += 1\n",
    "        if count > 0: # Avoid division by zero\n",
    "            romantic_confidences.append(sum_confidence / count)\n",
    "\n",
    "\n",
    "genre_df = pd.DataFrame({\n",
    "    'Genre': ['Science Fiction'] * len(scifi_confidences) + ['Romantic Fiction'] * len(romantic_confidences),\n",
    "    'Confidence': scifi_confidences + romantic_confidences\n",
    "})\n",
    "\n",
    "# Create a boxplot of the average confidence per genre going from 1.00 to 3.00\n",
    "plt.figure(figsize=(5, 5))\n",
    "boxplot_human = genre_df.boxplot(by='Genre', column='Confidence', grid=False)\n",
    "plt.suptitle('')\n",
    "# plt.title('Average confidence (1-3) per genre for human annotators')\n",
    "plt.ylabel('Average confidence')\n",
    "plt.xlabel('Genre')\n",
    "ax = plt.gca()\n",
    "ax.set_ylim(0.9, 3.1)\n",
    "plt.show()\n",
    "\n",
    "# Export the boxplot to a PDF file\n",
    "boxplot_human.get_figure().savefig('average_confidence_per_genre.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Done. We have successfully:\n",
    "\n",
    "1. Annotated a dataset of sentences with understatements using a LLM.\n",
    "2. Loaded human annotations\n",
    "3. Calculated Inter-Annotator Agreement metrics.\n",
    "4. Created a corpus of understatement annotations, complete with Inter-Annotator Agreement metrics and genre labels.\n",
    "5. Compared the genres to see how these affected annotator confidence."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "DH-venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}