doc/source/preprocessing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text preprocessing and basic text mining\n",
    "\n",
    "During text preprocessing, a corpus of documents is tokenized (i.e. the document strings are split into individual words, punctuation, numbers, etc.) and then these tokens can be transformed, filtered or annotated. The goal is to prepare the raw document texts in a way that makes it easier to perform eventual text mining and analysis methods in a later stage, e.g. by reducing noise in the dataset. The package tmtoolkit provides a rich set of tools for this purpose implemented as *corpus functions* in the [tmtoolkit.corpus](api.rst#tmtoolkit-corpus) module.\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reminder: Corpus functions**\n",
    "\n",
    "All *corpus functions* accept a [Corpus](api.rst#tmtoolkit.corpus.Corpus) object as first argument and operate on it. A corpus function may retrieve information from a corpus and/or modify the corpus object.\n",
    "\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optional: enabling logging output\n",
    "\n",
    "By default, tmtoolkit does not expose any internal logging messages. Sometimes, for example for diagnostic output during debugging or in order to see progress for long running operations, it's helpful to enable logging output display. For that, you can use the [enable_logging](api.rst#tmtoolkit.utils.enable_logging) function. By default, it enables logging to console for the `INFO` level."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:42.542921Z",
     "iopub.status.busy": "2023-05-03T13:22:42.542703Z",
     "iopub.status.idle": "2023-05-03T13:22:44.104586Z",
     "shell.execute_reply": "2023-05-03T13:22:44.104107Z"
    }
   },
   "outputs": [],
   "source": [
    "from tmtoolkit.utils import enable_logging\n",
    "\n",
    "enable_logging()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading example data\n",
    "\n",
    "Let's load a sample of ten documents from the built-in *NewsArticles* dataset. We'll use only a small number of documents here to have a better overview at the beginning. We can later use a larger sample. To apply sampling right at the beginning when loading the data, we pass the `sample=100` parameter to the [from_builtin_corpus](api.rst#tmtoolkit.corpus.Corpus.from_builtin_corpus) class method. We also use [print_summary](api.rst#tmtoolkit.corpus.print_summary) like shown in the previous chapter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:44.107111Z",
     "iopub.status.busy": "2023-05-03T13:22:44.106739Z",
     "iopub.status.idle": "2023-05-03T13:22:47.934948Z",
     "shell.execute_reply": "2023-05-03T13:22:47.934465Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-05-03 15:22:44,337:INFO:tmtoolkit:creating Corpus instance with no documents\n",
      "2023-05-03 15:22:44,338:INFO:tmtoolkit:using serial processing\n",
      "2023-05-03 15:22:44,564:INFO:tmtoolkit:sampling 100 documents(s) out of 3824\n",
      "2023-05-03 15:22:44,566:INFO:tmtoolkit:adding text from 100 documents(s)\n",
      "2023-05-03 15:22:44,566:INFO:tmtoolkit:running NLP pipeline on 100 documents\n",
      "2023-05-03 15:22:47,931:INFO:tmtoolkit:generating document texts\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Trump suggests Obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterates Govt support for Finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): President Trump says he has asked the Justice Depa...\n",
      "> NewsArticles-1119 (975 tokens): An amazing moment in history : Donald Trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland sees losses widening    Bai...\n",
      "> NewsArticles-1377 (774 tokens): Turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1185 (1271 tokens): For more than a week,-France - has been rocked by ...\n",
      "> NewsArticles-1263 (410 tokens): Russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-1353 (30 tokens): Islamic State battle : Fierce gunfight outside Mos...\n",
      "> NewsArticles-1387 (513 tokens): Protests after Anaheim policeman drags teen , fire...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 9223\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "random.seed(20220119)   # to make the sampling reproducible\n",
    "\n",
    "from tmtoolkit.corpus import Corpus, print_summary\n",
    "\n",
    "corpus_small = Corpus.from_builtin_corpus('en-NewsArticles', sample=100)\n",
    "print_summary(corpus_small)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The logging information was printed on red, the information below on white came from `print_summary`. We will disable logging again using [disable_logging](api.rst#tmtoolkit.utils.disable_logging):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:47.936671Z",
     "iopub.status.busy": "2023-05-03T13:22:47.936564Z",
     "iopub.status.idle": "2023-05-03T13:22:47.938997Z",
     "shell.execute_reply": "2023-05-03T13:22:47.938633Z"
    }
   },
   "outputs": [],
   "source": [
    "from tmtoolkit.utils import disable_logging\n",
    "\n",
    "disable_logging()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are the first 10 names of the documents that were loaded:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:47.940467Z",
     "iopub.status.busy": "2023-05-03T13:22:47.940370Z",
     "iopub.status.idle": "2023-05-03T13:22:47.944274Z",
     "shell.execute_reply": "2023-05-03T13:22:47.943927Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['NewsArticles-1100',\n",
       " 'NewsArticles-1119',\n",
       " 'NewsArticles-1185',\n",
       " 'NewsArticles-1263',\n",
       " 'NewsArticles-1353',\n",
       " 'NewsArticles-1377',\n",
       " 'NewsArticles-1387',\n",
       " 'NewsArticles-1407',\n",
       " 'NewsArticles-1472',\n",
       " 'NewsArticles-1515']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import doc_labels\n",
    "\n",
    "doc_labels(corpus_small)[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing tokens and token attributes\n",
    "\n",
    "We start with accessing the documents' tokens and their *token attributes* using [doc_tokens](api.rst#tmtoolkit.corpus.doc_tokens) and [tokens_table](api.rst#tmtoolkit.corpus.tokens_table). Token attributes are meta information attached to each token. These can be linguistic features, such as the Part of Speech (POS) tag, indicators for stopwords or punctuation, etc. The default attributes are a subset of [SpaCy's token attributes](https://spacy.io/api/token#attributes). You can configure which of these attributes are stored using the `spacy_token_attrs` parameter of the `Corpus` constructor. You can also add your own token attributes. This will be shown later on.\n",
    "\n",
    "At first we load the tokens along with their attributes via `doc_tokens`, which gives us a dictionary mapping document labels to document data. Each document data is another dictionary that contains the tokens and their attributes. We start by checking which token attributes are loaded by default in any document (here, we use \"NewsArticles-2433\"):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:47.945687Z",
     "iopub.status.busy": "2023-05-03T13:22:47.945589Z",
     "iopub.status.idle": "2023-05-03T13:22:47.977674Z",
     "shell.execute_reply": "2023-05-03T13:22:47.977132Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['token', 'is_punct', 'is_stop', 'like_num', 'tag', 'pos', 'lemma'])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import doc_tokens, tokens_table\n",
    "\n",
    "# with_attr=True adds default set of token attributes\n",
    "tok = doc_tokens(corpus_small, with_attr=True)\n",
    "tok['NewsArticles-2433'].keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So each document's data can be accessed like in the example above and it will contain the seven data entries listed above. The `'token'` entry gives the actual tokens of the document. Let's show the first five tokens for a document:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:47.979348Z",
     "iopub.status.busy": "2023-05-03T13:22:47.979244Z",
     "iopub.status.idle": "2023-05-03T13:22:47.982375Z",
     "shell.execute_reply": "2023-05-03T13:22:47.981954Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['DOJ', ':', '2', 'Russian', 'spies']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tok['NewsArticles-2433']['token'][:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The other entries are the attributes corresponding to each token. Here, we display the first five lemmata for the same document and the first five punctuation indicator values. The colon is correctly identified as punctuation character."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:47.984138Z",
     "iopub.status.busy": "2023-05-03T13:22:47.983780Z",
     "iopub.status.idle": "2023-05-03T13:22:47.986980Z",
     "shell.execute_reply": "2023-05-03T13:22:47.986628Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['doj', ':', '2', 'russian', 'spy']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tok['NewsArticles-2433']['lemma'][:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:47.988318Z",
     "iopub.status.busy": "2023-05-03T13:22:47.988181Z",
     "iopub.status.idle": "2023-05-03T13:22:47.990691Z",
     "shell.execute_reply": "2023-05-03T13:22:47.990352Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[False, True, False, False, False]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tok['NewsArticles-2433']['is_punct'][:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If your NLP pipeline performs sentence recognition, you can pass the parameter `sentences=True` which will add another level to the output representing sentences. This means that for each item like `'token'`, `'lemma'`, etc. we will get a list of sentences. For example, the following will print the tokens of the 8th sentence (index 7):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:47.992020Z",
     "iopub.status.busy": "2023-05-03T13:22:47.991846Z",
     "iopub.status.idle": "2023-05-03T13:22:48.032386Z",
     "shell.execute_reply": "2023-05-03T13:22:48.031995Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['A',\n",
       " 'Justice',\n",
       " 'Department',\n",
       " 'official',\n",
       " 'said',\n",
       " 'the',\n",
       " 'agency',\n",
       " 'has',\n",
       " 'not',\n",
       " 'confirmed',\n",
       " 'it',\n",
       " 'is',\n",
       " 'the',\n",
       " 'same',\n",
       " 'person',\n",
       " 'and',\n",
       " 'declined',\n",
       " 'further',\n",
       " 'comment',\n",
       " 'to',\n",
       " ...]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tok_sents = doc_tokens(corpus_small, sentences=True, with_attr=True)\n",
    "tok_sents['NewsArticles-2433']['token'][7]   # index 7 means 8th sentence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a more compact overview, it's better to use the [tokens_table](api.rst#tmtoolkit.corpus.tokens_table) function. This will generate a [pandas DataFrame](https://pandas.pydata.org/) from the documents in the corpus and it will by default include all token attributes, along with a column for the document label (`doc`) and the token position inside the document (`position`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.033982Z",
     "iopub.status.busy": "2023-05-03T13:22:48.033879Z",
     "iopub.status.idle": "2023-05-03T13:22:48.128083Z",
     "shell.execute_reply": "2023-05-03T13:22:48.127541Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>is_punct</th>\n",
       "      <th>is_stop</th>\n",
       "      <th>lemma</th>\n",
       "      <th>like_num</th>\n",
       "      <th>pos</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>President</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>President</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>Trump</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Trump</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>says</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>say</td>\n",
       "      <td>False</td>\n",
       "      <td>VERB</td>\n",
       "      <td>VBZ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>he</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>he</td>\n",
       "      <td>False</td>\n",
       "      <td>PRON</td>\n",
       "      <td>PRP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>has</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>have</td>\n",
       "      <td>False</td>\n",
       "      <td>AUX</td>\n",
       "      <td>VBZ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59593</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>282</td>\n",
       "      <td>priorities</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>priority</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NNS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59594</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>283</td>\n",
       "      <td>for</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>for</td>\n",
       "      <td>False</td>\n",
       "      <td>ADP</td>\n",
       "      <td>IN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59595</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>284</td>\n",
       "      <td>the</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>the</td>\n",
       "      <td>False</td>\n",
       "      <td>DET</td>\n",
       "      <td>DT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59596</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>285</td>\n",
       "      <td>nation</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>nation</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59597</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>286</td>\n",
       "      <td>.</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>.</td>\n",
       "      <td>False</td>\n",
       "      <td>PUNCT</td>\n",
       "      <td>.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59598 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position       token  is_punct  is_stop      lemma  \\\n",
       "0      NewsArticles-1100         0   President     False    False  President   \n",
       "1      NewsArticles-1100         1       Trump     False    False      Trump   \n",
       "2      NewsArticles-1100         2        says     False    False        say   \n",
       "3      NewsArticles-1100         3          he     False     True         he   \n",
       "4      NewsArticles-1100         4         has     False     True       have   \n",
       "...                  ...       ...         ...       ...      ...        ...   \n",
       "59593   NewsArticles-960       282  priorities     False    False   priority   \n",
       "59594   NewsArticles-960       283         for     False     True        for   \n",
       "59595   NewsArticles-960       284         the     False     True        the   \n",
       "59596   NewsArticles-960       285      nation     False    False     nation   \n",
       "59597   NewsArticles-960       286           .      True    False          .   \n",
       "\n",
       "       like_num    pos  tag  \n",
       "0         False  PROPN  NNP  \n",
       "1         False  PROPN  NNP  \n",
       "2         False   VERB  VBZ  \n",
       "3         False   PRON  PRP  \n",
       "4         False    AUX  VBZ  \n",
       "...         ...    ...  ...  \n",
       "59593     False   NOUN  NNS  \n",
       "59594     False    ADP   IN  \n",
       "59595     False    DET   DT  \n",
       "59596     False   NOUN   NN  \n",
       "59597     False  PUNCT    .  \n",
       "\n",
       "[59598 rows x 9 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tbl = tokens_table(corpus_small)\n",
    "tbl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use all sorts of filtering operations on this dataframe. See the [pandas documentation](https://pandas.pydata.org/docs/user_guide/indexing.html) for details. Here, we select all tokens that were identified as \"number-like\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.130105Z",
     "iopub.status.busy": "2023-05-03T13:22:48.129966Z",
     "iopub.status.idle": "2023-05-03T13:22:48.136831Z",
     "shell.execute_reply": "2023-05-03T13:22:48.136508Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>is_punct</th>\n",
       "      <th>is_stop</th>\n",
       "      <th>lemma</th>\n",
       "      <th>like_num</th>\n",
       "      <th>pos</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>288</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>64</td>\n",
       "      <td>fifteen</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>fifteen</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>320</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>96</td>\n",
       "      <td>one</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>one</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>328</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>104</td>\n",
       "      <td>four</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>four</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>759</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>535</td>\n",
       "      <td>100</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>100</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>787</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>563</td>\n",
       "      <td>four</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>four</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59253</th>\n",
       "      <td>NewsArticles-901</td>\n",
       "      <td>856</td>\n",
       "      <td>85</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>85</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59256</th>\n",
       "      <td>NewsArticles-901</td>\n",
       "      <td>859</td>\n",
       "      <td>9</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>9</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59374</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>63</td>\n",
       "      <td>2021</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>2021</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59400</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>89</td>\n",
       "      <td>2010</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>2010</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59413</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>102</td>\n",
       "      <td>1,550</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>1,550</td>\n",
       "      <td>True</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1139 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position    token  is_punct  is_stop    lemma  \\\n",
       "288    NewsArticles-1119        64  fifteen     False     True  fifteen   \n",
       "320    NewsArticles-1119        96      one     False     True      one   \n",
       "328    NewsArticles-1119       104     four     False     True     four   \n",
       "759    NewsArticles-1119       535      100     False    False      100   \n",
       "787    NewsArticles-1119       563     four     False     True     four   \n",
       "...                  ...       ...      ...       ...      ...      ...   \n",
       "59253   NewsArticles-901       856       85     False    False       85   \n",
       "59256   NewsArticles-901       859        9     False    False        9   \n",
       "59374   NewsArticles-960        63     2021     False    False     2021   \n",
       "59400   NewsArticles-960        89     2010     False    False     2010   \n",
       "59413   NewsArticles-960       102    1,550     False    False    1,550   \n",
       "\n",
       "       like_num  pos tag  \n",
       "288        True  NUM  CD  \n",
       "320        True  NUM  CD  \n",
       "328        True  NUM  CD  \n",
       "759        True  NUM  CD  \n",
       "787        True  NUM  CD  \n",
       "...         ...  ...  ..  \n",
       "59253      True  NUM  CD  \n",
       "59256      True  NUM  CD  \n",
       "59374      True  NUM  CD  \n",
       "59400      True  NUM  CD  \n",
       "59413      True  NUM  CD  \n",
       "\n",
       "[1139 rows x 9 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tbl[tbl.like_num]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This however only filters the table *output*. We will later see how to filter corpus documents and tokens.\n",
    "\n",
    "If you want to generate the table only for a subset of documents, you can use the `select` parameter and provide one or more document labels. Similar to that, you can use the `with_attr` parameter to list only a subset of the token attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.138444Z",
     "iopub.status.busy": "2023-05-03T13:22:48.138288Z",
     "iopub.status.idle": "2023-05-03T13:22:48.145184Z",
     "shell.execute_reply": "2023-05-03T13:22:48.144863Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>sent</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>pos</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>DOJ</td>\n",
       "      <td>NOUN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>:</td>\n",
       "      <td>PUNCT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>NUM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Russian</td>\n",
       "      <td>ADJ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>spies</td>\n",
       "      <td>NOUN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>837</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>27</td>\n",
       "      <td>837</td>\n",
       "      <td>to</td>\n",
       "      <td>PART</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>838</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>27</td>\n",
       "      <td>838</td>\n",
       "      <td>reflect</td>\n",
       "      <td>VERB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>839</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>27</td>\n",
       "      <td>839</td>\n",
       "      <td>new</td>\n",
       "      <td>ADJ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>840</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>27</td>\n",
       "      <td>840</td>\n",
       "      <td>developments</td>\n",
       "      <td>NOUN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>841</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>27</td>\n",
       "      <td>841</td>\n",
       "      <td>.</td>\n",
       "      <td>PUNCT</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>842 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   doc  sent  position         token    pos\n",
       "0    NewsArticles-2433     0         0           DOJ   NOUN\n",
       "1    NewsArticles-2433     0         1             :  PUNCT\n",
       "2    NewsArticles-2433     0         2             2    NUM\n",
       "3    NewsArticles-2433     0         3       Russian    ADJ\n",
       "4    NewsArticles-2433     0         4         spies   NOUN\n",
       "..                 ...   ...       ...           ...    ...\n",
       "837  NewsArticles-2433    27       837            to   PART\n",
       "838  NewsArticles-2433    27       838       reflect   VERB\n",
       "839  NewsArticles-2433    27       839           new    ADJ\n",
       "840  NewsArticles-2433    27       840  developments   NOUN\n",
       "841  NewsArticles-2433    27       841             .  PUNCT\n",
       "\n",
       "[842 rows x 5 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# select a single document and only show the \"pos\" attribute (coarse POS tag)\n",
    "tokens_table(corpus_small, select='NewsArticles-2433', sentences=True, with_attr='pos')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.146696Z",
     "iopub.status.busy": "2023-05-03T13:22:48.146562Z",
     "iopub.status.idle": "2023-05-03T13:22:48.154807Z",
     "shell.execute_reply": "2023-05-03T13:22:48.154532Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>pos</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>0</td>\n",
       "      <td>DOJ</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>1</td>\n",
       "      <td>:</td>\n",
       "      <td>PUNCT</td>\n",
       "      <td>:</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>NUM</td>\n",
       "      <td>CD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>3</td>\n",
       "      <td>Russian</td>\n",
       "      <td>ADJ</td>\n",
       "      <td>JJ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>4</td>\n",
       "      <td>spies</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NNS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949</th>\n",
       "      <td>NewsArticles-49</td>\n",
       "      <td>1107</td>\n",
       "      <td>fight</td>\n",
       "      <td>VERB</td>\n",
       "      <td>VB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1950</th>\n",
       "      <td>NewsArticles-49</td>\n",
       "      <td>1108</td>\n",
       "      <td>to</td>\n",
       "      <td>PART</td>\n",
       "      <td>TO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1951</th>\n",
       "      <td>NewsArticles-49</td>\n",
       "      <td>1109</td>\n",
       "      <td>defend</td>\n",
       "      <td>VERB</td>\n",
       "      <td>VB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1952</th>\n",
       "      <td>NewsArticles-49</td>\n",
       "      <td>1110</td>\n",
       "      <td>it</td>\n",
       "      <td>PRON</td>\n",
       "      <td>PRP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1953</th>\n",
       "      <td>NewsArticles-49</td>\n",
       "      <td>1111</td>\n",
       "      <td>.</td>\n",
       "      <td>PUNCT</td>\n",
       "      <td>.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1954 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                    doc  position    token    pos  tag\n",
       "0     NewsArticles-2433         0      DOJ   NOUN   NN\n",
       "1     NewsArticles-2433         1        :  PUNCT    :\n",
       "2     NewsArticles-2433         2        2    NUM   CD\n",
       "3     NewsArticles-2433         3  Russian    ADJ   JJ\n",
       "4     NewsArticles-2433         4    spies   NOUN  NNS\n",
       "...                 ...       ...      ...    ...  ...\n",
       "1949    NewsArticles-49      1107    fight   VERB   VB\n",
       "1950    NewsArticles-49      1108       to   PART   TO\n",
       "1951    NewsArticles-49      1109   defend   VERB   VB\n",
       "1952    NewsArticles-49      1110       it   PRON  PRP\n",
       "1953    NewsArticles-49      1111        .  PUNCT    .\n",
       "\n",
       "[1954 rows x 5 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# select two documents and only show the \"pos\" and \"tag\" attributes\n",
    "# (coarse and detailed POS tags)\n",
    "tokens_table(corpus_small, select=['NewsArticles-2433', 'NewsArticles-49'],\n",
    "             with_attr=['pos', 'tag'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Side note: Common corpus function parameters**\n",
    "    \n",
    "Many corpus functions share the same parameter names and when they do, they implicate the same behavior. As already explained, all corpus functions accept a `Corpus` object as first parameter. But next to that, many corpus functions also accept a `select` parameter, which can always be used to specify a subset of the documents to which the respective function is applied. We also already got to know the `sentences` parameter that some corpus functions accept in order to also represent the sentence structure of a document in their output.\n",
    "    \n",
    "To know which functions accept which parameter, check their documentation.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Corpus vocabulary\n",
    "\n",
    "The corpus *vocabulary* is the set of unique tokens (usually refered to as *token types*) in a corpus. We can get that set via [vocabulary](api.rst#tmtoolkit.corpus.vocabulary). By default, the set is converted to a sorted list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.156294Z",
     "iopub.status.busy": "2023-05-03T13:22:48.156192Z",
     "iopub.status.idle": "2023-05-03T13:22:48.160888Z",
     "shell.execute_reply": "2023-05-03T13:22:48.160591Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['\\n\\n',\n",
       " '\\n\\n ',\n",
       " ' ',\n",
       " '  ',\n",
       " '   ',\n",
       " '    ',\n",
       " '!',\n",
       " '\"',\n",
       " '\"-',\n",
       " '\"?\"But',\n",
       " '\"?\"I',\n",
       " '\"?\"hiba',\n",
       " '\"?\"the',\n",
       " '\"?Although',\n",
       " '\"?And',\n",
       " '\"?Depending',\n",
       " '\"?During',\n",
       " '\"?For',\n",
       " '\"?House',\n",
       " '\"?Indo',\n",
       " ...]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import vocabulary\n",
    "\n",
    "vocabulary(corpus_small)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The parameter setting `sort=False` results in emitting a Python *set.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This corpus function also accepts a `select` parameter. To get the sorted vocabulary for document \"NewsArticles-2433\", we can write:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.162457Z",
     "iopub.status.busy": "2023-05-03T13:22:48.162344Z",
     "iopub.status.idle": "2023-05-03T13:22:48.165313Z",
     "shell.execute_reply": "2023-05-03T13:22:48.165020Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['\\n\\n',\n",
       " '\"',\n",
       " \"'s\",\n",
       " '(',\n",
       " ')',\n",
       " ',',\n",
       " '-',\n",
       " '--',\n",
       " '.',\n",
       " '2',\n",
       " '2014',\n",
       " '22',\n",
       " '29',\n",
       " '33',\n",
       " '43',\n",
       " '500',\n",
       " ':',\n",
       " 'A',\n",
       " 'Akehmet',\n",
       " 'Aleksandrovich',\n",
       " ...]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocabulary(corpus_small, select='NewsArticles-2433')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get the number of unique tokens in the corpus, i.e. the vocabulary size, we can use [vocabulary_size](api.rst#tmtoolkit.corpus.vocabulary_size), which is basically a shortcut for `len(vocabulary(<Corpus object>))`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.167003Z",
     "iopub.status.busy": "2023-05-03T13:22:48.166871Z",
     "iopub.status.idle": "2023-05-03T13:22:48.169441Z",
     "shell.execute_reply": "2023-05-03T13:22:48.169191Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9223"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import vocabulary_size\n",
    "\n",
    "vocabulary_size(corpus_small)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The corpus function [vocabulary_counts](api.rst#tmtoolkit.corpus.vocabulary_size) is useful to find out how often each token in the vocabulary occurs in the corpus:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.171206Z",
     "iopub.status.busy": "2023-05-03T13:22:48.170980Z",
     "iopub.status.idle": "2023-05-03T13:22:48.181512Z",
     "shell.execute_reply": "2023-05-03T13:22:48.181231Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ID': 1,\n",
       " 'agent': 6,\n",
       " 'cop': 1,\n",
       " 'mark': 1,\n",
       " 'number': 18,\n",
       " 'passion': 1,\n",
       " 'unable': 1,\n",
       " 'Where': 2,\n",
       " 'types.-': 1,\n",
       " 'Closer': 1,\n",
       " 'Reflection': 1,\n",
       " 'approach': 8,\n",
       " 'users': 10,\n",
       " 'average': 11,\n",
       " 'designed': 2,\n",
       " 'geared': 1,\n",
       " 'stream': 3,\n",
       " 'sites': 2,\n",
       " 'deportation': 4,\n",
       " 'discourse': 1,\n",
       " ...}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import vocabulary_counts\n",
    "\n",
    "vocabulary_counts(corpus_small)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you don't want to obtain absolute counts, you can use the `proportions` parameter. Setting it to `1` gives you ordinary proportions (i.e. $\\frac{x_i}{\\sum_j x_j}$) and `2` gives you these proportions on a log10 scale."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.183092Z",
     "iopub.status.busy": "2023-05-03T13:22:48.182937Z",
     "iopub.status.idle": "2023-05-03T13:22:48.192993Z",
     "shell.execute_reply": "2023-05-03T13:22:48.192625Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ID': 1.6779086546528407e-05,\n",
       " 'agent': 0.00010067451927917044,\n",
       " 'cop': 1.6779086546528407e-05,\n",
       " 'mark': 1.6779086546528407e-05,\n",
       " 'number': 0.0003020235578375113,\n",
       " 'passion': 1.6779086546528407e-05,\n",
       " 'unable': 1.6779086546528407e-05,\n",
       " 'Where': 3.3558173093056814e-05,\n",
       " 'types.-': 1.6779086546528407e-05,\n",
       " 'Closer': 1.6779086546528407e-05,\n",
       " 'Reflection': 1.6779086546528407e-05,\n",
       " 'approach': 0.00013423269237222726,\n",
       " 'users': 0.00016779086546528407,\n",
       " 'average': 0.00018456995201181247,\n",
       " 'designed': 3.3558173093056814e-05,\n",
       " 'geared': 1.6779086546528407e-05,\n",
       " 'stream': 5.033725963958522e-05,\n",
       " 'sites': 3.3558173093056814e-05,\n",
       " 'deportation': 6.711634618611363e-05,\n",
       " 'discourse': 1.6779086546528407e-05,\n",
       " ...}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab_proportions = vocabulary_counts(corpus_small, proportions=1)\n",
    "vocab_proportions   # will reuse that later"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tabular output is often more convenient for displaying results. You can set the `as_table` parameter to `True` to get a dataframe of tokens and their frequency. You can also specify to sort the dataframe by specifying the column to sort by in the `as_table` parameter. By default, this will sort in ascending order, but if you prefix the column name by \"-\", you obtain a descending sort order. Here, we will get a table of tokens with their frequencies in descending order:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.194806Z",
     "iopub.status.busy": "2023-05-03T13:22:48.194617Z",
     "iopub.status.idle": "2023-05-03T13:22:48.215865Z",
     "shell.execute_reply": "2023-05-03T13:22:48.215572Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>token</th>\n",
       "      <th>freq</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3742</th>\n",
       "      <td>the</td>\n",
       "      <td>2670</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1339</th>\n",
       "      <td>,</td>\n",
       "      <td>2426</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6325</th>\n",
       "      <td>.</td>\n",
       "      <td>2175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7933</th>\n",
       "      <td>\"</td>\n",
       "      <td>1417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>457</th>\n",
       "      <td>of</td>\n",
       "      <td>1387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4062</th>\n",
       "      <td>colours</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4059</th>\n",
       "      <td>foolish</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4056</th>\n",
       "      <td>59</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4053</th>\n",
       "      <td>shift</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9222</th>\n",
       "      <td>ageing</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9223 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        token  freq\n",
       "3742      the  2670\n",
       "1339        ,  2426\n",
       "6325        .  2175\n",
       "7933        \"  1417\n",
       "457        of  1387\n",
       "...       ...   ...\n",
       "4062  colours     1\n",
       "4059  foolish     1\n",
       "4056       59     1\n",
       "4053    shift     1\n",
       "9222   ageing     1\n",
       "\n",
       "[9223 rows x 2 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocabulary_counts(corpus_small, as_table='-freq')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Common parameter `as_table`**\n",
    "    \n",
    "Just like `select` or `sentences`, the `as_table` parameter is also a common parameter available for many corpus functions, e.g. [doc_lengths](api.rst#tmtoolkit.corpus.doc_lengths), [doc_num_sents](api.rst#tmtoolkit.corpus.doc_num_sents) or [doc_texts](api.rst#tmtoolkit.corpus.doc_texts).\n",
    "\n",
    "</div>\n",
    "\n",
    "We can see that \"the\" and \"to\" are top-ranking tokens, along with some punctuation characters. We can check the share of tokens for \"the\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.217488Z",
     "iopub.status.busy": "2023-05-03T13:22:48.217378Z",
     "iopub.status.idle": "2023-05-03T13:22:48.220231Z",
     "shell.execute_reply": "2023-05-03T13:22:48.219837Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.04480016107923085"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab_proportions['the']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the token \"the\" occurs more the 4% of the time in the whole corpus."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing corpus summary statistics\n",
    "\n",
    "There are several functions for visualizing summary statistics of corpora which are implemented in the [corpus.visualize](api.rst#module-tmtoolkit.corpus.visualize) module. These are especially useful to see how certain processing steps influence summary statistics like token distributions and document length in a corpus. We will start with a few visualizations for the current corpus and can later compare these with plots generated after text processing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's import the plotting functions that we'll use. We also need to import `matplotlib.pyplot` in order to generate a Figure and an Axes object on which the actual plot is drawn. Most plotting functions in tmtoolkit work this way that you need to pass these two objects. This allows for full flexibility since you can adjust the plot before and after applying the plotting function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.222246Z",
     "iopub.status.busy": "2023-05-03T13:22:48.221922Z",
     "iopub.status.idle": "2023-05-03T13:22:48.224204Z",
     "shell.execute_reply": "2023-05-03T13:22:48.223870Z"
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from tmtoolkit.corpus.visualize import (plot_doc_lengths_hist,\n",
    "    plot_doc_frequencies_hist, plot_ranked_vocab_counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "    \n",
    "**Side Note: So much `from tmtoolkit.corpus import ...`**\n",
    "    \n",
    "You'll see a lot of import statements from the `tmtoolkit.corpus` module in this chapter, because all corpus functions are defined in this module. In this manual, I like to explicitly point out from where to import an object (like a function) and only import those objects that I actually need. However, it's completely fine to make a wildcard import `from tmtoolkit.corpus import *` at the beginning of your own code so that all objects in that module are directly available. An alternative approach is to import the corpus module with a short alias name, e.g. `import tmtoolkit.corpus as crp`. Then, you can access all objects in that module via `crp.<...>`.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll use [plot_doc_lengths_hist](api.rst#tmtoolkit.corpus.visualize.plot_doc_lengths_hist) to show the distribution of document lengths (i.e. number of tokens in each document) in our corpus. By default, the y-axis uses a log10 scale which is useful for medium and large scaled corpora, but since our corpus is so small we'll use a linear scale instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.225740Z",
     "iopub.status.busy": "2023-05-03T13:22:48.225485Z",
     "iopub.status.idle": "2023-05-03T13:22:48.299911Z",
     "shell.execute_reply": "2023-05-03T13:22:48.299351Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAHHCAYAAACle7JuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA2/UlEQVR4nO3deVyVZf7/8fdB4YCyibKIC+CSmuuEG+UeiaamZWWO04hpmrmMaVqOk8o3G0q/pVPjOpU4TVnpTDqVmeaCZmqjuYwthIZlKWoq4goq1++Pfp5vR8AFkcOFr+fjcR5yX/d17vtz39fh+OZeznEYY4wAAAAs5OXpAgAAAIqKIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgg5tGdHS0EhMTPV1GmTdt2jTVqlVL5cqVU7Nmza75+R06dFCHDh2Kva6bUWJioqKjoz1dxhWlpKTI4XBoy5Ytni4FFiLIwEpXeuPr0KGDGjVqdN3rWbZsmSZPnnzdy7lZrFixQuPGjdMdd9yh+fPn689//rOnSypTPvvsM02ePFlZWVmeLqVIZs2apZSUFE+XgTKmvKcLAEpKWlqavLyuLbsvW7ZMM2fOJMxcpdWrV8vLy0uvvfaafHx8PF1OmfPZZ58pKSlJiYmJCg4O9nQ512zWrFmqUqUKR0ZRrDgig5uG0+mUt7e3p8u4JqdOnfJ0Cdfk0KFD8vPzI8QAKDEEGdw0Lr1G5ty5c0pKSlLdunXl6+urypUrq02bNlq5cqWkX64vmDlzpiTJ4XC4HhedOnVKY8aMUY0aNeR0OlWvXj397//+ry79QvkzZ85o5MiRqlKligICAnTPPffop59+ksPhcDvSM3nyZDkcDn311Vf67W9/q0qVKqlNmzaSpJ07dyoxMVG1atWSr6+vIiIi9Mgjj+jIkSNu67q4jG+//Va/+93vFBQUpNDQUD3zzDMyxmjfvn3q2bOnAgMDFRERoRdffPGq9t358+f17LPPqnbt2nI6nYqOjtYf//hH5eTkuPo4HA7Nnz9fp06dcu2rK51GmDdvnmrXri0/Pz+1bNlS69evL7DfoUOHNHDgQIWHh8vX11dNmzbVggUL8vXLy8vTX/7yFzVu3Fi+vr4KDQ1Vly5dXKcg9+7dW2hdhY3H9ezLnJwcTZo0SXXq1JHT6VSNGjU0btw4t/12cd3Dhw/XkiVL1KhRIzmdTjVs2FDLly93q2fs2LGSpJiYGNc+3rt372X3cUH7aMaMGWrYsKF8fX0VHh6uIUOG6NixY279oqOj1b17d3366adq2bKlfH19VatWLf3973/Pt8ydO3eqffv28vPzU/Xq1TVlyhTNnz/frb7o6Gh9+eWXSk1NddV+6bVQOTk5Gj16tEJDQ1WxYkXde++9Onz4sFufLVu2KCEhQVWqVJGfn59iYmL0yCOPXNM+QNnCqSVY7fjx4/r555/ztZ87d+6Kz508ebKSk5M1aNAgtWzZUtnZ2dqyZYu++OIL3XXXXRoyZIj279+vlStX6o033nB7rjFG99xzj9asWaOBAweqWbNm+vjjjzV27Fj99NNPmj59uqtvYmKi3n33XT388MNq3bq1UlNT1a1bt0LreuCBB1S3bl39+c9/doWilStX6rvvvtOAAQMUERGhL7/8UvPmzdOXX36pTZs2uQUsSerTp48aNGig559/Xh9++KGmTJmikJAQzZ07V506ddILL7ygN998U08++aRatGihdu3aXXZfDRo0SAsWLND999+vMWPGaPPmzUpOTtbXX3+t9957T5L0xhtvaN68efr888/16quvSpJuv/32Qpf52muvaciQIbr99ts1atQofffdd7rnnnsUEhKiGjVquPqdOXNGHTp00O7duzV8+HDFxMRo0aJFSkxMVFZWlv7whz+4+g4cOFApKSnq2rWrBg0apPPnz2v9+vXatGmTmjdvftltLExR92VeXp7uueceffrppxo8eLAaNGig//73v5o+fbq+/fZbLVmyxG09n376qf71r3/p8ccfV0BAgF5++WX17t1bP/zwgypXrqz77rtP3377rRYuXKjp06erSpUqkqTQ0NBr2p4hQ4YoJSVFAwYM0MiRI5WRkaG//vWv2rZtmzZs2OB21HL37t26//77NXDgQPXv31+vv/66EhMTFRsbq4YNG0qSfvrpJ3Xs2FEOh0Pjx49XxYoV9eqrr8rpdLqtd8aMGRoxYoT8/f01YcIESVJ4eLhbnxEjRqhSpUqaNGmS9u7dqxkzZmj48OF65513JP0SaDt37qzQ0FA9/fTTCg4O1t69e/Wvf/3rmvYByhgDWGj+/PlG0mUfDRs2dHtOVFSU6d+/v2u6adOmplu3bpddz7Bhw0xBvyZLliwxksyUKVPc2u+//37jcDjM7t27jTHGbN261Ugyo0aNcuuXmJhoJJlJkya52iZNmmQkmb59++Zb3+nTp/O1LVy40Egy69aty7eMwYMHu9rOnz9vqlevbhwOh3n++edd7ceOHTN+fn5u+6Qg27dvN5LMoEGD3NqffPJJI8msXr3a1da/f39TsWLFyy7PGGNyc3NNWFiYadasmcnJyXG1z5s3z0gy7du3d7XNmDHDSDL/+Mc/3J4fFxdn/P39TXZ2tjHGmNWrVxtJZuTIkfnWl5eXZ4wxJiMjw0gy8+fPz9ensPEo6r584403jJeXl1m/fr3beubMmWMkmQ0bNrit28fHx/W6McaYHTt2GEnmlVdecbVNmzbNSDIZGRn56i9I//79TVRUlGt6/fr1RpJ588033fotX748X3tUVFS+19ehQ4eM0+k0Y8aMcbWNGDHCOBwOs23bNlfbkSNHTEhISL5aGzZs6Da2F138fY6Pj3eNlTHGPPHEE6ZcuXImKyvLGGPMe++9ZySZ//znP1e1/bg5cGoJVps5c6ZWrlyZ79GkSZMrPjc4OFhffvml0tPTr3m9y5YtU7ly5TRy5Ei39jFjxsgYo48++kiSXKcGHn/8cbd+I0aMKHTZjz32WL42Pz8/189nz57Vzz//rNatW0uSvvjii3z9Bw0a5Pq5XLlyat68uYwxGjhwoKs9ODhY9erV03fffVdoLdIv2ypJo0ePdmsfM2aMJOnDDz+87PMLsmXLFh06dEiPPfaY2/U0iYmJCgoKyrf+iIgI9e3b19Xm7e2tkSNH6uTJk0pNTZUk/fOf/5TD4dCkSZPyre/SI1bXoqj7ctGiRWrQoIHq16+vn3/+2fXo1KmTJGnNmjVu64mPj1ft2rVd002aNFFgYOAVx+daLFq0SEFBQbrrrrvcaoqNjZW/v3++mm699Va1bdvWNR0aGppvO5cvX664uDi3W+1DQkLUr1+/a65v8ODBbmPVtm1bXbhwQd9//70kuS5w/uCDD67qqCtuDpxagtVatmxZ4CmDSpUqFXjK6df+53/+Rz179tQtt9yiRo0aqUuXLnr44YevKgR9//33ioyMVEBAgFt7gwYNXPMv/uvl5aWYmBi3fnXq1Cl02Zf2laSjR48qKSlJb7/9tg4dOuQ27/jx4/n616xZ0206KChIvr6+rtMRv26/9DqbS13chktrjoiIUHBwsGtbr8XF59StW9et3dvbW7Vq1crXt27duvnuOLt0X+/Zs0eRkZEKCQm55noup6j7Mj09XV9//XWhp34uHcdL1yP98jq+9NqV65Genq7jx48rLCys2Gr6/vvvFRcXl6/f5V7jhbl0fZUqVZIk1/rat2+v3r17KykpSdOnT1eHDh3Uq1cv/fa3v813Kgs3D4IMblrt2rXTnj17tHTpUq1YsUKvvvqqpk+frjlz5rj9FV7Sfn305aIHH3xQn332mcaOHatmzZrJ399feXl56tKli/Ly8vL1L1eu3FW1Scp3cXJhrueoRmlR2DZcuHCh0OcUdV/m5eWpcePGeumllwrs++vrgK52mdcrLy9PYWFhevPNNwucf2noKomarmV9DodDixcv1qZNm/T+++/r448/1iOPPKIXX3xRmzZtkr+//w2pC6UbQQY3tZCQEA0YMEADBgzQyZMn1a5dO02ePNkVZAr7jy8qKkqffPKJTpw44XZU5ptvvnHNv/hvXl6eMjIy3I4+7N69+6prPHbsmFatWqWkpCRNnDjR1V6UU2JFcXEb0tPTXUdBJOngwYPKyspybeu1LlP6ZRsunmqRfrlIOyMjQ02bNnXru3PnTuXl5bkdlbl0X9euXVsff/yxjh49WuhRmYt/4V/6gXJFOap0JbVr19aOHTt05513FlsIvN7l1K5dW5988onuuOOOAgNzUURFRRX4ei6orbj2Q+vWrdW6dWs999xzeuutt9SvXz+9/fbbHv0DBJ7DNTK4aV16SsXf31916tRxuzW2YsWKkvL/x3f33XfrwoUL+utf/+rWPn36dDkcDnXt2lWSlJCQIOmXDwL7tVdeeeWq67z4V+qlfwXPmDHjqpdxPe6+++4C13fxSMPl7sAqTPPmzRUaGqo5c+YoNzfX1Z6SklLgvs7MzHTduSL9cjv4K6+8In9/f7Vv316S1Lt3bxljlJSUlG99F/ddYGCgqlSponXr1rnNv3R8isODDz6on376SX/729/yzTtz5kyRPiOosNfjtdR04cIFPfvss/nmnT9/vkjLTUhI0MaNG7V9+3ZX29GjRws86lOxYsXr+lTiY8eO5fs9uHhtzqW3tOPmwREZ3LRuvfVWdejQQbGxsQoJCdGWLVu0ePFiDR8+3NUnNjZWkjRy5EglJCSoXLlyeuihh9SjRw917NhREyZM0N69e9W0aVOtWLFCS5cu1ahRo1wXbcbGxqp3796aMWOGjhw54rr9+ttvv5V0dX+hBgYGql27dpo6darOnTunatWqacWKFcrIyLgBeyW/pk2bqn///po3b56ysrLUvn17ff7551qwYIF69eqljh07XvMyvb29NWXKFA0ZMkSdOnVSnz59lJGRofnz5+e7Rmbw4MGaO3euEhMTtXXrVkVHR2vx4sXasGGDZsyY4Toi1rFjRz388MN6+eWXlZ6e7jrttn79enXs2NE1roMGDdLzzz+vQYMGqXnz5lq3bp1rPIrTww8/rHfffVePPfaY1qxZozvuuEMXLlzQN998o3fffVcff/zxNd8SfvH1OGHCBD300EPy9vZWjx49XAHnStq3b68hQ4YoOTlZ27dvV+fOneXt7a309HQtWrRIf/nLX3T//fdfU03jxo3TP/7xD911110aMWKE6/brmjVr6ujRo26v8djYWM2ePVtTpkxRnTp1FBYW5nZE7koWLFigWbNm6d5771Xt2rV14sQJ/e1vf1NgYKArcOMm5JmbpYDrc/F2zcJuw2zfvv0Vb7+eMmWKadmypQkODjZ+fn6mfv365rnnnjO5ubmuPufPnzcjRowwoaGhxuFwuN2KfeLECfPEE0+YyMhI4+3tberWrWumTZvmdvuoMcacOnXKDBs2zISEhBh/f3/Tq1cvk5aWZiS53cJ78Xbfw4cP59ueH3/80dx7770mODjYBAUFmQceeMDs37+/0FuGL11GYbdFF7SfCnLu3DmTlJRkYmJijLe3t6lRo4YZP368OXv27FWtpzCzZs0yMTExxul0mubNm5t169aZ9u3b57tF9+DBg2bAgAGmSpUqxsfHxzRu3LjAW6jPnz9vpk2bZurXr298fHxMaGio6dq1q9m6daurz+nTp83AgQNNUFCQCQgIMA8++KA5dOjQDdmXubm55oUXXjANGzY0TqfTVKpUycTGxpqkpCRz/PhxVz9JZtiwYfmWeelr1hhjnn32WVOtWjXj5eV1xVuxL739+qJ58+aZ2NhY4+fnZwICAkzjxo3NuHHjzP79+93WXdDHExQ0Ptu2bTNt27Y1TqfTVK9e3SQnJ5uXX37ZSDKZmZmufpmZmaZbt24mICDA7Tb7wn6f16xZYySZNWvWGGOM+eKLL0zfvn1NzZo1jdPpNGFhYaZ79+5my5Ythe4DlH0OY27QVVsACrV9+3b95je/0T/+8Y8i3aYKlHajRo3S3LlzdfLkyUIv4gWKA9fIADfYmTNn8rXNmDFDXl5eV/xEXcAGl77Gjxw5ojfeeENt2rQhxOCG4xoZ4AabOnWqtm7dqo4dO6p8+fL66KOP9NFHH2nw4MH5bsEFbBQXF6cOHTqoQYMGOnjwoF577TVlZ2frmWee8XRpuAlwagm4wVauXKmkpCR99dVXOnnypGrWrKmHH35YEyZMUPny/C0B+/3xj3/U4sWL9eOPP8rhcOi2227TpEmTFB8f7+nScBMgyAAAAGtxjQwAALAWQQYAAFirzJ+gz8vL0/79+xUQEFAmvisGAICbgTFGJ06cUGRkZL4vjf21Mh9k9u/fz50hAABYat++fapevXqh88t8kLn48eX79u1TYGCgh6sBAABXIzs7WzVq1HD7Yt6ClPkgc/F0UmBgIEEGAADLXOmyEC72BQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALAWQQYAAFirvKcLQMmLfvpDT5dwU9j7fDdPlwAAZR5HZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALAWQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoeDTLJyclq0aKFAgICFBYWpl69eiktLc2tz9mzZzVs2DBVrlxZ/v7+6t27tw4ePOihigEAQGni0SCTmpqqYcOGadOmTVq5cqXOnTunzp0769SpU64+TzzxhN5//30tWrRIqamp2r9/v+677z4PVg0AAEqL8p5c+fLly92mU1JSFBYWpq1bt6pdu3Y6fvy4XnvtNb311lvq1KmTJGn+/Plq0KCBNm3apNatW3uibAAAUEqUqmtkjh8/LkkKCQmRJG3dulXnzp1TfHy8q0/9+vVVs2ZNbdy40SM1AgCA0sOjR2R+LS8vT6NGjdIdd9yhRo0aSZIyMzPl4+Oj4OBgt77h4eHKzMwscDk5OTnKyclxTWdnZ9+wmgEAgGeVmiMyw4YN065du/T2229f13KSk5MVFBTketSoUaOYKgQAAKVNqQgyw4cP1wcffKA1a9aoevXqrvaIiAjl5uYqKyvLrf/BgwcVERFR4LLGjx+v48ePux779u27kaUDAAAP8miQMcZo+PDheu+997R69WrFxMS4zY+NjZW3t7dWrVrlaktLS9MPP/yguLi4ApfpdDoVGBjo9gAAAGWTR6+RGTZsmN566y0tXbpUAQEBrutegoKC5Ofnp6CgIA0cOFCjR49WSEiIAgMDNWLECMXFxXHHEgAA8GyQmT17tiSpQ4cObu3z589XYmKiJGn69Ony8vJS7969lZOTo4SEBM2aNauEKwUAAKWRR4OMMeaKfXx9fTVz5kzNnDmzBCoCAAA2KRUX+wIAABQFQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWMujX1Fgu+inP/R0CQAA3NQ4IgMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALAWQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALBWeU8XAKD0iH76Q0+XcM32Pt/N0yUA8CCOyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtTwaZNatW6cePXooMjJSDodDS5YscZufmJgoh8Ph9ujSpYtnigUAAKWOR4PMqVOn1LRpU82cObPQPl26dNGBAwdcj4ULF5ZghQAAoDTz6JdGdu3aVV27dr1sH6fTqYiIiBKqCAAA2KTUXyOzdu1ahYWFqV69eho6dKiOHDni6ZIAAEAp4dEjMlfSpUsX3XfffYqJidGePXv0xz/+UV27dtXGjRtVrly5Ap+Tk5OjnJwc13R2dnZJlQsAAEpYqQ4yDz30kOvnxo0bq0mTJqpdu7bWrl2rO++8s8DnJCcnKykpqaRKBAAAHlTqTy39Wq1atVSlShXt3r270D7jx4/X8ePHXY99+/aVYIUAAKAkleojMpf68ccfdeTIEVWtWrXQPk6nU06nswSrAgAAnuLRIHPy5Em3oysZGRnavn27QkJCFBISoqSkJPXu3VsRERHas2ePxo0bpzp16ighIcGDVQMAgNLCo0Fmy5Yt6tixo2t69OjRkqT+/ftr9uzZ2rlzpxYsWKCsrCxFRkaqc+fOevbZZzniAgAAJHk4yHTo0EHGmELnf/zxxyVYDQAAsI1VF/sCAAD8GkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALAWQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGCtIgWZTp06KSsrK197dna2OnXqdL01AQAAXJUiBZm1a9cqNzc3X/vZs2e1fv366y4KAADgapS/ls47d+50/fzVV18pMzPTNX3hwgUtX75c1apVK77qAAAALuOagkyzZs3kcDjkcDgKPIXk5+enV155pdiKAwAAuJxrCjIZGRkyxqhWrVr6/PPPFRoa6prn4+OjsLAwlStXrtiLBAAAKMg1BZmoqChJUl5e3g0pBgAA4FpcU5D5tfT0dK1Zs0aHDh3KF2wmTpx43YUBAABcSZGCzN/+9jcNHTpUVapUUUREhBwOh2uew+EgyAAAgBJRpCAzZcoUPffcc3rqqaeKux4AAICrVqTPkTl27JgeeOCB4q4FAADgmhQpyDzwwANasWJFcdcCAABwTYp0aqlOnTp65plntGnTJjVu3Fje3t5u80eOHFksxQEAAFxOkYLMvHnz5O/vr9TUVKWmprrNczgcBBkAAFAiihRkMjIyirsOAACAa1aka2QAAABKgyIdkXnkkUcuO//1118vUjEAAADXokhB5tixY27T586d065du5SVlVXgl0kCAADcCEUKMu+9916+try8PA0dOlS1a9e+7qIAAACuRrFdI+Pl5aXRo0dr+vTpxbVIAACAyyrWi3337Nmj8+fPF+ciAQAAClWkU0ujR492mzbG6MCBA/rwww/Vv3//YikMAADgSooUZLZt2+Y27eXlpdDQUL344otXvKMJAACguBQpyKxZs6a46wAAALhmRQoyFx0+fFhpaWmSpHr16ik0NLRYigIAALgaRbrY99SpU3rkkUdUtWpVtWvXTu3atVNkZKQGDhyo06dPF3eNAAAABSpSkBk9erRSU1P1/vvvKysrS1lZWVq6dKlSU1M1ZsyY4q4RAACgQEU6tfTPf/5TixcvVocOHVxtd999t/z8/PTggw9q9uzZxVUfAABAoYp0ROb06dMKDw/P1x4WFsapJQAAUGKKFGTi4uI0adIknT171tV25swZJSUlKS4urtiKAwAAuJwinVqaMWOGunTpourVq6tp06aSpB07dsjpdGrFihXFWiAAAEBhihRkGjdurPT0dL355pv65ptvJEl9+/ZVv3795OfnV6wFAgAAFKZIQSY5OVnh4eF69NFH3dpff/11HT58WE899VSxFAcAAHA5RbpGZu7cuapfv36+9oYNG2rOnDnXXRQAAMDVKFKQyczMVNWqVfO1h4aG6sCBA9ddFAAAwNUoUpCpUaOGNmzYkK99w4YNioyMvO6iAAAArkaRrpF59NFHNWrUKJ07d06dOnWSJK1atUrjxo3jk30BAECJKVKQGTt2rI4cOaLHH39cubm5kiRfX1899dRTGj9+fLEWCAAAUJginVpyOBx64YUXdPjwYW3atEk7duzQ0aNHNXHixGtazrp169SjRw9FRkbK4XBoyZIlbvONMZo4caKqVq0qPz8/xcfHKz09vSglAwCAMqhIQeYif39/tWjRQo0aNZLT6bzm5586dUpNmzbVzJkzC5w/depUvfzyy5ozZ442b96sihUrKiEhwe0ThQEAwM2rSKeWikvXrl3VtWvXAucZYzRjxgz96U9/Us+ePSVJf//73xUeHq4lS5booYceKslSAQBAKXRdR2RupIyMDGVmZio+Pt7VFhQUpFatWmnjxo0erAwAAJQWHj0iczmZmZmSlO9btsPDw13zCpKTk6OcnBzXdHZ29o0pEAAAeFypDTJFlZycrKSkJE+XASj66Q89XQIAlHml9tRSRESEJOngwYNu7QcPHnTNK8j48eN1/Phx12Pfvn03tE4AAOA5pTbIxMTEKCIiQqtWrXK1ZWdna/PmzYqLiyv0eU6nU4GBgW4PAABQNnn01NLJkye1e/du13RGRoa2b9+ukJAQ1axZU6NGjdKUKVNUt25dxcTE6JlnnlFkZKR69erluaIBAECp4dEgs2XLFnXs2NE1PXr0aElS//79lZKSonHjxunUqVMaPHiwsrKy1KZNGy1fvly+vr6eKhkAAJQiDmOM8XQRN1J2draCgoJ0/PjxYj/NxMWcgOftfb6bp0sAcANc7f/fpfYaGQAAgCshyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALAWQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALAWQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGCtUh1kJk+eLIfD4faoX7++p8sCAAClRHlPF3AlDRs21CeffOKaLl++1JcMAABKSKlPBeXLl1dERISnywAAAKVQqT61JEnp6emKjIxUrVq11K9fP/3www+eLgkAAJQSpfqITKtWrZSSkqJ69erpwIEDSkpKUtu2bbVr1y4FBAQU+JycnBzl5OS4prOzs0uqXAAAUMJKdZDp2rWr6+cmTZqoVatWioqK0rvvvquBAwcW+Jzk5GQlJSWVVIkAcM2in/7Q0yVcs73Pd/N0CUCBSv2ppV8LDg7WLbfcot27dxfaZ/z48Tp+/LjrsW/fvhKsEAAAlCSrgszJkye1Z88eVa1atdA+TqdTgYGBbg8AAFA2leog8+STTyo1NVV79+7VZ599pnvvvVflypVT3759PV0aAAAoBUr1NTI//vij+vbtqyNHjig0NFRt2rTRpk2bFBoa6unSAABAKVCqg8zbb7/t6RIAAEApVqpPLQEAAFwOQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWKtUf0UBAFxJ9NMferoEAB7EERkAAGAtggwAALAWQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALAWQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsBZBBgAAWIsgAwAArEWQAQAA1iLIAAAAaxFkAACAtQgyAADAWgQZAABgLYIMAACwFkEGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1ynu6AABA6Rf99IeeLuGa7X2+m6dLuGbs52vHERkAAGAtggwAALAWQQYAAFiLIAMAAKxFkAEAANYiyAAAAGsRZAAAgLUIMgAAwFoEGQAAYC2CDAAAsJYVQWbmzJmKjo6Wr6+vWrVqpc8//9zTJQEAgFKg1AeZd955R6NHj9akSZP0xRdfqGnTpkpISNChQ4c8XRoAAPCwUh9kXnrpJT366KMaMGCAbr31Vs2ZM0cVKlTQ66+/7unSAACAh5XqIJObm6utW7cqPj7e1ebl5aX4+Hht3LjRg5UBAIDSoLynC7icn3/+WRcuXFB4eLhbe3h4uL755psCn5OTk6OcnBzX9PHjxyVJ2dnZxV5fXs7pYl8mAKB43Ij3/RvNxv9XbtR+vrhcY8xl+5XqIFMUycnJSkpKytdeo0YND1QDAPCUoBmeruDmcKP384kTJxQUFFTo/FIdZKpUqaJy5crp4MGDbu0HDx5UREREgc8ZP368Ro8e7ZrOy8vT0aNHVblyZTkcjiLXkp2drRo1amjfvn0KDAws8nJw4zFW9mCs7MA42aMsjZUxRidOnFBkZORl+5XqIOPj46PY2FitWrVKvXr1kvRLMFm1apWGDx9e4HOcTqecTqdbW3BwcLHVFBgYaP2L42bBWNmDsbID42SPsjJWlzsSc1GpDjKSNHr0aPXv31/NmzdXy5YtNWPGDJ06dUoDBgzwdGkAAMDDSn2Q6dOnjw4fPqyJEycqMzNTzZo10/Lly/NdAAwAAG4+pT7ISNLw4cMLPZVUUpxOpyZNmpTvtBVKH8bKHoyVHRgne9yMY+UwV7qvCQAAoJQq1R+IBwAAcDkEGQAAYC2CDAAAsBZBBgAAWIsgc5Vmzpyp6Oho+fr6qlWrVvr88889XdJNZfLkyXI4HG6P+vXru+afPXtWw4YNU+XKleXv76/evXvn+0ToH374Qd26dVOFChUUFhamsWPH6vz58yW9KWXOunXr1KNHD0VGRsrhcGjJkiVu840xmjhxoqpWrSo/Pz/Fx8crPT3drc/Ro0fVr18/BQYGKjg4WAMHDtTJkyfd+uzcuVNt27aVr6+vatSooalTp97oTStTrjROiYmJ+X7HunTp4taHcSoZycnJatGihQICAhQWFqZevXopLS3NrU9xveetXbtWt912m5xOp+rUqaOUlJQbvXnFjiBzFd555x2NHj1akyZN0hdffKGmTZsqISFBhw4d8nRpN5WGDRvqwIEDrsenn37qmvfEE0/o/fff16JFi5Samqr9+/frvvvuc82/cOGCunXrptzcXH322WdasGCBUlJSNHHiRE9sSply6tQpNW3aVDNnzixw/tSpU/Xyyy9rzpw52rx5sypWrKiEhASdPXvW1adfv3768ssvtXLlSn3wwQdat26dBg8e7JqfnZ2tzp07KyoqSlu3btW0adM0efJkzZs374ZvX1lxpXGSpC5durj9ji1cuNBtPuNUMlJTUzVs2DBt2rRJK1eu1Llz59S5c2edOnXK1ac43vMyMjLUrVs3dezYUdu3b9eoUaM0aNAgffzxxyW6vdfN4Ipatmxphg0b5pq+cOGCiYyMNMnJyR6s6uYyadIk07Rp0wLnZWVlGW9vb7No0SJX29dff20kmY0bNxpjjFm2bJnx8vIymZmZrj6zZ882gYGBJicn54bWfjORZN577z3XdF5enomIiDDTpk1ztWVlZRmn02kWLlxojDHmq6++MpLMf/7zH1efjz76yDgcDvPTTz8ZY4yZNWuWqVSpkttYPfXUU6ZevXo3eIvKpkvHyRhj+vfvb3r27Fnocxgnzzl06JCRZFJTU40xxfeeN27cONOwYUO3dfXp08ckJCTc6E0qVhyRuYLc3Fxt3bpV8fHxrjYvLy/Fx8dr48aNHqzs5pOenq7IyEjVqlVL/fr10w8//CBJ2rp1q86dO+c2RvXr11fNmjVdY7Rx40Y1btzY7ROhExISlJ2drS+//LJkN+QmkpGRoczMTLexCQoKUqtWrdzGJjg4WM2bN3f1iY+Pl5eXlzZv3uzq065dO/n4+Lj6JCQkKC0tTceOHSuhrSn71q5dq7CwMNWrV09Dhw7VkSNHXPMYJ885fvy4JCkkJERS8b3nbdy40W0ZF/vY9n8bQeYKfv75Z124cCHfVyKEh4crMzPTQ1XdfFq1aqWUlBQtX75cs2fPVkZGhtq2basTJ04oMzNTPj4++b4c9NdjlJmZWeAYXpyHG+Pivr3c709mZqbCwsLc5pcvX14hISGMXwnq0qWL/v73v2vVqlV64YUXlJqaqq5du+rChQuSGCdPycvL06hRo3THHXeoUaNGklRs73mF9cnOztaZM2duxObcEFZ8RQHQtWtX189NmjRRq1atFBUVpXfffVd+fn4erAwoGx566CHXz40bN1aTJk1Uu3ZtrV27VnfeeacHK7u5DRs2TLt27XK7JhDuOCJzBVWqVFG5cuXyXQ1+8OBBRUREeKgqBAcH65ZbbtHu3bsVERGh3NxcZWVlufX59RhFREQUOIYX5+HGuLhvL/f7ExERke/C+fPnz+vo0aOMnwfVqlVLVapU0e7duyUxTp4wfPhwffDBB1qzZo2qV6/uai+u97zC+gQGBlr1ByJB5gp8fHwUGxurVatWudry8vK0atUqxcXFebCym9vJkye1Z88eVa1aVbGxsfL29nYbo7S0NP3www+uMYqLi9N///tftzfilStXKjAwULfeemuJ13+ziImJUUREhNvYZGdna/PmzW5jk5WVpa1bt7r6rF69Wnl5eWrVqpWrz7p163Tu3DlXn5UrV6pevXqqVKlSCW3NzeXHH3/UkSNHVLVqVUmMU0kyxmj48OF67733tHr1asXExLjNL673vLi4OLdlXOxj3f9tnr7a2AZvv/22cTqdJiUlxXz11Vdm8ODBJjg42O1qcNxYY8aMMWvXrjUZGRlmw4YNJj4+3lSpUsUcOnTIGGPMY489ZmrWrGlWr15ttmzZYuLi4kxcXJzr+efPnzeNGjUynTt3Ntu3bzfLly83oaGhZvz48Z7apDLjxIkTZtu2bWbbtm1GknnppZfMtm3bzPfff2+MMeb55583wcHBZunSpWbnzp2mZ8+eJiYmxpw5c8a1jC5dupjf/OY3ZvPmzebTTz81devWNX379nXNz8rKMuHh4ebhhx82u3btMm+//bapUKGCmTt3bolvr60uN04nTpwwTz75pNm4caPJyMgwn3zyibnttttM3bp1zdmzZ13LYJxKxtChQ01QUJBZu3atOXDggOtx+vRpV5/ieM/77rvvTIUKFczYsWPN119/bWbOnGnKlStnli9fXqLbe70IMlfplVdeMTVr1jQ+Pj6mZcuWZtOmTZ4u6abSp08fU7VqVePj42OqVatm+vTpY3bv3u2af+bMGfP444+bSpUqmQoVKph7773XHDhwwG0Ze/fuNV27djV+fn6mSpUqZsyYMebcuXMlvSllzpo1a4ykfI/+/fsbY365BfuZZ54x4eHhxul0mjvvvNOkpaW5LePIkSOmb9++xt/f3wQGBpoBAwaYEydOuPXZsWOHadOmjXE6naZatWrm+eefL6lNLBMuN06nT582nTt3NqGhocbb29tERUWZRx99NN8fa4xTyShonCSZ+fPnu/oU13vemjVrTLNmzYyPj4+pVauW2zps4TDGmJI+CgQAAFAcuEYGAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggxQhnXo0EGjRo3ydBmlSkpKSr5vDfak6OhozZgxw9NlANYiyAAoE9auXSuHw5Hvi/RKi9IWoICygiADAACsRZAByohTp07p97//vfz9/VW1alW9+OKL+focO3ZMv//971WpUiVVqFBBXbt2VXp6ulufDRs2qEOHDqpQoYIqVaqkhIQEHTt2TFLBp0GaNWumyZMnu6YdDofmzp2r7t27q0KFCmrQoIE2btyo3bt3q0OHDqpYsaJuv/127dmzx205S5cu1W233SZfX1/VqlVLSUlJOn/+vNtyX331Vd17772qUKGC6tatq3//+9+SpL1796pjx46SpEqVKsnhcCgxMfGq9931rPuif//736pbt658fX3VsWNHLViwwHWEaO3atRowYICOHz8uh8Mhh8Phts9Onz6tRx55RAEBAapZs6bmzZvnmpebm6vhw4eratWq8vX1VVRUlJKTk69624Ayz9Nf9gSgeAwdOtTUrFnTfPLJJ2bnzp2me/fuJiAgwPzhD39w9bnnnntMgwYNzLp168z27dtNQkKCqVOnjsnNzTXGGLNt2zbjdDrN0KFDzfbt282uXbvMK6+8Yg4fPmyMMSYqKspMnz7dbb1NmzY1kyZNck1LMtWqVTPvvPOOSUtLM7169TLR0dGmU6dOZvny5earr74yrVu3Nl26dHE9Z926dSYwMNCkpKSYPXv2mBUrVpjo6GgzefJkt+VWr17dvPXWWyY9Pd2MHDnS+Pv7myNHjpjz58+bf/7zn0aSSUtLMwcOHDBZWVkF7qf58+eboKCgYlu3Mb98i7C3t7d58sknzTfffGMWLlxoqlWrZiSZY8eOmZycHDNjxgwTGBjo+ibji1+2GBUVZUJCQszMmTNNenq6SU5ONl5eXuabb74xxhgzbdo0U6NGDbNu3Tqzd+9es379evPWW29d5asCKPsIMkAZcOLECePj42PeffddV9uRI0eMn5+fK8h8++23RpLZsGGDq8/PP/9s/Pz8XM/r27evueOOOwpdz9UGmT/96U+u6Y0bNxpJ5rXXXnO1LVy40Pj6+rqm77zzTvPnP//ZbblvvPGGqVq1aqHLPXnypJFkPvroI2PM/32787Fjxwqt35j8QaY41v3UU0+ZRo0auS1jwoQJbvVcut6LoqKizO9+9zvXdF5engkLCzOzZ882xhgzYsQI06lTJ5OXl3fZ7QJuVuU9cRQIQPHas2ePcnNz1apVK1dbSEiI6tWr55r++uuvVb58ebc+lStXVr169fT1119LkrZv364HHnjguutp0qSJ6+fw8HBJUuPGjd3azp49q+zsbAUGBmrHjh3asGGDnnvuOVefCxcu6OzZszp9+rQqVKiQb7kVK1ZUYGCgDh06dF21Fse609LS1KJFC7fltmzZ8qpr+PWyHQ6HIiIiXMtOTEzUXXfdpXr16qlLly7q3r27OnfufO0bCpRRBBkALn5+fped7+XlJWOMW9u5c+fy9fP29nb97HA4Cm3Ly8uTJJ08eVJJSUm677778i3L19e3wOVeXM7FZRSVJ9d9Ncu+7bbblJGRoY8++kiffPKJHnzwQcXHx2vx4sXFsm7AdgQZoAyoXbu2vL29tXnzZtWsWVPSLxf2fvvtt2rfvr0kqUGDBjp//rw2b96s22+/XZJ05MgRpaWl6dZbb5X0y5GBVatWKSkpqcD1hIaG6sCBA67p7OxsZWRkXHf9t912m9LS0lSnTp0iL8PHx0fSL0dTSnrd9erV07Jly9za/vOf/+Sr71pruygwMFB9+vRRnz59dP/996tLly46evSoQkJCilwzUFYQZIAywN/fXwMHDtTYsWNVuXJlhYWFacKECfLy+r8bE+vWrauePXvq0Ucf1dy5cxUQEKCnn35a1apVU8+ePSVJ48ePV+PGjfX444/rsccek4+Pj9asWaMHHnhAVapUUadOnZSSkqIePXooODhYEydOVLly5a67/okTJ6p79+6qWbOm7r//fnl5eWnHjh3atWuXpkyZclXLiIqKksPh0AcffKC7775bfn5+8vf3L5F1DxkyRC+99JKeeuopDRw4UNu3b1dKSoqk/zv6FB0drZMnT2rVqlVq2rSpKlSo4DptdTkvvfSSqlatqt/85jfy8vLSokWLFBERwWfSAP8ft18DZcS0adPUtm1b9ejRQ/Hx8WrTpo1iY2Pd+syfP1+xsbHq3r274uLiZIzRsmXLXKc2brnlFq1YsUI7duxQy5YtFRcXp6VLl6p8+V/+5hk/frzat2+v7t27q1u3burVq5dq16593bUnJCTogw8+0IoVK9SiRQu1bt1a06dPV1RU1FUvo1q1akpKStLTTz+t8PBwDR8+vMTWHRMTo8WLF+tf//qXmjRpotmzZ2vChAmSJKfTKUm6/fbb9dhjj6lPnz4KDQ3V1KlTr2rZAQEBmjp1qpo3b64WLVpo7969WrZsmVtIBW5mDnPpCW8AwHV77rnnNGfOHO3bt8/TpQBlGqeWAKAYzJo1Sy1atFDlypW1YcMGTZs27aqPCgEoOoIMABSD9PR0TZkyRUePHlXNmjU1ZswYjR8/3tNlAWUep5YAAIC1uFoMAABYiyADAACsRZABAADWIsgAAABrEWQAAIC1CDIAAMBaBBkAAGAtggwAALAWQQYAAFjr/wGE0iEmhGAvBwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "plot_doc_lengths_hist(fig, ax, corpus_small, y_log=False)  # use linear scale\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can improve this plot, e.g. to better see the distribution of small documents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.301890Z",
     "iopub.status.busy": "2023-05-03T13:22:48.301732Z",
     "iopub.status.idle": "2023-05-03T13:22:48.392363Z",
     "shell.execute_reply": "2023-05-03T13:22:48.391763Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAIjCAYAAAA0vUuxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABFcElEQVR4nO3deVxUZf//8fcgsriA4gKSAm6ZW5pbqd1uUWpuLZZ1m6lZarlkepuSXxfKwuouKTOXFrWyzLxzuXPPcM0lNS1bFAvN21wqBdxChev3hz/GRsALEZmJeT0fj3nUXOc653zOhxHmzTlzcBhjjAAAAAAAOfJxdwEAAAAA4OkITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAJCNqKgo9erVy91lFHovv/yyqlSpoiJFiqh+/fpXvH6rVq3UqlWrfK/LG/Xq1UtRUVHuLsNq5syZcjgc2rp1q7tLAeBlCE4ACj3bG61WrVqpTp06V72fJUuWaNy4cVe9HW+xYsUKPf3002revLlmzJihF154wd0lFSpffvmlxo0bp+TkZHeXkidvvvmmZs6c6e4yAMDJ190FAIAn2r17t3x8rux3S0uWLNHkyZMJT7n0xRdfyMfHR++88478/PzcXU6h8+WXXyo2Nla9evVSqVKl3F3OFXvzzTdVtmxZzvwC8BiccQKAbPj7+6to0aLuLuOKnDp1yt0lXJGjR48qMDCQ0AQA+FsgOAFANi79jNO5c+cUGxur6tWrKyAgQGXKlNGtt96qlStXSrrw+ZDJkydLkhwOh/OR6dSpUxo2bJgqVaokf39/1ahRQ//+979ljHHZ75kzZzR48GCVLVtWJUuWVOfOnXXw4EE5HA6XM1njxo2Tw+HQ999/r3/+858qXbq0br31VknSN998o169eqlKlSoKCAhQWFiYHnnkEf3xxx8u+8rcxp49e/TQQw8pODhY5cqV0+jRo2WM0YEDB9SlSxcFBQUpLCxMr7zySq56d/78eT333HOqWrWq/P39FRUVpWeeeUZpaWnOOQ6HQzNmzNCpU6ecvbJdljV9+nRVrVpVgYGBatKkidatW5ftvKNHj6pPnz4KDQ1VQECA6tWrp1mzZmWZl5GRoddee01169ZVQECAypUrp3bt2jkv6dy3b1+OdeX09biaXqalpWns2LGqVq2a/P39ValSJT399NMufcvc98CBA7VgwQLVqVNH/v7+ql27tpYtW+ZSz/DhwyVJlStXdvZ43759l+1xdj2Kj49X7dq1FRAQoNDQUPXr10/Hjx93mRcVFaWOHTtq/fr1atKkiQICAlSlShW99957Wbb5zTffqGXLlgoMDFTFihU1fvx4zZgxw6W+qKgofffdd1qzZo2z9ks/y5aWlqahQ4eqXLlyKl68uO6++2799ttvLnO2bt2qtm3bqmzZsgoMDFTlypX1yCOPXFEPACATl+oB8BopKSn6/fffs4yfO3fOuu64ceMUFxenRx99VE2aNFFqaqq2bt2q7du36/bbb1e/fv3066+/auXKlXr//fdd1jXGqHPnzkpISFCfPn1Uv359LV++XMOHD9fBgwc1ceJE59xevXpp7ty56tGjh2655RatWbNGHTp0yLGu++67T9WrV9cLL7zgDGErV67Uzz//rN69eyssLEzfffedpk+fru+++06bNm1yCXSS1K1bN9WsWVMTJkzQ4sWLNX78eIWEhGjatGlq06aNXnzxRc2ePVv/+te/1LhxY7Vo0eKyvXr00Uc1a9Ysde3aVcOGDdPmzZsVFxenH374QfPnz5ckvf/++5o+fbq2bNmit99+W5LUrFmzHLf5zjvvqF+/fmrWrJmGDBmin3/+WZ07d1ZISIgqVarknHfmzBm1atVKe/fu1cCBA1W5cmV98skn6tWrl5KTk/Xkk0865/bp00czZ85U+/bt9eijj+r8+fNat26dNm3apEaNGl32GHOS115mZGSoc+fOWr9+vfr27auaNWvq22+/1cSJE7Vnzx4tWLDAZT/r16/Xp59+qieeeEIlS5bU66+/rnvvvVe//PKLypQpo3vuuUd79uzRRx99pIkTJ6ps2bKSpHLlyl3R8fTr108zZ85U7969NXjwYCUlJemNN97Q119/rQ0bNricld27d6+6du2qPn36qGfPnnr33XfVq1cvNWzYULVr15YkHTx4UK1bt5bD4VBMTIyKFy+ut99+W/7+/i77jY+P16BBg1SiRAmNGjVKkhQaGuoyZ9CgQSpdurTGjh2rffv2KT4+XgMHDtTHH38s6UKAvuOOO1SuXDmNHDlSpUqV0r59+/Tpp59eUQ8AwMkAQCE3Y8YMI+myj9q1a7usExkZaXr27Ol8Xq9ePdOhQ4fL7mfAgAEmu2+rCxYsMJLM+PHjXca7du1qHA6H2bt3rzHGmG3bthlJZsiQIS7zevXqZSSZsWPHOsfGjh1rJJkHH3wwy/5Onz6dZeyjjz4ykszatWuzbKNv377OsfPnz5uKFSsah8NhJkyY4Bw/fvy4CQwMdOlJdnbs2GEkmUcffdRl/F//+peRZL744gvnWM+ePU3x4sUvuz1jjDl79qwpX768qV+/vklLS3OOT58+3UgyLVu2dI7Fx8cbSeaDDz5wWb9p06amRIkSJjU11RhjzBdffGEkmcGDB2fZX0ZGhjHGmKSkJCPJzJgxI8ucnL4eee3l+++/b3x8fMy6detc9jN16lQjyWzYsMFl335+fs7XjTHG7Ny500gykyZNco69/PLLRpJJSkrKUn92evbsaSIjI53P161bZySZ2bNnu8xbtmxZlvHIyMgsr6+jR48af39/M2zYMOfYoEGDjMPhMF9//bVz7I8//jAhISFZaq1du7bL1zZT5r/n6Oho59fKGGOeeuopU6RIEZOcnGyMMWb+/PlGkvnqq69ydfwAYMOlegC8xuTJk7Vy5cosjxtvvNG6bqlSpfTdd98pMTHxive7ZMkSFSlSRIMHD3YZHzZsmIwxWrp0qSQ5L7V64oknXOYNGjQox233798/y1hgYKDz///880/9/vvvuuWWWyRJ27dvzzL/0Ucfdf5/kSJF1KhRIxlj1KdPH+d4qVKlVKNGDf3888851iJdOFZJGjp0qMv4sGHDJEmLFy++7PrZ2bp1q44ePar+/fu7fB6qV69eCg4OzrL/sLAwPfjgg86xokWLavDgwTp58qTWrFkjSfrPf/4jh8OhsWPHZtnfpWfkrkRee/nJJ5+oZs2auuGGG/T77787H23atJEkJSQkuOwnOjpaVatWdT6/8cYbFRQUZP36XIlPPvlEwcHBuv32211qatiwoUqUKJGlplq1aukf//iH83m5cuWyHOeyZcvUtGlTl1vPh4SEqHv37ldcX9++fV2+Vv/4xz+Unp6u/fv3S5LzhhifffZZrs4qA4ANl+oB8BpNmjTJ9hKs0qVLZ3sJ3189++yz6tKli66//nrVqVNH7dq1U48ePXIVuvbv36/w8HCVLFnSZbxmzZrO5Zn/9fHxUeXKlV3mVatWLcdtXzpXko4dO6bY2FjNmTNHR48edVmWkpKSZX5ERITL8+DgYAUEBDgv7/rr+KWfk7pU5jFcWnNYWJhKlSrlPNYrkblO9erVXcaLFi2qKlWqZJlbvXr1LHdEvLTXP/30k8LDwxUSEnLF9VxOXnuZmJioH374IcdL6S79Ol66H+nC6/jSzx5djcTERKWkpKh8+fL5VtP+/fvVtGnTLPMu9xrPyaX7K126tCQ599eyZUvde++9io2N1cSJE9WqVSvddddd+uc//5nl0kAAyA2CEwDkQosWLfTTTz9p4cKFWrFihd5++21NnDhRU6dOdTnLUND+enYp0/33368vv/xSw4cPV/369VWiRAllZGSoXbt2ysjIyDK/SJEiuRqTlOVmFjm5mrM2niKnY0hPT89xnbz2MiMjQ3Xr1tWrr76a7dy/fo4rt9u8WhkZGSpfvrxmz56d7fJLQ15B1HQl+3M4HJo3b542bdqk//73v1q+fLkeeeQRvfLKK9q0aZNKlChxTeoCUHgRnAAgl0JCQtS7d2/17t1bJ0+eVIsWLTRu3DhncMrpjXZkZKQ+//xznThxwuWs048//uhcnvnfjIwMJSUluZxd2bt3b65rPH78uFatWqXY2FiNGTPGOZ6XSwzzIvMYEhMTnWd5JOnIkSNKTk52HuuVblO6cAyZl65JF27qkZSUpHr16rnM/eabb5SRkeFy1unSXletWlXLly/XsWPHcjzrlHkG49I/IJuXs2Y2VatW1c6dO3XbbbflW+i82u1UrVpVn3/+uZo3b55tQM+LyMjIbF/P2Y3lVx9uueUW3XLLLXr++ef14Ycfqnv37pozZ45bf+EB4O+JzzgBQC5ceolaiRIlVK1aNZdbRRcvXlxS1jfad955p9LT0/XGG2+4jE+cOFEOh0Pt27eXJLVt21bShT/8+VeTJk3KdZ2Zv4W/9Lf88fHxud7G1bjzzjuz3V/mmZTL3SEwJ40aNVK5cuU0depUnT171jk+c+bMbHt9+PBh553VpAu3R580aZJKlCihli1bSpLuvfdeGWMUGxubZX+ZvQsKClLZsmW1du1al+WXfn3yw/3336+DBw/qrbfeyrLszJkzefobXTm9Hq+kpvT0dD333HNZlp0/fz5P223btq02btyoHTt2OMeOHTuW7Vmt4sWL57l26cIvES79d5D52apLb/EOALnBGScAyIVatWqpVatWatiwoUJCQrR161bNmzdPAwcOdM5p2LChJGnw4MFq27atihQpogceeECdOnVS69atNWrUKO3bt0/16tXTihUrtHDhQg0ZMsT5If+GDRvq3nvvVXx8vP744w/n7cj37NkjKXe/gQ8KClKLFi300ksv6dy5c7ruuuu0YsUKJSUlXYOuZFWvXj317NlT06dPV3Jyslq2bKktW7Zo1qxZuuuuu9S6desr3mbRokU1fvx49evXT23atFG3bt2UlJSkGTNmZPmMU9++fTVt2jT16tVL27ZtU1RUlObNm6cNGzYoPj7eecavdevW6tGjh15//XUlJiY6L2Nct26dWrdu7fy6Pvroo5owYYIeffRRNWrUSGvXrnV+PfJTjx49NHfuXPXv318JCQlq3ry50tPT9eOPP2ru3Llavnz5Fd8iPfP1OGrUKD3wwAMqWrSoOnXq5AxUNi1btlS/fv0UFxenHTt26I477lDRokWVmJioTz75RK+99pq6du16RTU9/fTT+uCDD3T77bdr0KBBztuRR0RE6NixYy6v8YYNG2rKlCkaP368qlWrpvLly7uccbSZNWuW3nzzTd19992qWrWqTpw4obfeektBQUHOgA8AV4LgBAC5MHjwYC1atEgrVqxQWlqaIiMjNX78eOcfGZWke+65R4MGDdKcOXP0wQcfyBijBx54QD4+Plq0aJHGjBmjjz/+WDNmzFBUVJRefvll593mMr333nsKCwvTRx99pPnz5ys6Oloff/yxatSooYCAgFzV+uGHH2rQoEGaPHmyjDG64447tHTpUoWHh+drT3Ly9ttvq0qVKpo5c6bmz5+vsLAwxcTEZHsHu9zq27ev0tPT9fLLL2v48OGqW7euFi1apNGjR7vMCwwM1OrVqzVy5EjNmjVLqampqlGjhmbMmOHyB40lacaMGbrxxhv1zjvvaPjw4QoODlajRo1c/p7UmDFj9Ntvv2nevHmaO3eu2rdvr6VLl+Z4w4S88vHx0YIFCzRx4kS99957mj9/vooVK6YqVaroySef1PXXX3/F22zcuLGee+45TZ06VcuWLXNeBprb4CRJU6dOVcOGDTVt2jQ988wz8vX1VVRUlB566CE1b978imuqVKmSEhISNHjwYL3wwgsqV66cBgwYoOLFi2vw4MEur/ExY8Zo//79eumll3TixAm1bNnyioJTZmifM2eOjhw5ouDgYDVp0kSzZ8/O9qYqAGDjMNfqU5sAgHyxY8cO3XTTTfrggw/ydNtmwNMNGTJE06ZN08mTJ3O86QMAuBufcQIAD3LmzJksY/Hx8fLx8VGLFi3cUBGQvy59jf/xxx96//33deuttxKaAHg0LtUDAA/y0ksvadu2bWrdurV8fX21dOlSLV26VH379s1yS2rg76hp06Zq1aqVatasqSNHjuidd95RampqlssuAcDTcKkeAHiQlStXKjY2Vt9//71OnjypiIgI9ejRQ6NGjZKvL7/rwt/fM888o3nz5ul///ufHA6HGjRooLFjxyo6OtrdpQHAZRGcAAAAAMCCzzgBAAAAgAXBCQAAAAAsCv0F8xkZGfr1119VsmTJXP3xSAAAAACFkzFGJ06cUHh4uHx8ruwcUqEPTr/++it3ogIAAADgdODAAVWsWPGK1in0walkyZKSLjQnKCjIzdUAAAAAcJfU1FRVqlTJmRGuRKEPTpmX5wUFBRGcAAAAAOTpIzzcHAIAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALBwa3Bau3atOnXqpPDwcDkcDi1YsCDHuf3795fD4VB8fHyB1QcAAAAAkpuD06lTp1SvXj1Nnjz5svPmz5+vTZs2KTw8vIAqAwAAAICLfN258/bt26t9+/aXnXPw4EENGjRIy5cvV4cOHQqoMgAAAAC4yK3BySYjI0M9evTQ8OHDVbt27Vytk5aWprS0NOfz1NTUa1UeAAAAAC/h0cHpxRdflK+vrwYPHpzrdeLi4hQbG3sNqwIKVtTIxe4uwWnfBM76AgAA7+Sxd9Xbtm2bXnvtNc2cOVMOhyPX68XExCglJcX5OHDgwDWsEgAAAIA38NjgtG7dOh09elQRERHy9fWVr6+v9u/fr2HDhikqKirH9fz9/RUUFOTyAAAAAICr4bGX6vXo0UPR0dEuY23btlWPHj3Uu3dvN1UFAAAAwBu5NTidPHlSe/fudT5PSkrSjh07FBISooiICJUpU8ZlftGiRRUWFqYaNWoUdKkAAAAAvJhbg9PWrVvVunVr5/OhQ4dKknr27KmZM2e6qSoAAAAAcOXW4NSqVSsZY3I9f9++fdeuGAAAAADIgcfeHAIAAAAAPAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACw8HV3AUCmqJGL3V2C074JHdxdAgAAADwIZ5wAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsHBrcFq7dq06deqk8PBwORwOLViwwLns3LlzGjFihOrWravixYsrPDxcDz/8sH799Vf3FQwAAADAK7k1OJ06dUr16tXT5MmTsyw7ffq0tm/frtGjR2v79u369NNPtXv3bnXu3NkNlQIAAADwZr7u3Hn79u3Vvn37bJcFBwdr5cqVLmNvvPGGmjRpol9++UUREREFUSIAAAAAuDc4XamUlBQ5HA6VKlUqxzlpaWlKS0tzPk9NTS2AygAAAAAUZn+b4PTnn39qxIgRevDBBxUUFJTjvLi4OMXGxhZgZX9fUSMXu7sEAAAA4G/hb3FXvXPnzun++++XMUZTpky57NyYmBilpKQ4HwcOHCigKgEAAAAUVh5/xikzNO3fv19ffPHFZc82SZK/v7/8/f0LqDoAAAAA3sCjg1NmaEpMTFRCQoLKlCnj7pIAAAAAeCG3BqeTJ09q7969zudJSUnasWOHQkJCVKFCBXXt2lXbt2/XZ599pvT0dB0+fFiSFBISIj8/P3eVDQAAAMDLuDU4bd26Va1bt3Y+Hzp0qCSpZ8+eGjdunBYtWiRJql+/vst6CQkJatWqVUGVCQAAAMDLuTU4tWrVSsaYHJdfbhkAAAAAFJS/xV31AAAAAMCdCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWLg1OK1du1adOnVSeHi4HA6HFixY4LLcGKMxY8aoQoUKCgwMVHR0tBITE91TLAAAAACv5dbgdOrUKdWrV0+TJ0/OdvlLL72k119/XVOnTtXmzZtVvHhxtW3bVn/++WcBVwoAAADAm/m6c+ft27dX+/bts11mjFF8fLz+7//+T126dJEkvffeewoNDdWCBQv0wAMPZLteWlqa0tLSnM9TU1Pzv3AAAAAAXsVjP+OUlJSkw4cPKzo62jkWHBysm2++WRs3bsxxvbi4OAUHBzsflSpVKohyAQAAABRiHhucDh8+LEkKDQ11GQ8NDXUuy05MTIxSUlKcjwMHDlzTOgEAAAAUfm69VO9a8Pf3l7+/v7vLAAAAAFCIeOwZp7CwMEnSkSNHXMaPHDniXAYAAAAABcFjg1PlypUVFhamVatWOcdSU1O1efNmNW3a1I2VAQAAAPA2br1U7+TJk9q7d6/zeVJSknbs2KGQkBBFRERoyJAhGj9+vKpXr67KlStr9OjRCg8P11133eW+ogEAAAB4HbcGp61bt6p169bO50OHDpUk9ezZUzNnztTTTz+tU6dOqW/fvkpOTtatt96qZcuWKSAgwF0lAwAAAPBCbg1OrVq1kjEmx+UOh0PPPvusnn322QKsCgAAAABceexnnAAAAADAUxCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALX3cXAHiiqJGL3V0CkGee9PrdN6GDu0sAACBfcMYJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABg4dHBKT09XaNHj1blypUVGBioqlWr6rnnnpMxxt2lAQAAAPAivu4u4HJefPFFTZkyRbNmzVLt2rW1detW9e7dW8HBwRo8eLC7ywMAAADgJTw6OH355Zfq0qWLOnToIEmKiorSRx99pC1btri5MgAAAADexKMv1WvWrJlWrVqlPXv2SJJ27typ9evXq3379jmuk5aWptTUVJcHAAAAAFwNjz7jNHLkSKWmpuqGG25QkSJFlJ6erueff17du3fPcZ24uDjFxsYWYJWA94gaudjdJTjtm9DB3SUAAAAv4tFnnObOnavZs2frww8/1Pbt2zVr1iz9+9//1qxZs3JcJyYmRikpKc7HgQMHCrBiAAAAAIWRR59xGj58uEaOHKkHHnhAklS3bl3t379fcXFx6tmzZ7br+Pv7y9/fvyDLBAAAAFDIefQZp9OnT8vHx7XEIkWKKCMjw00VAQAAAPBGeQpObdq0UXJycpbx1NRUtWnT5mprcurUqZOef/55LV68WPv27dP8+fP16quv6u677863fQAAAACATZ4u1Vu9erXOnj2bZfzPP//UunXrrrqoTJMmTdLo0aP1xBNP6OjRowoPD1e/fv00ZsyYfNsHAAAAANhcUXD65ptvnP///fff6/Dhw87n6enpWrZsma677rp8K65kyZKKj49XfHx8vm0TAAAAAK7UFQWn+vXry+FwyOFwZHtJXmBgoCZNmpRvxQEAAACAJ7ii4JSUlCRjjKpUqaItW7aoXLlyzmV+fn4qX768ihQpku9FAgAAAIA7XVFwioyMlCTuagcAAADAq+T57zglJiYqISFBR48ezRKkuHkDAAAAgMIkT8Hprbfe0uOPP66yZcsqLCxMDofDuczhcBCcAAAAABQqeQpO48eP1/PPP68RI0bkdz0AAAAA4HHy9Adwjx8/rvvuuy+/awEAAAAAj5Sn4HTfffdpxYoV+V0LAAAAAHikPF2qV61aNY0ePVqbNm1S3bp1VbRoUZflgwcPzpfiAAAAAMAT5Ck4TZ8+XSVKlNCaNWu0Zs0al2UOh4PgBAAAAKBQyVNwSkpKyu86AAAAAMBj5ekzTgAAAADgTfJ0xumRRx657PJ33303T8UAAAAAgCfKU3A6fvy4y/Nz585p165dSk5OVps2bfKlMAAAAADwFHkKTvPnz88ylpGRoccff1xVq1a96qIAAAAAwJPk22ecfHx8NHToUE2cODG/NgkAAAAAHiFfbw7x008/6fz58/m5SQAAAABwuzxdqjd06FCX58YYHTp0SIsXL1bPnj3zpTAAAAAA8BR5Ck5ff/21y3MfHx+VK1dOr7zyivWOewAAAADwd5On4JSQkJDfdQAAAACAx8pTcMr022+/affu3ZKkGjVqqFy5cvlSFAAAAAB4kjzdHOLUqVN65JFHVKFCBbVo0UItWrRQeHi4+vTpo9OnT+d3jQAAAADgVnkKTkOHDtWaNWv03//+V8nJyUpOTtbChQu1Zs0aDRs2LL9rBAAAAAC3ytOlev/5z380b948tWrVyjl25513KjAwUPfff7+mTJmSX/UBAAAAgNvl6YzT6dOnFRoammW8fPnyXKoHAAAAoNDJ0xmnpk2bauzYsXrvvfcUEBAgSTpz5oxiY2PVtGnTfC2wsIkaudjdJQBAgfGk73n7JnRwdwkAgL+xPAWn+Ph4tWvXThUrVlS9evUkSTt37pS/v79WrFiRrwUCAAAAgLvlKTjVrVtXiYmJmj17tn788UdJ0oMPPqju3bsrMDAwXwsEAAAAAHfLU3CKi4tTaGioHnvsMZfxd999V7/99ptGjBiRL8UBAAAAgCfI080hpk2bphtuuCHLeO3atTV16tSrLgoAAAAAPEmegtPhw4dVoUKFLOPlypXToUOHrrooAAAAAPAkeQpOlSpV0oYNG7KMb9iwQeHh4VddFAAAAAB4kjx9xumxxx7TkCFDdO7cObVp00aStGrVKj399NMaNmxYvhYIAAAAAO6Wp+A0fPhw/fHHH3riiSd09uxZSVJAQIBGjBihmJiYfC0QAAAAANwtT8HJ4XDoxRdf1OjRo/XDDz8oMDBQ1atXl7+/f37XBwAAAABul6fglKlEiRJq3LhxftUCAAAAAB4pTzeHAAAAAABvQnACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsPD44HTx4UA899JDKlCmjwMBA1a1bV1u3bnV3WQAAAAC8iK+7C7ic48ePq3nz5mrdurWWLl2qcuXKKTExUaVLl3Z3aQAAAAC8iEcHpxdffFGVKlXSjBkznGOVK1d2Y0UAAAAAvJFHX6q3aNEiNWrUSPfdd5/Kly+vm266SW+99dZl10lLS1NqaqrLAwAAAACuhkefcfr55581ZcoUDR06VM8884y++uorDR48WH5+furZs2e268TFxSk2NraAKwVQ0KJGLnZ3CS72Tejg7hIAAMA15NFnnDIyMtSgQQO98MILuummm9S3b1899thjmjp1ao7rxMTEKCUlxfk4cOBAAVYMAAAAoDDy6OBUoUIF1apVy2WsZs2a+uWXX3Jcx9/fX0FBQS4PAAAAALgaHh2cmjdvrt27d7uM7dmzR5GRkW6qCAAAAIA38ujg9NRTT2nTpk164YUXtHfvXn344YeaPn26BgwY4O7SAAAAAHgRjw5OjRs31vz58/XRRx+pTp06eu655xQfH6/u3bu7uzQAAAAAXsSj76onSR07dlTHjh3dXQYAAAAAL+bRZ5wAAAAAwBMQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACw+FsFpwkTJsjhcGjIkCHuLgUAAACAF/nbBKevvvpK06ZN04033ujuUgAAAAB4mb9FcDp58qS6d++ut956S6VLl3Z3OQAAAAC8zN8iOA0YMEAdOnRQdHS0dW5aWppSU1NdHgAAAABwNXzdXYDNnDlztH37dn311Ve5mh8XF6fY2NhrXBUAuIoaudjdJQAAgGvIo884HThwQE8++aRmz56tgICAXK0TExOjlJQU5+PAgQPXuEoAAAAAhZ1Hn3Hatm2bjh49qgYNGjjH0tPTtXbtWr3xxhtKS0tTkSJFXNbx9/eXv79/QZcKAAAAoBDz6OB022236dtvv3UZ6927t2644QaNGDEiS2gCAAAAgGvBo4NTyZIlVadOHZex4sWLq0yZMlnGAQAAAOBa8ejPOAEAAACAJ/DoM07ZWb16tbtLAAAAAOBlOOMEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAwtfdBQAAAPeKGrnY3SU47ZvQwd0lAEC2OOMEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgIVHB6e4uDg1btxYJUuWVPny5XXXXXdp9+7d7i4LAAAAgJfx6OC0Zs0aDRgwQJs2bdLKlSt17tw53XHHHTp16pS7SwMAAADgRXzdXcDlLFu2zOX5zJkzVb58eW3btk0tWrRwU1UAAAAAvI1HB6dLpaSkSJJCQkJynJOWlqa0tDTn89TU1GteFwAAAIDCzWGMMe4uIjcyMjLUuXNnJScna/369TnOGzdunGJjY7OMp6SkKCgo6FqWmCtRIxe7uwQAAJAL+yZ0cHcJAPJZamqqgoOD85QNPPozTn81YMAA7dq1S3PmzLnsvJiYGKWkpDgfBw4cKKAKAQAAABRWf4tL9QYOHKjPPvtMa9euVcWKFS8719/fX/7+/gVUGQAAAABv4NHByRijQYMGaf78+Vq9erUqV67s7pIAAAAAeCGPDk4DBgzQhx9+qIULF6pkyZI6fPiwJCk4OFiBgYFurg4AAACAt/DozzhNmTJFKSkpatWqlSpUqOB8fPzxx+4uDQAAAIAX8egzTn+TG/4BAAAAKOQ8+owTAAAAAHgCghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYOHr7gIAAABgFzVysbtLcNo3oYO7S4CFJ71epMLxmuGMEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAAAAAFgQnAAAAADAguAEAAAAABYEJwAAAACwIDgBAAAAgAXBCQAAAAAsCE4AAAAAYEFwAgAAAAALghMAAAAAWBCcAAAAAMCC4AQAAAAAFgQnAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAWf4vgNHnyZEVFRSkgIEA333yztmzZ4u6SAAAAAHgRjw9OH3/8sYYOHaqxY8dq+/btqlevntq2baujR4+6uzQAAAAAXsLjg9Orr76qxx57TL1791atWrU0depUFStWTO+++667SwMAAADgJXzdXcDlnD17Vtu2bVNMTIxzzMfHR9HR0dq4cWO266SlpSktLc35PCUlRZKUmpp6bYvNpYy00+4uAQAA5IKnvHfI5EnvITytN8jKk14vkue8ZjLrMMZc8boeHZx+//13paenKzQ01GU8NDRUP/74Y7brxMXFKTY2Nst4pUqVrkmNAACgcAqOd3cFnove4Ep52mvmxIkTCg4OvqJ1PDo45UVMTIyGDh3qfJ6RkaFjx46pTJkycjgcBVJDamqqKlWqpAMHDigoKKhA9umJ6MNF9OIienERvbiAPlxELy6iFxfQh4voxUX04oK89MEYoxMnTig8PPyK9+fRwals2bIqUqSIjhw54jJ+5MgRhYWFZbuOv7+//P39XcZKlSp1rUq8rKCgIK9+MWeiDxfRi4voxUX04gL6cBG9uIheXEAfLqIXF9GLC660D1d6pimTR98cws/PTw0bNtSqVaucYxkZGVq1apWaNm3qxsoAAAAAeBOPPuMkSUOHDlXPnj3VqFEjNWnSRPHx8Tp16pR69+7t7tIAAAAAeAmPD07dunXTb7/9pjFjxujw4cOqX7++li1bluWGEZ7E399fY8eOzXLJoLehDxfRi4voxUX04gL6cBG9uIheXEAfLqIXF9GLCwq6Dw6Tl3vxAQAAAIAX8ejPOAEAAACAJyA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcMpnkydPVlRUlAICAnTzzTdry5Yt7i4pX8XFxalx48YqWbKkypcvr7vuuku7d+92mfPnn39qwIABKlOmjEqUKKF77703yx8x/uWXX9ShQwcVK1ZM5cuX1/Dhw3X+/PmCPJR8N2HCBDkcDg0ZMsQ55k29OHjwoB566CGVKVNGgYGBqlu3rrZu3epcbozRmDFjVKFCBQUGBio6OlqJiYku2zh27Ji6d++uoKAglSpVSn369NHJkycL+lDyLD09XaNHj1blypUVGBioqlWr6rnnntNf78FTWPuwdu1aderUSeHh4XI4HFqwYIHL8vw67m+++Ub/+Mc/FBAQoEqVKumll1661od2xS7Xi3PnzmnEiBGqW7euihcvrvDwcD388MP69ddfXbbhDb24VP/+/eVwOBQfH+8yXhh6kZs+/PDDD+rcubOCg4NVvHhxNW7cWL/88otzeWH5eWLrxcmTJzVw4EBVrFhRgYGBqlWrlqZOneoypzD0oiDfT61evVoNGjSQv7+/qlWrppkzZ17rw7sitl4cO3ZMgwYNUo0aNRQYGKiIiAgNHjxYKSkpLtspkF4Y5Js5c+YYPz8/8+6775rvvvvOPPbYY6ZUqVLmyJEj7i4t37Rt29bMmDHD7Nq1y+zYscPceeedJiIiwpw8edI5p3///qZSpUpm1apVZuvWreaWW24xzZo1cy4/f/68qVOnjomOjjZff/21WbJkiSlbtqyJiYlxxyHliy1btpioqChz4403mieffNI57i29OHbsmImMjDS9evUymzdvNj///LNZvny52bt3r3POhAkTTHBwsFmwYIHZuXOn6dy5s6lcubI5c+aMc067du1MvXr1zKZNm8y6detMtWrVzIMPPuiOQ8qT559/3pQpU8Z89tlnJikpyXzyySemRIkS5rXXXnPOKax9WLJkiRk1apT59NNPjSQzf/58l+X5cdwpKSkmNDTUdO/e3ezatct89NFHJjAw0EybNq2gDjNXLteL5ORkEx0dbT7++GPz448/mo0bN5omTZqYhg0bumzDG3rxV59++qmpV6+eCQ8PNxMnTnRZVhh6YevD3r17TUhIiBk+fLjZvn272bt3r1m4cKHL+4fC8vPE1ovHHnvMVK1a1SQkJJikpCQzbdo0U6RIEbNw4ULnnMLQi4J6P/Xzzz+bYsWKmaFDh5rvv//eTJo0yRQpUsQsW7asQI/3cmy9+Pbbb80999xjFi1aZPbu3WtWrVplqlevbu69917nNgqqFwSnfNSkSRMzYMAA5/P09HQTHh5u4uLi3FjVtXX06FEjyaxZs8YYc+FNQdGiRc0nn3zinPPDDz8YSWbjxo3GmAvfNH18fMzhw4edc6ZMmWKCgoJMWlpawR5APjhx4oSpXr26WblypWnZsqUzOHlTL0aMGGFuvfXWHJdnZGSYsLAw8/LLLzvHkpOTjb+/v/noo4+MMcZ8//33RpL56quvnHOWLl1qHA6HOXjw4LUrPh916NDBPPLIIy5j99xzj+nevbsxxnv6cOmbofw67jfffNOULl3a5d/GiBEjTI0aNa7xEeXd5cJCpi1bthhJZv/+/cYY7+vF//73P3PdddeZXbt2mcjISJfgVBh7kV0funXrZh566KEc1ymsP0+y60Xt2rXNs88+6zLWoEEDM2rUKGNM4e3FtXo/9fTTT5vatWu77Ktbt26mbdu21/qQ8uzSXmRn7ty5xs/Pz5w7d84YU3C94FK9fHL27Flt27ZN0dHRzjEfHx9FR0dr48aNbqzs2so8TRoSEiJJ2rZtm86dO+fShxtuuEERERHOPmzcuFF169Z1+SPGbdu2VWpqqr777rsCrD5/DBgwQB06dHA5Zsm7erFo0SI1atRI9913n8qXL6+bbrpJb731lnN5UlKSDh8+7NKL4OBg3XzzzS69KFWqlBo1auScEx0dLR8fH23evLngDuYqNGvWTKtWrdKePXskSTt37tT69evVvn17Sd7Th0vl13Fv3LhRLVq0kJ+fn3NO27ZttXv3bh0/fryAjib/paSkyOFwqFSpUpK8qxcZGRnq0aOHhg8frtq1a2dZ7g29yMjI0OLFi3X99derbdu2Kl++vG6++WaXS9i86edJs2bNtGjRIh08eFDGGCUkJGjPnj264447JBXeXlyr91MbN27M8v6kbdu2Hv3e9NJe5DQnKChIvr6+kgquFwSnfPL7778rPT3d5QsmSaGhoTp8+LCbqrq2MjIyNGTIEDVv3lx16tSRJB0+fFh+fn7ONwCZ/tqHw4cPZ9unzGV/J3PmzNH27dsVFxeXZZk39eLnn3/WlClTVL16dS1fvlyPP/64Bg8erFmzZkm6eCyX+/dx+PBhlS9f3mW5r6+vQkJC/ja9GDlypB544AHdcMMNKlq0qG666SYNGTJE3bt3l+Q9fbhUfh13Yfn38ld//vmnRowYoQcffFBBQUGSvKsXL774onx9fTV48OBsl3tDL44ePaqTJ09qwoQJateunVasWKG7775b99xzj9asWSPJu36eTJo0SbVq1VLFihXl5+endu3aafLkyWrRooWkwtmLa/l+Kqc5qampOnPmzLU4nKuSXS8u9fvvv+u5555T3759nWMF1QvfXB8JcIkBAwZo165dWr9+vbtLcYsDBw7oySef1MqVKxUQEODuctwqIyNDjRo10gsvvCBJuummm7Rr1y5NnTpVPXv2dHN1BWfu3LmaPXu2PvzwQ9WuXVs7duzQkCFDFB4e7lV9QO6cO3dO999/v4wxmjJlirvLKXDbtm3Ta6+9pu3bt8vhcLi7HLfJyMiQJHXp0kVPPfWUJKl+/fr68ssvNXXqVLVs2dKd5RW4SZMmadOmTVq0aJEiIyO1du1aDRgwQOHh4VnOFhQW3v5+6q9svUhNTVWHDh1Uq1YtjRs3rmCLE2ec8k3ZsmVVpEiRLHc7OXLkiMLCwtxU1bUzcOBAffbZZ0pISFDFihWd42FhYTp79qySk5Nd5v+1D2FhYdn2KXPZ38W2bdt09OhRNWjQQL6+vvL19dWaNWv0+uuvy9fXV6GhoV7TiwoVKqhWrVouYzVr1nTeESrzWC737yMsLExHjx51WX7+/HkdO3bsb9OL4cOHO8861a1bVz169NBTTz3lPCPpLX24VH4dd2H59yJdDE379+/XypUrnWebJO/pxbp163T06FFFREQ4v4fu379fw4YNU1RUlCTv6EXZsmXl6+tr/R7qDT9Pzpw5o2eeeUavvvqqOnXqpBtvvFEDBw5Ut27d9O9//1tS4evFtX4/ldOcoKAgBQYG5vfhXJWcepHpxIkTateunUqWLKn58+eraNGizmUF1QuCUz7x8/NTw4YNtWrVKudYRkaGVq1apaZNm7qxsvxljNHAgQM1f/58ffHFF6pcubLL8oYNG6po0aIufdi9e7d++eUXZx+aNm2qb7/91uWHYeYbh0t/cHiy2267Td9++6127NjhfDRq1Ejdu3d3/r+39KJ58+ZZbqO6Z88eRUZGSpIqV66ssLAwl16kpqZq8+bNLr1ITk7Wtm3bnHO++OILZWRk6Oabby6Ao7h6p0+flo+P67fVIkWKOH+j7C19uFR+HXfTpk21du1anTt3zjln5cqVqlGjhkqXLl1AR3P1MkNTYmKiPv/8c5UpU8Zlubf0okePHvrmm29cvoeGh4dr+PDhWr58uSTv6IWfn58aN2582e+h3vKz9dy5czp37txlv48Wll4U1Puppk2bumwjc44nvTe19UK68DPjjjvukJ+fnxYtWpTlSp8C60WubyMBqzlz5hh/f38zc+ZM8/3335u+ffuaUqVKudzh4+/u8ccfN8HBwWb16tXm0KFDzsfp06edc/r3728iIiLMF198YbZu3WqaNm1qmjZt6lyeecvIO+64w+zYscMsW7bMlCtXzqNuE5pXf72rnjHe04stW7YYX19f8/zzz5vExEQze/ZsU6xYMfPBBx8450yYMMGUKlXKLFy40HzzzTemS5cu2d6O+qabbjKbN28269evN9WrV/f423D/Vc+ePc11113nvB35p59+asqWLWuefvpp55zC2ocTJ06Yr7/+2nz99ddGknn11VfN119/7bxTXH4cd3JysgkNDTU9evQwu3btMnPmzDHFihXzqNtOG3P5Xpw9e9Z07tzZVKxY0ezYscPl++hf7/blDb3IzqV31TOmcPTC1odPP/3UFC1a1EyfPt0kJiY6b5O8bt065zYKy88TWy9atmxpateubRISEszPP/9sZsyYYQICAsybb77p3EZh6EVBvZ/KvAX38OHDzQ8//GAmT57scbcjt/UiJSXF3HzzzaZu3bpm7969LnPOnz9vjCm4XhCc8tmkSZNMRESE8fPzM02aNDGbNm1yd0n5SlK2jxkzZjjnnDlzxjzxxBOmdOnSplixYubuu+82hw4dctnOvn37TPv27U1gYKApW7asGTZsmPOWkn9nlwYnb+rFf//7X1OnTh3j7+9vbrjhBjN9+nSX5RkZGWb06NEmNDTU+Pv7m9tuu83s3r3bZc4ff/xhHnzwQVOiRAkTFBRkevfubU6cOFGQh3FVUlNTzZNPPmkiIiJMQECAqVKlihk1apTLG+LC2oeEhIRsvzf07NnTGJN/x71z505z6623Gn9/f3PdddeZCRMmFNQh5trlepGUlJTj99GEhATnNryhF9nJLjgVhl7kpg/vvPOOqVatmgkICDD16tUzCxYscNlGYfl5YuvFoUOHTK9evUx4eLgJCAgwNWrUMK+88orJyMhwbqMw9KIg308lJCSY+vXrGz8/P1OlShWXfXgCWy9yes1IMklJSc7tFEQvHP+/YAAAAABADviMEwAAAABYEJwAAAAAwILgBAAAAAAWBCcAAAAAsCA4AQAAAIAFwQkAAAAALAhOAAAAAGBBcAIAAAAAC4ITAECS1KpVKw0ZMsTdZXiUmTNnqlSpUu4uwykqKkrx8fHuLgMAvBLBCQDgdVavXi2Hw6Hk5GR3l5ItTwtsAACCEwAAAABYEZwAwAudOnVKDz/8sEqUKKEKFSrolVdeyTLn+PHjevjhh1W6dGkVK1ZM7du3V2JiosucDRs2qFWrVipWrJhKly6ttm3b6vjx45Kyv6ysfv36GjdunPO5w+HQtGnT1LFjRxUrVkw1a9bUxo0btXfvXrVq1UrFixdXs2bN9NNPP7lsZ+HChWrQoIECAgJUpUoVxcbG6vz58y7bffvtt3X33XerWLFiql69uhYtWiRJ2rdvn1q3bi1JKl26tBwOh3r16pXr3l3NvjMtWrRI1atXV0BAgFq3bq1Zs2Y5z4CtXr1avXv3VkpKihwOhxwOh0vPTp8+rUceeUQlS5ZURESEpk+f7lx29uxZDRw4UBUqVFBAQIAiIyMVFxeX62MDAOSM4AQAXmj48OFas2aNFi5cqBUrVmj16tXavn27y5xevXpp69atWrRokTZu3ChjjO68806dO3dOkrRjxw7ddtttqlWrljZu3Kj169erU6dOSk9Pv6JannvuOT388MPasWOHbrjhBv3zn/9Uv379FBMTo61bt8oYo4EDBzrnr1u3Tg8//LCefPJJff/995o2bZpmzpyp559/3mW7sbGxuv/++/XNN9/ozjvvVPfu3XXs2DFVqlRJ//nPfyRJu3fv1qFDh/Taa6/lqtar3bckJSUlqWvXrrrrrru0c+dO9evXT6NGjXKu26xZM8XHxysoKEiHDh3SoUOH9K9//cu5/JVXXlGjRo309ddf64knntDjjz+u3bt3S5Jef/11LVq0SHPnztXu3bs1e/ZsRUVF5f6LAQDImQEAeJUTJ04YPz8/M3fuXOfYH3/8YQIDA82TTz5pjDFmz549RpLZsGGDc87vv/9uAgMDnes9+OCDpnnz5jnuJzIy0kycONFlrF69embs2LHO55LM//3f/zmfb9y40Ugy77zzjnPso48+MgEBAc7nt912m3nhhRdctvv++++bChUq5LjdkydPGklm6dKlxhhjEhISjCRz/PjxHOs3xpgZM2aY4ODgfN33iBEjTJ06dVy2MWrUKJd6Lt1vpsjISPPQQw85n2dkZJjy5cubKVOmGGOMGTRokGnTpo3JyMi47HEBAK6cr9sSGwDALX766SedPXtWN998s3MsJCRENWrUcD7/4Ycf5Ovr6zKnTJkyqlGjhn744QdJF8443XfffVddz4033uj8/9DQUElS3bp1Xcb+/PNPpaamKigoSDt37tSGDRtczvKkp6frzz//1OnTp1WsWLEs2y1evLiCgoJ09OjRq6o1P/a9e/duNW7c2GW7TZo0yXUNf922w+FQWFiYc9u9evXS7bffrho1aqhdu3bq2LGj7rjjjis/UABAFgQnAECeBAYGXna5j4+PjDEuY5mX+f1V0aJFnf/vcDhyHMvIyJAknTx5UrGxsbrnnnuybCsgICDb7WZuJ3MbeeXOfedm2w0aNFBSUpKWLl2qzz//XPfff7+io6M1b968fNk3AHgzghMAeJmqVauqaNGi2rx5syIiIiRduBHEnj171LJlS0lSzZo1df78eW3evFnNmjWTJP3xxx/avXu3atWqJenCmY9Vq1YpNjY22/2UK1dOhw4dcj5PTU1VUlLSVdffoEED7d69W9WqVcvzNvz8/CTpij+PlR/7rlGjhpYsWeIy9tVXX2Wp70pryxQUFKRu3bqpW7du6tq1q9q1a6djx44pJCQkzzUDAAhOAOB1SpQooT59+mj48OEqU6aMypcvr1GjRsnH5+L9gqpXr64uXbroscce07Rp01SyZEmNHDlS1113nbp06SJJiomJUd26dfXEE0+of//+8vPzU0JCgu677z6VLVtWbdq00cyZM9WpUyeVKlVKY8aMUZEiRa66/jFjxqhjx46KiIhQ165d5ePjo507d2rXrl0aP358rrYRGRkph8Ohzz77THfeeacCAwNVokSJAtl3v3799Oqrr2rEiBHq06ePduzYoZkzZ0q6eHYtKipKJ0+e1KpVq1SvXj0VK1bMeRng5bz66quqUKGCbrrpJvn4+OiTTz5RWFgYfxMKAPIBd9UDAC/08ssv6x//+Ic6deqk6Oho3XrrrWrYsKHLnBkzZqhhw4bq2LGjmjZtKmOMlixZ4rxU7Prrr9eKFSu0c+dONWnSRE2bNtXChQvl63vhd3IxMTFq2bKlOnbsqA4dOuiuu+5S1apVr7r2tm3b6rPPPtOKFSvUuHFj3XLLLZo4caIiIyNzvY3rrrtOsbGxGjlypEJDQ13u2net9125cmXNmzdPn376qW688UZNmTLFeVc9f39/SRfurNe/f39169ZN5cqV00svvZSrbZcsWVIvvfSSGjVqpMaNG2vfvn1asmSJSygGAOSNw1x6AToAAChQzz//vKZOnaoDBw64uxQAQA64VA8AgAL25ptvqnHjxipTpow2bNigl19+OddnvQAA7kFwAgCggCUmJmr8+PE6duyYIiIiNGzYMMXExLi7LADAZXCpHgAAAABY8GlRAAAAALAgOAEAAACABcEJAAAAACwITgAAAABgQXACAAAAAAuCEwAAAABYEJwAAAAAwILgBAAAAAAW/w9AZf+j1c4n+AAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10, 6))   # make the plot larger\n",
    "plot_doc_lengths_hist(fig, ax, corpus_small, y_log=False, bins=20)  # use 20 bins\n",
    "ax.set_xticks(range(0, 2201, 200))    # set x axis ticks and range\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, this is a right skewed distribution with a few quite large documents with more than 1500 tokens, but also several documents that are very small (less than 100 tokens). The above example shows nicely how you can adjust the plot before and after applying tmtoolkit's plotting function.\n",
    "\n",
    "The function [plot_doc_frequencies_hist](api.rst#tmtoolkit.corpus.visualize.plot_doc_frequencies_hist) lets us plot the distribution of document frequencies of each token type. This time, we stick with the log10 scale on the y-axis, because otherwise the token types with high document frequency would be hardly visible in the plot, since there are so few of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.394243Z",
     "iopub.status.busy": "2023-05-03T13:22:48.394082Z",
     "iopub.status.idle": "2023-05-03T13:22:48.655441Z",
     "shell.execute_reply": "2023-05-03T13:22:48.654888Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAHHCAYAAAC2rPKaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABGO0lEQVR4nO3deVyVdf7//+cB2UwBDQV3FM1coURNzNyYMTVTKzPHMVAzNczMltHp41Y2WraYxYzZTDrtZabWWC7hmpk7muGChmkmkgsiaCDw/v3Rz/PtCCgHDx64eNxvN25x3teb9/U6b06Hp9d1vc9lM8YYAQAAWJCHuwsAAAAoLQQdAABgWQQdAABgWQQdAABgWQQdAABgWQQdAABgWQQdAABgWQQdAABgWQQdAABgWQQdlHuhoaGKjY11dxmWN2vWLDVq1Eienp6KiIhw+ue7dOmiLl26uLyuiurdd9/VzTffLC8vLwUGBrq7nDLn8OHDstlsWrBggbtLgZsRdFCmLFiwQDabTdu2bSt0e5cuXdSyZctr3s+XX36pqVOnXvM4FcXKlSv19NNPq2PHjpo/f77+8Y9/uLskS/n22281depUpaenF6v/vn37FBsbq7CwML311luaN29e6RYIlGOV3F0AcK32798vDw/nMvuXX36p+Ph4wk4xrV69Wh4eHvrPf/4jb29vd5djOd9++62mTZum2NjYYh2dWbt2rfLz8/Xaa6+pcePGpV9gOdSgQQNduHBBXl5e7i4FbsYRHZR7Pj4+5e7NLCsry90lOCUtLU1+fn6EnDIiLS1Nkq4aiowxunDhwnWoqOyx2Wzy9fWVp6enu0uBmxF0UO5dfo3OxYsXNW3aNDVp0kS+vr668cYbdfvtt2vVqlWSpNjYWMXHx0v6/c3w0tclWVlZeuKJJ1SvXj35+PioadOmeumll2SMcdjvhQsXNHbsWAUFBalq1aq6++67dezYMdlsNocjRVOnTpXNZlNSUpL+8pe/qFq1arr99tslSbt371ZsbKwaNWokX19fhYSEaNiwYTp16pTDvi6NceDAAf31r39VQECAatSooUmTJskYo6NHj6pv377y9/dXSEiIXn755WLNXW5urp577jmFhYXJx8dHoaGh+vvf/67s7Gx7H5vNpvnz5ysrK8s+V1e77mHevHkKCwuTn5+f2rVrpw0bNhTaLy0tTcOHD1dwcLB8fX0VHh6u//73vwX6XTp60apVK/n6+qpGjRq688477ac4r3Q9RlG/j2uZy+zsbE2ZMkWNGzeWj4+P6tWrp6efftph3i7te8yYMVqyZIlatmwpHx8ftWjRQsuXL3eo56mnnpIkNWzY0D7Hhw8fLnTOQkNDNWXKFElSjRo1HJ5faGio7rrrLq1YsUKRkZHy8/PTm2++KUlKT0/XuHHj7K/rxo0b64UXXlB+fr7D+Onp6YqNjVVAQIACAwMVExOjxMTEAvNb1DVXsbGxCg0NdWjLz8/X7Nmz1aJFC/n6+io4OFgjR47UmTNnCjy3u+66S998843atWsnX19fNWrUSO+8806B/aSnp+vxxx9XaGiofHx8VLduXT344IM6efKkpKJfE/v27dN9992n6tWry9fXV5GRkfr8888d+lztPQTlC6euUCadPXvW/ob1RxcvXrzqz06dOlUzZszQQw89pHbt2ikjI0Pbtm3Tjh079Kc//UkjR47UL7/8olWrVundd991+FljjO6++26tWbNGw4cPV0REhFasWKGnnnpKx44d06uvvmrvGxsbq08++URDhgzRbbfdpnXr1ql3795F1jVgwAA1adJE//jHP+yhadWqVfrxxx81dOhQhYSE6IcfftC8efP0ww8/6LvvvnMIYJI0cOBANWvWTDNnztSyZcs0ffp0Va9eXW+++aa6deumF154Qe+//76efPJJtW3bVnfccccV5+qhhx7Sf//7X91333164okntHnzZs2YMUN79+7V4sWLJf1+0eu8efO0ZcsW/fvf/5YkRUVFFTnmf/7zH40cOVJRUVEaN26cfvzxR919992qXr266tWrZ+934cIFdenSRQcPHtSYMWPUsGFDLVy4ULGxsUpPT9djjz1m7zt8+HAtWLBAPXv21EMPPaTc3Fxt2LBB3333nSIjI6/4HItS0rnMz8/X3XffrW+++UYPP/ywmjVrpu+//16vvvqqDhw4oCVLljjs55tvvtFnn32mRx55RFWrVtWcOXN077336siRI7rxxht1zz336MCBA/rwww/16quvKigoSNLvIaYws2fP1jvvvKPFixfrX//6l6pUqaLWrVvbt+/fv1+DBg3SyJEjNWLECDVt2lTnz59X586ddezYMY0cOVL169fXt99+q4kTJ+r48eOaPXu2pN9f/3379tU333yjUaNGqVmzZlq8eLFiYmJKNMeXjBw5UgsWLNDQoUM1duxYpaSk6I033tDOnTu1ceNGhyOyBw8e1H333afhw4crJiZGb7/9tmJjY9WmTRu1aNFCkpSZmalOnTpp7969GjZsmG699VadPHlSn3/+uX7++Wf7HF7uhx9+UMeOHVWnTh1NmDBBN9xwgz755BP169dPixYtUv/+/SVd/T0E5YwBypD58+cbSVf8atGihcPPNGjQwMTExNgfh4eHm969e19xP3Fxcaawl/+SJUuMJDN9+nSH9vvuu8/YbDZz8OBBY4wx27dvN5LMuHHjHPrFxsYaSWbKlCn2tilTphhJZtCgQQX2d/78+QJtH374oZFk1q9fX2CMhx9+2N6Wm5tr6tata2w2m5k5c6a9/cyZM8bPz89hTgqTmJhoJJmHHnrIof3JJ580kszq1avtbTExMeaGG2644njGGJOTk2Nq1qxpIiIiTHZ2tr193rx5RpLp3LmzvW327NlGknnvvfccfr5Dhw6mSpUqJiMjwxhjzOrVq40kM3bs2AL7y8/PN8YYk5KSYiSZ+fPnF+hT1O+jpHP57rvvGg8PD7NhwwaH/cydO9dIMhs3bnTYt7e3t/11Y4wxu3btMpLM66+/bm+bNWuWkWRSUlIK1F+YS8/h119/dWhv0KCBkWSWL1/u0P7cc8+ZG264wRw4cMChfcKECcbT09McOXLEGPP/Xv8vvviiw9x06tSpwPx27tzZ4fd5SUxMjGnQoIH98YYNG4wk8/777zv0W758eYH2S/X/8bWflpZmfHx8zBNPPGFvmzx5spFkPvvsswL7v9Jronv37qZVq1bmt99+c+gfFRVlmjRpYm8rznsIyg9OXaFMio+P16pVqwp8/fFfrkUJDAzUDz/8oOTkZKf3++WXX8rT01Njx451aH/iiSdkjNFXX30lSfZTD4888ohDv0cffbTIsUeNGlWgzc/Pz/79b7/9ppMnT+q2226TJO3YsaNA/4ceesj+vaenpyIjI2WM0fDhw+3tgYGBatq0qX788ccia5F+f66SNH78eIf2J554QpK0bNmyK/58YbZt26a0tDSNGjXK4XqeS6dCLt9/SEiIBg0aZG/z8vLS2LFjlZmZqXXr1kmSFi1aJJvNZj9d80eXH/FyRknncuHChWrWrJluvvlmnTx50v7VrVs3SdKaNWsc9hMdHa2wsDD749atW8vf3/+qv5+SatiwoXr06OHQtnDhQnXq1EnVqlVzqDk6Olp5eXlav369pN9/J5UqVdLo0aPtP+vp6XnF1/XVLFy4UAEBAfrTn/7ksO82bdqoSpUqBearefPm6tSpk/1xjRo1CvwOFi1apPDwcPsRmD8q6jVx+vRprV69Wvfff7/OnTtnr+PUqVPq0aOHkpOTdezYMUnX9h6CsodTVyiT2rVrV+gpiUtv1Ffy7LPPqm/fvrrpppvUsmVL3XnnnRoyZEixQtJPP/2k2rVrq2rVqg7tzZo1s2+/9F8PDw81bNjQod+VVsBc3lf6/c132rRp+uijj+wXmF5y9uzZAv3r16/v8DggIEC+vr4FDtUHBAQUuM7ncpeew+U1h4SEKDAw0P5cnXHpZ5o0aeLQ7uXlpUaNGhXo26RJkwIr5i6f60OHDql27dqqXr260/VcSUnnMjk5WXv37i3y1NLlv8fL9yP9/jq+/PoUVynsdZacnKzdu3dfteaffvpJtWrVUpUqVRy2N23atMT1JCcn6+zZs6pZs+YV931Jcebr0KFDuvfee52q4+DBgzLGaNKkSZo0aVKRtdSpU+ea3kNQ9hB0YDl33HGHDh06pKVLl2rlypX697//rVdffVVz5851+Ff89fbHozeX3H///fr222/11FNPKSIiQlWqVFF+fr7uvPPOAheJSip0BUlRq0rMZRdPF+VajoqUFUU9h7y8vCJ/pqRzmZ+fr1atWumVV14ptO8fr0Mq7piuVNjrLD8/X3/605/09NNPF/ozN910k9P7sdlshT6Hy+c8Pz9fNWvW1Pvvv1/oOJeHr9Kar0v/Pz355JMFjnhdcin0l9X3EJQMQQeWVL16dQ0dOlRDhw5VZmam7rjjDk2dOtX+JlXUH8YGDRro66+/1rlz5xyO6uzbt8++/dJ/8/PzlZKS4nD04uDBg8Wu8cyZM0pISNC0adM0efJke/v1Olx+6TkkJyfbj6JI0okTJ5Senm5/rs6OKf3+HC6dypF+v4g8JSVF4eHhDn13796t/Px8h6M6l891WFiYVqxYodOnTxd5VKdatWqSVOAD90pyVOpqwsLCtGvXLnXv3t1lIbG0w2ZYWJgyMzMVHR19xX4NGjRQQkKCMjMzHY7q7N+/v0DfatWqFXr67fI5DwsL09dff62OHTsWGsJKIiwsTHv27HHqZy4dUfTy8rrqPEhXfw9B+cE1OrCcy0/ZVKlSRY0bN3ZY+nvDDTdIKviHsVevXsrLy9Mbb7zh0P7qq6/KZrOpZ8+ekmT/F+E///lPh36vv/56seu89C/Xy/+lemkFTGnr1atXofu7dKTiSivIihIZGakaNWpo7ty5ysnJsbcvWLCg0LlOTU3Vxx9/bG/Lzc3V66+/ripVqqhz586SpHvvvVfGGE2bNq3A/i7Nnb+/v4KCguzXmlxy+e/HFe6//34dO3ZMb731VoFtFy5cKNFnJBX1enSV+++/X5s2bdKKFSsKbEtPT1dubq6k338nubm5+te//mXfnpeXV+jrOiwsTPv27dOvv/5qb9u1a5c2btxYYN95eXl67rnnCoyRm5tboud87733ateuXfaVgX9U1JGfmjVrqkuXLnrzzTd1/PjxAtv/+DyK8x6C8oMjOrCc5s2bq0uXLmrTpo2qV6+ubdu26dNPP9WYMWPsfdq0aSNJGjt2rHr06CFPT0898MAD6tOnj7p27apnnnlGhw8fVnh4uFauXKmlS5dq3Lhx9otK27Rpo3vvvVezZ8/WqVOn7MvLDxw4IKl4/0L39/fXHXfcoRdffFEXL15UnTp1tHLlSqWkpJTCrBQUHh6umJgYzZs3T+np6ercubO2bNmi//73v+rXr5+6du3q9JheXl6aPn26Ro4cqW7dumngwIFKSUnR/PnzC1yj8/DDD+vNN99UbGystm/frtDQUH366afauHGjZs+ebT+i1rVrVw0ZMkRz5sxRcnKy/bTehg0b1LVrV/vv9aGHHtLMmTP10EMPKTIyUuvXr7f/PlxpyJAh+uSTTzRq1CitWbNGHTt2VF5envbt26dPPvnE/hk2zrj0enzmmWf0wAMPyMvLS3369LEHoGv11FNP6fPPP9ddd91lX6qdlZWl77//Xp9++qkOHz6soKAg9enTRx07dtSECRN0+PBhNW/eXJ999lmh14sNGzZMr7zyinr06KHhw4crLS1Nc+fOVYsWLZSRkWHv17lzZ40cOVIzZsxQYmKi/vznP8vLy0vJyclauHChXnvtNd13331OP59PP/1UAwYM0LBhw9SmTRudPn1an3/+uebOnetw5PCP4uPjdfvtt6tVq1YaMWKEGjVqpBMnTmjTpk36+eeftWvXLknFew9BOeKWtV5AES4tL9+6dWuh2zt37nzV5eXTp0837dq1M4GBgcbPz8/cfPPN5vnnnzc5OTn2Prm5uebRRx81NWrUMDabzWGp+blz58zjjz9uateubby8vEyTJk3MrFmz7MtWL8nKyjJxcXGmevXqpkqVKqZfv35m//79RpLDEuWilgIbY8zPP/9s+vfvbwIDA01AQIAZMGCA+eWXX4pcEn35GEUt+y5sngpz8eJFM23aNNOwYUPj5eVl6tWrZyZOnOiw/PZK+ynKP//5T9OwYUPj4+NjIiMjzfr16wtdjnzixAkzdOhQExQUZLy9vU2rVq0KXSKem5trZs2aZW6++Wbj7e1tatSoYXr27Gm2b99u73P+/HkzfPhwExAQYKpWrWruv/9+k5aWVipzmZOTY1544QXTokUL4+PjY6pVq2batGljpk2bZs6ePWvvJ8nExcUVGPPy16wxvy8Br1OnjvHw8LjqUvMrLS8valn0uXPnzMSJE03jxo2Nt7e3CQoKMlFRUeall15y+H/j1KlTZsiQIcbf398EBASYIUOGmJ07dxa6fP+9994zjRo1Mt7e3iYiIsKsWLGiwPLyS+bNm2fatGlj/Pz8TNWqVU2rVq3M008/bX755Zer1l/Ya+fUqVNmzJgxpk6dOsbb29vUrVvXxMTEmJMnTxpjiv7IgUOHDpkHH3zQhISEGC8vL1OnTh1z1113mU8//dTepzjvISg/bMaU0hVxQAWUmJioW265Re+9954GDx7s7nIAlzh8+LAaNmyo+fPnO3wKOVAecI0OUEKF3UNo9uzZ8vDwuOonEgMArg+u0QFK6MUXX9T27dvVtWtXVapUSV999ZW++uorPfzwwwWWGAMA3IOgA5RQVFSUVq1apeeee06ZmZmqX7++pk6dqmeeecbdpQEA/n9cowMAACyLa3QAAIBlEXQAAIBlVfhrdPLz8/XLL7+oatWqlrjnDwAAFYExRufOnVPt2rUL3Bz4jyp80Pnll19YIQMAQDl19OhR1a1bt8jtFT7oXPqY+aNHj8rf39/N1QAAgOLIyMhQvXr1HG7AXJgKH3Quna7y9/cn6AAAUM5c7bITLkYGAACWRdABAACWZZmgc/78eTVo0EBPPvmku0sBAABlhGWCzvPPP6/bbrvN3WUAAIAyxBJBJzk5Wfv27VPPnj3dXQoAAChD3B501q9frz59+qh27dqy2WxasmRJgT7x8fEKDQ2Vr6+v2rdvry1btjhsf/LJJzVjxozrVDEAACgv3B50srKyFB4ervj4+EK3f/zxxxo/frymTJmiHTt2KDw8XD169FBaWpokaenSpbrpppt00003Xc+yAQBAOVCm7l5us9m0ePFi9evXz97Wvn17tW3bVm+88Yak32/ZUK9ePT366KOaMGGCJk6cqPfee0+enp7KzMzUxYsX9cQTT2jy5MmF7iM7O1vZ2dn2x5c+cOjs2bN8jg4AAOVERkaGAgICrvr32+1HdK4kJydH27dvV3R0tL3Nw8ND0dHR2rRpkyRpxowZOnr0qA4fPqyXXnpJI0aMKDLkXOofEBBg/+L2DwAAWFeZDjonT55UXl6egoODHdqDg4OVmppaojEnTpyos2fP2r+OHj3qilIBAEAZZKlbQMTGxl61j4+Pj3x8fEq/GAAA4HZl+ohOUFCQPD09deLECYf2EydOKCQkxE1VAQCA8qJMBx1vb2+1adNGCQkJ9rb8/HwlJCSoQ4cObqwMAACUB24/dZWZmamDBw/aH6ekpCgxMVHVq1dX/fr1NX78eMXExCgyMlLt2rXT7NmzlZWVpaFDh17TfuPj4xUfH6+8vLxrfQoAAKCMcvvy8rVr16pr164F2mNiYrRgwQJJ0htvvKFZs2YpNTVVERERmjNnjtq3b++S/Rd3eVpJhE5Y5tLxrofDM3u7uwQAAK6quH+/3R503I2g44igAwAoDyzxOToAAADXgqADAAAsi6ADAAAsq8IGnfj4eDVv3lxt27Z1dykAAKCUVNigExcXp6SkJG3dutXdpQAAgFJSYYMOAACwPoIOAACwLIIOAACwLIIOAACwrAobdFh1BQCA9VXYoMOqKwAArK/CBh0AAGB9BB0AAGBZBB0AAGBZBB0AAGBZBB0AAGBZFTbosLwcAADrq7BBh+XlAABYX4UNOgAAwPoIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIqbNDhAwMBALC+Cht0+MBAAACsr8IGHQAAYH0EHQAAYFkEHQAAYFkEHQAAYFkEHQAAYFkEHQAAYFkEHQAAYFkEHQAAYFkVNujwycgAAFhfhQ06fDIyAADWV2GDDgAAsD6CDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsKwKG3S4ezkAANZXYYMOdy8HAMD6KmzQAQAA1kfQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAlkXQAQAAllVhg058fLyaN2+utm3bursUAABQSips0ImLi1NSUpK2bt3q7lIAAEApqbBBBwAAWB9BBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWFa5Dzrp6emKjIxURESEWrZsqbfeesvdJQEAgDKikrsLuFZVq1bV+vXrVblyZWVlZally5a65557dOONN7q7NAAA4Gbl/oiOp6enKleuLEnKzs6WMUbGGDdXBQAAygK3B53169erT58+ql27tmw2m5YsWVKgT3x8vEJDQ+Xr66v27dtry5YtDtvT09MVHh6uunXr6qmnnlJQUNB1qh4AAJRlbg86WVlZCg8PV3x8fKHbP/74Y40fP15TpkzRjh07FB4erh49eigtLc3eJzAwULt27VJKSoo++OADnThx4nqVDwAAyjC3B52ePXtq+vTp6t+/f6HbX3nlFY0YMUJDhw5V8+bNNXfuXFWuXFlvv/12gb7BwcEKDw/Xhg0bitxfdna2MjIyHL4AAIA1uT3oXElOTo62b9+u6Ohoe5uHh4eio6O1adMmSdKJEyd07tw5SdLZs2e1fv16NW3atMgxZ8yYoYCAAPtXvXr1SvdJAAAAtylR0Ll48aKOHj2q/fv36/Tp066uye7kyZPKy8tTcHCwQ3twcLBSU1MlST/99JM6deqk8PBwderUSY8++qhatWpV5JgTJ07U2bNn7V9Hjx4ttfoBAIB7FXt5+blz5/Tee+/po48+0pYtW5STkyNjjGw2m+rWras///nPevjhh9W2bdvSrLeAdu3aKTExsdj9fXx85OPjU3oFAQCAMqNYR3ReeeUVhYaGav78+YqOjtaSJUuUmJioAwcOaNOmTZoyZYpyc3P15z//WXfeeaeSk5NdUlxQUJA8PT0LXFx84sQJhYSEuGQfAADAuop1RGfr1q1av369WrRoUej2du3aadiwYZo7d67mz5+vDRs2qEmTJtdcnLe3t9q0aaOEhAT169dPkpSfn6+EhASNGTPmmscHAADWVqyg8+GHHxZrMB8fH40aNcqpAjIzM3Xw4EH745SUFCUmJqp69eqqX7++xo8fr5iYGEVGRqpdu3aaPXu2srKyNHToUKf2c7n4+HjFx8crLy/vmsYBAABll82U8GOEDx48qEOHDumOO+6Qn5+f/XodZ61du1Zdu3Yt0B4TE6MFCxZIkt544w3NmjVLqampioiI0Jw5c9S+ffuSlF1ARkaGAgICdPbsWfn7+7tkzEtCJyxz6XjXw+GZvd1dAgAAV1Xcv99OB51Tp05p4MCBWr16tWw2m5KTk9WoUSMNGzZM1apV08svv3zNxV9PBB1HBB0AQHlQ3L/fTi8vf/zxx1WpUiUdOXLEfo8pSRo4cKCWL19esmoBAABKgdN3L1+5cqVWrFihunXrOrQ3adJEP/30k8sKAwAAuFZOH9HJyspyOJJzyenTp/l8GgAAUKY4HXQ6deqkd955x/7YZrMpPz9fL774YqEXFZdV8fHxat68+XX/gEMAAHD9OH3q6sUXX1T37t21bds25eTk6Omnn9YPP/yg06dPa+PGjaVRY6mIi4tTXFyc/WImAABgPU4f0WnZsqUOHDig22+/XX379lVWVpbuuece7dy5U2FhYaVRIwAAQIk4fURHkgICAvTMM8+4uhYAAACXKlbQ2b17d7EHbN26dYmLAQAAcKViBZ2IiAjZbDZd7bMFbTYbt1QAAABlRrGCTkpKSmnXcd1xrysAAKyvxPe6sgpuAeGIW0AAAMqD4v79LtHFyJKUlJSkI0eOKCcnx6H97rvvLumQAAAALuV00Pnxxx/Vv39/ff/99w7X7Vy6czmnggAAQFnh9OfoPPbYY2rYsKHS0tJUuXJl/fDDD1q/fr0iIyO1du3aUigRAACgZJw+orNp0yatXr1aQUFB8vDwkIeHh26//XbNmDFDY8eO1c6dO0ujTgAAAKc5fUQnLy9PVatWlSQFBQXpl19+kSQ1aNBA+/fvd211AAAA18DpIzotW7bUrl271LBhQ7Vv314vvviivL29NW/ePDVq1Kg0aiwVLC8HAMD6nD6i83//93/Kz8+XJD377LNKSUlRp06d9OWXX2rOnDkuL7C0xMXFKSkpSVu3bnV3KQAAoJQ4fUSnR48e9u8bN26sffv26fTp06pWrZp95RUAAEBZ4PQRnbNnz+r06dMObdWrV9eZM2eUkZHhssIAAACuldNB54EHHtBHH31UoP2TTz7RAw884JKiAAAAXMHpoLN582Z17dq1QHuXLl20efNmlxQFAADgCk4HnezsbOXm5hZov3jxoi5cuOCSogAAAFzB6aDTrl07zZs3r0D73Llz1aZNG5cUBQAA4ApOr7qaPn26oqOjtWvXLnXv3l2SlJCQoK1bt2rlypUuLxAAAKCknD6i07FjR23atEn16tXTJ598oi+++EKNGzfW7t271alTp9KosVTEx8erefPmatu2rbtLAQAApcRmLt1+vILKyMhQQECAzp49K39/f5eOHTphmUvHux4Oz+zt7hIAALiq4v79dvqIzo4dO/T999/bHy9dulT9+vXT3//+d+Xk5JSsWgAAgFLgdNAZOXKkDhw4IEn68ccfNXDgQFWuXFkLFy7U008/7fICAQAASsrpoHPgwAFFRERIkhYuXKjOnTvrgw8+0IIFC7Ro0SJX1wcAAFBiTgcdY4z9pp5ff/21evXqJUmqV6+eTp486drqAAAAroHTQScyMlLTp0/Xu+++q3Xr1ql3798vXk1JSVFwcLDLCwQAACgpp4PO7NmztWPHDo0ZM0bPPPOMGjduLEn69NNPFRUV5fICAQAASsrpDwxs3bq1w6qrS2bNmiVPT0+XFAUAAOAKTgedovj6+rpqKAAAAJdw+tQVAABAeVFhgw63gAAAwPoqbNCJi4tTUlKStm7d6u5SAABAKamwQQcAAFifU0EnKSlJjzzyiG655RbVqlVLtWrV0i233KJHHnlESUlJpVUjAABAiRR71dVXX32lfv366dZbb1Xfvn3tHw544sQJrVq1SrfeequWLl2qHj16lFqxAAAAzrAZY0xxOoaHh6tv37569tlnC90+depUffbZZ9q9e7dLCyxtxb3Ne0mETljm0vGuh8Mze7u7BAAArqq4f7+LferqwIEDGjx4cJHbBw0apOTkZOeqBAAAKEXFDjqhoaFatqzoIxTLli1TgwYNXFIUAACAKxT7Gp1nn31Wf/nLX7R27VpFR0c7XKOTkJCg5cuX64MPPii1QgEAAJxV7KAzYMAA1alTR3PmzNHLL7+s1NRUSVJISIg6dOigtWvXqkOHDqVWKAAAgLOcutdVVFQUdygHAADlBh8YCAAALMtlQWfv3r1q1KiRq4YDAAC4Zi4LOjk5Ofrpp59cNVyp46aeAABYX7Gv0Rk/fvwVt//666/XXMz1FBcXp7i4OPsHDgEAAOspdtB57bXXFBERUeSnD2ZmZrqsKAAAAFcodtBp3LixHn/8cf31r38tdHtiYqLatGnjssIAAACuVbGv0YmMjNT27duL3G6z2VTM22YBAABcF8U+ovPyyy8rOzu7yO3h4eHKz893SVEAAACuUOygExISUpp1AAAAuBwfGAgAACzLqVtASFK1atVks9kKtNtsNvn6+qpx48aKjY3V0KFDXVIgAABASTkddCZPnqznn39ePXv2VLt27SRJW7Zs0fLlyxUXF6eUlBSNHj1aubm5GjFihMsLBgAAKC6ng84333yj6dOna9SoUQ7tb775plauXKlFixapdevWmjNnDkEHAAC4ldPX6KxYsULR0dEF2rt3764VK1ZIknr16qUff/zx2qsDAAC4Bk4f0alevbq++OILPf744w7tX3zxhapXry5JysrKUtWqVV1TIa6r0AnL3F2C0w7P7O3uEgAAZZTTQWfSpEkaPXq01qxZY79GZ+vWrfryyy81d+5cSdKqVavUuXNn11YKAADgJKeDzogRI9S8eXO98cYb+uyzzyRJTZs21bp16xQVFSVJeuKJJ1xbJQAAQAk4HXQkqWPHjurYsaOrawEAAHCpEgWdvLw8LVmyRHv37pUktWjRQnfffbc8PT1dWhwAAMC1cDroHDx4UL169dKxY8fUtGlTSdKMGTNUr149LVu2TGFhYS4vEgAAoCScXl4+duxYhYWF6ejRo9qxY4d27NihI0eOqGHDhho7dmxp1AgAAFAiTh/RWbdunb777jv7UnJJuvHGGzVz5sxydd1OfHy84uPjlZeX5+5SAABAKXH6iI6Pj4/OnTtXoD0zM1Pe3t4uKep6iIuLU1JSkrZu3eruUgAAQClxOujcddddevjhh7V582YZY2SM0XfffadRo0bp7rvvLo0aAQAASsTpoDNnzhyFhYWpQ4cO8vX1la+vrzp27KjGjRvrtddeK40aAQAASsTpa3QCAwO1dOlSJScna9++fZKkZs2aqXHjxi4vDgAA4FqU6HN0JKlJkyZq0qSJK2sBAABwqWIFnfHjxxd7wFdeeaXExQAAALhSsYLOzp07izWYzWa7pmIAAABcqVhBZ82aNaVdBwAAgMs5veoKAACgvChW0Bk1apR+/vnnYg348ccf6/3337+mogAAAFyhWKeuatSooRYtWqhjx47q06ePIiMjVbt2bfn6+urMmTNKSkrSN998o48++ki1a9fWvHnzSrtuAACAqypW0Hnuuec0ZswY/fvf/9Y///lPJSUlOWyvWrWqoqOjNW/ePN15552lUigAAICzbMYY4+wPnTlzRkeOHNGFCxcUFBSksLCwcrviKiMjQwEBATp79qz8/f1dOnbohGUuHQ+FOzyzt7tLAABcZ8X9+12iDwysVq2aqlWrVuLiAAAArgdWXQEAAMsi6AAAAMsi6AAAAMsi6AAAAMtyOuh069ZN6enpBdozMjLUrVs3V9QEAADgEk4HnbVr1yonJ6dA+2+//aYNGza4pCgAAABXKPby8t27d9u/T0pKUmpqqv1xXl6eli9frjp16ri2OgAAgGtQ7KATEREhm80mm81W6CkqPz8/vf766y4tDgAA4FoUO+ikpKTIGKNGjRppy5YtqlGjhn2bt7e3atasKU9Pz1IpEgAAoCSKHXQaNGggScrPzy+1YgAAAFypRLeASE5O1po1a5SWllYg+EyePNklhQEAAFwrp4POW2+9pdGjRysoKEghISEON/O02WwEHQAAUGY4HXSmT5+u559/Xn/7299Kox6nHT16VEOGDFFaWpoqVaqkSZMmacCAAe4uCwAAlAFOB50zZ86UqSBRqVIlzZ49WxEREUpNTVWbNm3Uq1cv3XDDDe4uDQAAuJnTHxg4YMAArVy5sjRqKZFatWopIiJCkhQSEqKgoCCdPn3avUUBAIAywekjOo0bN9akSZP03XffqVWrVvLy8nLYPnbsWKfGW79+vWbNmqXt27fr+PHjWrx4sfr16+fQJz4+XrNmzVJqaqrCw8P1+uuvq127dgXG2r59u/Ly8lSvXj1nnxYAALAgp4POvHnzVKVKFa1bt07r1q1z2Gaz2ZwOOllZWQoPD9ewYcN0zz33FNj+8ccfa/z48Zo7d67at2+v2bNnq0ePHtq/f79q1qxp73f69Gk9+OCDeuutt5x9SgAAwKKcDjopKSkuLaBnz57q2bNnkdtfeeUVjRgxQkOHDpUkzZ07V8uWLdPbb7+tCRMmSJKys7PVr18/TZgwQVFRUVfcX3Z2trKzs+2PMzIyXPAsAABAWeT0NTrXU05OjrZv367o6Gh7m4eHh6Kjo7Vp0yZJkjFGsbGx6tatm4YMGXLVMWfMmKGAgAD7F6e5AACwLqeP6AwbNuyK299+++0SF3O5kydPKi8vT8HBwQ7twcHB2rdvnyRp48aN+vjjj9W6dWstWbJEkvTuu++qVatWhY45ceJEjR8/3v44IyODsAMAgEWVaHn5H128eFF79uxRenp6oTf7LG233367U7el8PHxkY+PTylWBAAAygqng87ixYsLtOXn52v06NEKCwtzSVGXBAUFydPTUydOnHBoP3HihEJCQly6LwAAYD0uuUbHw8ND48eP16uvvuqK4ey8vb3Vpk0bJSQk2Nvy8/OVkJCgDh06uHRfAADAekp0U8/CHDp0SLm5uU7/XGZmpg4ePGh/nJKSosTERFWvXl3169fX+PHjFRMTo8jISLVr106zZ89WVlaWfRVWScXHxys+Pl55eXnXNA4AACi7nA46f7yQV/p91dPx48e1bNkyxcTEOF3Atm3b1LVr1wLjx8TEaMGCBRo4cKB+/fVXTZ48WampqYqIiNDy5csLXKDsrLi4OMXFxSkjI0MBAQHXNBYAACibbMYY48wP/DGUSL+ftqpRo4a6deumYcOGqVIllx0kui4uBZ2zZ8/K39/fpWOHTljm0vFQuMMze7u7BADAdVbcv99Op5I1a9ZcU2EAAADXS4kPv/z666/av3+/JKlp06aqUaOGy4oCAABwBadXXWVlZWnYsGGqVauW7rjjDt1xxx2qXbu2hg8frvPnz5dGjaUiPj5ezZs3V9u2bd1dCgAAKCVOB53x48dr3bp1+uKLL5Senq709HQtXbpU69at0xNPPFEaNZaKuLg4JSUlaevWre4uBQAAlBKnT10tWrRIn376qbp06WJv69Wrl/z8/HT//ffrX//6lyvrAwAAKDGnj+icP3++0KXdNWvWLFenrgAAgPU5HXQ6dOigKVOm6LfffrO3XbhwQdOmTePTigEAQJni9Kmr1157TT169FDdunUVHh4uSdq1a5d8fX21YsUKlxcIAABQUk4HnZYtWyo5OVnvv/++9u3bJ0kaNGiQBg8eLD8/P5cXCAAAUFIl+hydypUra8SIEa6u5briXlcAAFif09fozJgxQ2+//XaB9rffflsvvPCCS4q6HlheDgCA9TkddN58803dfPPNBdpbtGihuXPnuqQoAAAAV3A66KSmpqpWrVoF2mvUqKHjx4+7pCgAAABXcDro1KtXTxs3bizQvnHjRtWuXdslRQEAALiC0xcjjxgxQuPGjdPFixfVrVs3SVJCQoKefvrpcnULCAAAYH1OB52nnnpKp06d0iOPPKKcnBxJkq+vr/72t79p4sSJLi8QAACgpJwOOjabTS+88IImTZqkvXv3ys/PT02aNJGPj09p1FdqWF4OAID12Ywxxt1FuFNGRoYCAgJ09uxZ+fv7u3Ts0AnLXDoeCnd4Zm93lwAAuM6K+/fb6YuRAQAAyguCDgAAsCyCDgAAsCyCDgAAsKwS3dQTKEvK40XfXEANANcHR3QAAIBlVdigEx8fr+bNm6tt27buLgUAAJSSCht04uLilJSUpK1bt7q7FAAAUEoqbNABAADWR9ABAACWRdABAACWRdABAACWRdABAACWRdABAACWRdABAACWRdABAACWRdABAACWVWGDDreAAADA+ips0OEWEAAAWF8ldxcAVEShE5a5u4QSOTyzt7tLAACnVNgjOgAAwPoIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIqbNDhpp4AAFhfhQ063NQTAADrq7BBBwAAWB9BBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWBZBBwAAWFYldxfgLvHx8YqPj1deXp67SwHKjdAJy9xdgtMOz+zt7hIAuFGFPaITFxenpKQkbd261d2lAACAUlJhgw4AALA+gg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsgg4AALAsSwSd/v37q1q1arrvvvvcXQoAAChDLBF0HnvsMb3zzjvuLgMAAJQxldxdgCt06dJFa9eudXcZAACUqtAJy9xdgtMOz+zt1v27/YjO+vXr1adPH9WuXVs2m01Lliwp0Cc+Pl6hoaHy9fVV+/bttWXLlutfKAAAKHfcHnSysrIUHh6u+Pj4Qrd//PHHGj9+vKZMmaIdO3YoPDxcPXr0UFpa2nWuFAAAlDduP3XVs2dP9ezZs8jtr7zyikaMGKGhQ4dKkubOnatly5bp7bff1oQJE5zeX3Z2trKzs+2PMzIynC8aAACUC24/onMlOTk52r59u6Kjo+1tHh4eio6O1qZNm0o05owZMxQQEGD/qlevnqvKBQAAZUyZDjonT55UXl6egoODHdqDg4OVmppqfxwdHa0BAwboyy+/VN26da8YgiZOnKizZ8/av44ePVpq9QMAAPdy+6krV/j666+L3dfHx0c+Pj6lWA0AACgryvQRnaCgIHl6eurEiRMO7SdOnFBISIibqgIAAOVFmQ463t7eatOmjRISEuxt+fn5SkhIUIcOHdxYGQAAKA/cfuoqMzNTBw8etD9OSUlRYmKiqlevrvr162v8+PGKiYlRZGSk2rVrp9mzZysrK8u+Cquk4uPjFR8fr7y8vGt9CgBQ4fFBdiir3B50tm3bpq5du9ofjx8/XpIUExOjBQsWaODAgfr11181efJkpaamKiIiQsuXLy9wgbKz4uLiFBcXp4yMDAUEBFzTWAAAoGxye9Dp0qWLjDFX7DNmzBiNGTPmOlUEAACsokxfowMAAHAtCDoAAMCyCDoAAMCy3H6Njruw6gpAWVUeVzABZVWFPaITFxenpKQkbd261d2lAACAUlJhgw4AALA+gg4AALAsgg4AALAsgg4AALAsVl2x6gqwNFYwoSi8NiqGCntEh1VXAABYX4UNOgAAwPoIOgAAwLIIOgAAwLIIOgAAwLIIOgAAwLIqbNCJj49X8+bN1bZtW3eXAgAASkmFDTosLwcAwPoqbNABAADWR9ABAACWRdABAACWRdABAACWRdABAACWRdABAACWRdABAACWVcndBbhLfHy84uPjlZubK0nKyMhw+T7ys8+7fEwAAMqT0vj7+sdxjTFX7GczV+thcT///LPq1avn7jIAAEAJHD16VHXr1i1ye4UPOvn5+frll19UtWpV2Wy2Eo+TkZGhevXq6ejRo/L393dhhbgcc339MNfXD3N9/TDX109pzrUxRufOnVPt2rXl4VH0lTgV9tTVJR4eHldMgs7y9/fnf5zrhLm+fpjr64e5vn6Y6+untOY6ICDgqn24GBkAAFgWQQcAAFgWQcdFfHx8NGXKFPn4+Li7FMtjrq8f5vr6Ya6vH+b6+ikLc13hL0YGAADWxREdAABgWQQdAABgWQQdAABgWQQdAABgWQQdF4iPj1doaKh8fX3Vvn17bdmyxd0llXszZsxQ27ZtVbVqVdWsWVP9+vXT/v37Hfr89ttviouL04033qgqVaro3nvv1YkTJ9xUsXXMnDlTNptN48aNs7cx165z7Ngx/fWvf9WNN94oPz8/tWrVStu2bbNvN8Zo8uTJqlWrlvz8/BQdHa3k5GQ3Vlw+5eXladKkSWrYsKH8/PwUFham5557zuG+SMx1yaxfv159+vRR7dq1ZbPZtGTJEoftxZnX06dPa/DgwfL391dgYKCGDx+uzMzM0inY4Jp89NFHxtvb27z99tvmhx9+MCNGjDCBgYHmxIkT7i6tXOvRo4eZP3++2bNnj0lMTDS9evUy9evXN5mZmfY+o0aNMvXq1TMJCQlm27Zt5rbbbjNRUVFurLr827JliwkNDTWtW7c2jz32mL2duXaN06dPmwYNGpjY2FizefNm8+OPP5oVK1aYgwcP2vvMnDnTBAQEmCVLlphdu3aZu+++2zRs2NBcuHDBjZWXP88//7y58cYbzf/+9z+TkpJiFi5caKpUqWJee+01ex/mumS+/PJL88wzz5jPPvvMSDKLFy922F6ceb3zzjtNeHi4+e6778yGDRtM48aNzaBBg0qlXoLONWrXrp2Ji4uzP87LyzO1a9c2M2bMcGNV1pOWlmYkmXXr1hljjElPTzdeXl5m4cKF9j579+41ksymTZvcVWa5du7cOdOkSROzatUq07lzZ3vQYa5d529/+5u5/fbbi9yen59vQkJCzKxZs+xt6enpxsfHx3z44YfXo0TL6N27txk2bJhD2z333GMGDx5sjGGuXeXyoFOceU1KSjKSzNatW+19vvrqK2Oz2cyxY8dcXiOnrq5BTk6Otm/frujoaHubh4eHoqOjtWnTJjdWZj1nz56VJFWvXl2StH37dl28eNFh7m+++WbVr1+fuS+huLg49e7d22FOJebalT7//HNFRkZqwIABqlmzpm655Ra99dZb9u0pKSlKTU11mOuAgAC1b9+euXZSVFSUEhISdODAAUnSrl279M0336hnz56SmOvSUpx53bRpkwIDAxUZGWnvEx0dLQ8PD23evNnlNVX4m3pei5MnTyovL0/BwcEO7cHBwdq3b5+bqrKe/Px8jRs3Th07dlTLli0lSampqfL29lZgYKBD3+DgYKWmprqhyvLto48+0o4dO7R169YC25hr1/nxxx/1r3/9S+PHj9ff//53bd26VWPHjpW3t7diYmLs81nYewpz7ZwJEyYoIyNDN998szw9PZWXl6fnn39egwcPliTmupQUZ15TU1NVs2ZNh+2VKlVS9erVS2XuCToo8+Li4rRnzx5988037i7Fko4eParHHntMq1atkq+vr7vLsbT8/HxFRkbqH//4hyTplltu0Z49ezR37lzFxMS4uTpr+eSTT/T+++/rgw8+UIsWLZSYmKhx48apdu3azHUFw6mraxAUFCRPT88Cq09OnDihkJAQN1VlLWPGjNH//vc/rVmzRnXr1rW3h4SEKCcnR+np6Q79mXvnbd++XWlpabr11ltVqVIlVapUSevWrdOcOXNUqVIlBQcHM9cuUqtWLTVv3tyhrVmzZjpy5Igk2eeT95Rr99RTT2nChAl64IEH1KpVKw0ZMkSPP/64ZsyYIYm5Li3FmdeQkBClpaU5bM/NzdXp06dLZe4JOtfA29tbbdq0UUJCgr0tPz9fCQkJ6tChgxsrK/+MMRozZowWL16s1atXq2HDhg7b27RpIy8vL4e5379/v44cOcLcO6l79+76/vvvlZiYaP+KjIzU4MGD7d8z167RsWPHAh+TcODAATVo0ECS1LBhQ4WEhDjMdUZGhjZv3sxcO+n8+fPy8HD8E+fp6an8/HxJzHVpKc68dujQQenp6dq+fbu9z+rVq5Wfn6/27du7viiXX95cwXz00UfGx8fHLFiwwCQlJZmHH37YBAYGmtTUVHeXVq6NHj3aBAQEmLVr15rjx4/bv86fP2/vM2rUKFO/fn2zevVqs23bNtOhQwfToUMHN1ZtHX9cdWUMc+0qW7ZsMZUqVTLPP/+8SU5ONu+//76pXLmyee+99+x9Zs6caQIDA83SpUvN7t27Td++fVnyXAIxMTGmTp069uXln332mQkKCjJPP/20vQ9zXTLnzp0zO3fuNDt37jSSzCuvvGJ27txpfvrpJ2NM8eb1zjvvNLfccovZvHmz+eabb0yTJk1YXl6Wvf7666Z+/frG29vbtGvXznz33XfuLqnck1To1/z58+19Lly4YB555BFTrVo1U7lyZdO/f39z/Phx9xVtIZcHHebadb744gvTsmVL4+PjY26++WYzb948h+35+flm0qRJJjg42Pj4+Jju3bub/fv3u6na8isjI8M89thjpn79+sbX19c0atTIPPPMMyY7O9veh7kumTVr1hT6/hwTE2OMKd68njp1ygwaNMhUqVLF+Pv7m6FDh5pz586VSr02Y/7wMZEAAAAWwjU6AADAsgg6AADAsgg6AADAsgg6AADAsgg6AADAsgg6AADAsgg6AADAsgg6gEV16dJF48aNc3cZZc6SJUvUuHFjeXp6Mj9ABUDQAVDurV27VjabrcCNRwszcuRI3XfffTp69Kiee+650i8OgFtVcncBAHC9ZGZmKi0tTT169FDt2rUL7ZOXlyebzVbghpAAyif+TwYsICsrSw8++KCqVKmiWrVq6eWXXy7Q58yZM3rwwQdVrVo1Va5cWT179lRycrJDn40bN6pLly6qXLmyqlWrph49eujMmTOSpNDQUM2ePduhf0REhKZOnWp/bLPZ9Oabb+quu+5S5cqV1axZM23atEkHDx5Uly5ddMMNNygqKkqHDh1yGGfp0qW69dZb5evrq0aNGmnatGnKzc11GPff//63+vfvr8qVK6tJkyb6/PPPJUmHDx9W165dJUnVqlWTzWZTbGxsgee/du1aVa1aVZLUrVs32Ww2rV27VgsWLFBgYKA+//xzNW/eXD4+Pjpy5Iiys7P15JNPqk6dOrrhhhvUvn17rV271mHMBQsWqH79+qpcubL69++vl19+WYGBgfbtsbGx6tevn8PPjBs3Tl26dLE/zs/P14wZM9SwYUP5+fkpPDxcn376qUPdNptNCQkJioyMVOXKlRUVFVXgLuhffPGF2rZtK19fXwUFBal///6SpGeffVYtW7YsMB8RERGaNGlSgXbAckrlDloArqvRo0eb+vXrm6+//trs3r3b3HXXXaZq1aoON+a8++67TbNmzcz69etNYmKi6dGjh2ncuLHJyckxxhizc+dO4+PjY0aPHm0SExPNnj17zOuvv25+/fVXY4wxDRo0MK+++qrDfsPDw82UKVPsjyWZOnXqmI8//tjs37/f9OvXz4SGhppu3bqZ5cuXm6SkJHPbbbeZO++80/4z69evN/7+/mbBggXm0KFDZuXKlSY0NNRMnTrVYdy6deuaDz74wCQnJ5uxY8eaKlWqmFOnTpnc3FyzaNEiI8ns37/fHD9+3KSnpxeYo+zsbLN//34jySxatMgcP37cZGdnm/nz5xsvLy8TFRVlNm7caPbt22eysrLMQw89ZKKiosz69evNwYMHzaxZs4yPj485cOCAMcaY7777znh4eJgXXnjB7N+/37z22msmMDDQBAQE2PcZExNj+vbt61DHY489Zjp37mx/PH36dHPzzTeb5cuXm0OHDpn58+cbHx8fs3btWmPM/7uBYvv27c3atWvNDz/8YDp16mSioqLsY/zvf/8znp6eZvLkySYpKckkJiaaf/zjH8YYY44ePWo8PDzMli1b7P137NhhbDabOXToUIF5AqyGoAOUc+fOnTPe3t7mk08+sbedOnXK+Pn52YPOgQMHjCSzceNGe5+TJ08aPz8/+88NGjTIdOzYscj9FDfo/N///Z/98aZNm4wk85///Mfe9uGHHxpfX1/74+7du9v/KF/y7rvvmlq1ahU5bmZmppFkvvrqK2PM/wsDZ86cKbJ+Y4w5c+aMkWTWrFljb5s/f76RZBITE+1tP/30k/H09DTHjh1z+Pnu3bubiRMnGmN+n69evXo5bB84cKBTQee3334zlStXNt9++61Dn+HDh5tBgwY5PLevv/7avn3ZsmVGkrlw4YIxxpgOHTqYwYMHF/m8e/bsaUaPHm1//Oijj5ouXboU2R+wEk5dAeXcoUOHlJOTo/bt29vbqlevrqZNm9of7927V5UqVXLoc+ONN6pp06bau3evJCkxMVHdu3e/5npat25t/z44OFiS1KpVK4e23377TRkZGZKkXbt26dlnn1WVKlXsXyNGjNDx48d1/vz5Qse94YYb5O/vr7S0tGuuV5K8vb0dxv/++++Vl5enm266yaGudevW2U+77d2712E+JalDhw5O7ffgwYM6f/68/vSnPzns55133ilweu+P9dWqVUuS7M//ar+7ESNG6MMPP9Rvv/2mnJwcffDBBxo2bJhTtQLlFRcjA5Ak+fn5XXG7h4eHjDEObRcvXizQz8vLy/69zWYrsi0/P1/S7xcIT5s2Tffcc0+BsXx9fQsd99I4l8a4Vn5+fva6LtXk6emp7du3y9PT06FvlSpVij3u1eYsMzNTkrRs2TLVqVPHoZ+Pj4/D4yvN4dV+d3369JGPj48WL14sb29vXbx4Uffdd1+xnwdQnhF0gHIuLCxMXl5e2rx5s+rXry/p9wuPDxw4oM6dO0uSmjVrptzcXG3evFlRUVGSpFOnTmn//v1q3ry5pN+PGCQkJGjatGmF7qdGjRo6fvy4/XFGRoZSUlKuuf5bb71V+/fvV+PGjUs8hre3t6TfV0y5wi233KK8vDylpaWpU6dOhfZp1qyZNm/e7ND23XffOTyuUaOG9uzZ49CWmJhoDy1/vPj50u+qJC797oYOHVro9kqVKikmJkbz58+Xt7e3HnjggauGI8AqCDpAOVelShUNHz5cTz31lG688UbVrFlTzzzzjMPy6CZNmqhv374aMWKE3nzzTVWtWlUTJkxQnTp11LdvX0nSxIkT1apVKz3yyCMaNWqUvL29tWbNGg0YMEBBQUHq1q2bFixYoD59+igwMFCTJ08ucLSjJCZPnqy77rpL9evX13333ScPDw/t2rVLe/bs0fTp04s1RoMGDWSz2fS///1PvXr1kp+fn1NHXi530003afDgwXrwwQf18ssv65ZbbtGvv/6qhIQEtW7dWr1799bYsWPVsWNHvfTSS+rbt69WrFih5cuXO4zTrVs3zZo1S++88446dOig9957T3v27NEtt9wiSapataqefPJJPf7448rPz9ftt9+us2fPauPGjfL391dMTEyx6p0yZYq6d++usLAwPfDAA8rNzdWXX36pv/3tb/Y+Dz30kJo1aybp99V1QIXh7ouEAFy7c+fOmb/+9a+mcuXKJjg42Lz44oumc+fODquuTp8+bYYMGWICAgKMn5+f6dGjh30F0SVr1641UVFRxsfHxwQGBpoePXrYL/A9e/asGThwoPH39zf16tUzCxYsKPRi5MWLF9sfp6SkGElm586d9rbCLhxevny5iYqKMn5+fsbf39+0a9fOzJs3r8hxjTEmICDAzJ8/3/742WefNSEhIcZms5mYmJhC56moi5H/eAHxJTk5OWby5MkmNDTUeHl5mVq1apn+/fub3bt32/v85z//MXXr1jV+fn6mT58+5qWXXiow1uTJk01wcLAJCAgwjz/+uBkzZozDqqv8/Hwze/Zs07RpU+Pl5WVq1KhhevToYdatW1fkfO3cudNIMikpKfa2RYsWmYiICOPt7W2CgoLMPffcU+A5derUybRo0aLQuQGsymbMZSeQAQAlsmDBAo0bN65Yn9B8vRlj1KRJEz3yyCMaP368u8sBrhtOXQGAxf3666/66KOPlJqaWuR1PIBVEXQAwOJq1qypoKAgzZs3T9WqVXN3OcB1xakrAABgWXxgIAAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsCyCDgAAsKz/Dwiukg390HgbAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "plot_doc_frequencies_hist(fig, ax, corpus_small)\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows us that the large majority of token types has a low document frequency, i.e. they occur only in a few documents. There are only very few token types that occur in almost every document. Words like \"the\" or \"a\" are usually among these.\n",
    "\n",
    "Another common type of plot is a *rank-frequency distribution* plot for token frequencies. This means the tokens are ordered in descending order from the most frequent token to the least frequent token. This forms the x-axis. On the y-axis the frequency of the respective tokens is plotted. Rank and frequency of tokens in text corpora usually have an inverse relationship, i.e. the second most frequent token occurs only half as often as the most frequent token, the token on rank 100 only has 1/100 of the frequency of the most frequent token, etc. This is a power law distribution which appears as nearly straight line when plotted on a log-log scale (i.e. a log scale on both axes).\n",
    "\n",
    "We can also observe that in our small corpus using the [plot_ranked_vocab_counts](api.rst#tmtoolkit.corpus.visualize.plot_ranked_vocab_counts) function, which by default uses a log-log scale:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.657514Z",
     "iopub.status.busy": "2023-05-03T13:22:48.657415Z",
     "iopub.status.idle": "2023-05-03T13:22:48.762879Z",
     "shell.execute_reply": "2023-05-03T13:22:48.762454Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHHCAYAAABTMjf2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACU50lEQVR4nOzdd3hUZfbA8e+d3tI7EEJHOgiIoAgoilixIuqi2BUra0NdBV1lrT9dXRXXVXRFwYJlsQAiYAEVBKRJR3p6mUzJ1Pv745JASAIJJJlMcj7PMw/cd+7MPZNMMidvOa+iqqqKEEIIIUQLpIt0AEIIIYQQkSKJkBBCCCFaLEmEhBBCCNFiSSIkhBBCiBZLEiEhhBBCtFiSCAkhhBCixZJESAghhBAtliRCQgghhGixJBESQgghRIsliZAQtTR8+HCGDx/eKNfasmULZ511FnFxcSiKwmeffdYo122KFEXh9ttvr9fnbNeuHddee229PqcQTd3w4cPp2bNnpMNociQRauHWrl3LpZdeSlZWFhaLhdatW3PmmWfy8ssvN9g133//fV588cUq7fv27WPKlCmsXr26wa4dCR6PhylTprB48eJaP+aaa65h7dq1PPnkk/z3v/9lwIABDRegiLjm+t6PpGP5uRMtkyRCLdjSpUsZMGAAv//+OzfeeCOvvPIKN9xwAzqdjpdeeqnBrnukRGjq1KnN7sPA4/EwderUWv9C9nq9LFu2jOuvv57bb7+dq6++mjZt2jRskCKimut7P5Lq+nMnWi5DpAMQkfPkk08SFxfH8uXLiY+Pr3Rfbm5uZIJqAG63G7vdHukwai0vLw+gyvfkeETb1yCalJWVYTKZ0Oma3t+V8n2PPk35/dRcyVe6Bdu2bRs9evSo9gM3NTW1Stt7773HSSedhM1mIyEhgdNOO4358+dX3P/5559z7rnn0qpVK8xmMx07duSJJ54gFApVnDN8+HC+/PJLdu7ciaIoKIpCu3btWLx4MQMHDgRgwoQJFffNmDGj4rG//PILZ599NnFxcdhsNoYNG8ZPP/1UKcYpU6agKAobNmzgyiuvJCEhgVNPPbXGr8GMGTNQFIXvv/+em2++maSkJGJjYxk/fjxFRUVH/Rrm5uZy/fXXk5aWhsVioU+fPrzzzjsV9//555+kpKQAMHXq1IrXNWXKlGqfb8qUKWRlZQFw3333VXx9yq1atYrRo0cTGxuLw+HgjDPO4Oeff672NS1ZsoTbbruN1NTUGnuUcnJyMBgMTJ06tcp9mzZtQlEUXnnllYq27du3c9lll5GYmIjNZuPkk0/myy+/rPLYsrIypkyZQpcuXbBYLGRkZHDxxRezbdu2inOee+45hgwZQlJSElarlf79+/Pxxx9XGyfAzJkz6dq1KxaLhf79+/P9999Xuv/aa6+t9LUqV/6eOJLCwkLuvfdeevXqhcPhIDY2ltGjR/P7779XOm/x4sUoisKsWbN45JFHaN26NTabjdWrV6MoCv/3f/9X5bmXLl2Koih88MEH1V67Md777dq147zzzmPx4sUMGDAAq9VKr169KnpL5syZQ69evSq+tqtWrTri16tccXEx99xzD+3atcNsNtOmTRvGjx9Pfn5+xTlH+xk59Ot6eO/Nn3/+WeVrce211+JwONi7dy9jxozB4XCQkpLCvffeW/G7pq4/dytWrEBRlCpxAcybNw9FUZg7dy4ApaWl3H333RWvOTU1lTPPPJOVK1fW6mtW3es+/P3kdDrr/J788MMPefLJJ2nTpg0Wi4UzzjiDrVu3HjWG+fPnY7PZGDduHMFgsM6voTmQHqEWLCsri2XLlrFu3bqjTqCbOnUqU6ZMYciQITz++OOYTCZ++eUXvvvuO8466yxA+wB2OBxMmjQJh8PBd999x6OPPorT6eTZZ58F4OGHH6akpIQ9e/ZUfGg4HA66devG448/zqOPPspNN93E0KFDARgyZAgA3333HaNHj6Z///489thj6HQ63n77bU4//XR++OEHTjrppErxXnbZZXTu3JmnnnoKVVWP+rW4/fbbiY+PZ8qUKWzatInXXnuNnTt3VvySqY7X62X48OFs3bqV22+/nfbt2/PRRx9x7bXXUlxczF133UVKSgqvvfYat956KxdddBEXX3wxAL179672OS+++GLi4+O55557GDduHOeccw4OhwOA9evXM3ToUGJjY7n//vsxGo1Mnz6d4cOHs2TJEgYNGlTpuW677TZSUlJ49NFHcbvd1V4vLS2NYcOG8eGHH/LYY49Vum/27Nno9Xouu+wyQEuahgwZgsfj4c477yQpKYl33nmHCy64gI8//piLLroIgFAoxHnnncfChQu54ooruOuuuygtLWXBggWsW7eOjh07AvDSSy9xwQUXcNVVV+H3+5k1axaXXXYZc+fO5dxzz60Uy5IlS5g9ezZ33nknZrOZV199lbPPPptff/21XiZ/bt++nc8++4zLLruM9u3bk5OTw/Tp0xk2bBgbNmygVatWlc5/4oknMJlM3Hvvvfh8Pk444QROOeUUZs6cyT333FPp3JkzZxITE8OFF15Y7bUb672/detWrrzySm6++WauvvpqnnvuOc4//3xef/11HnroIW677TYApk2bxuWXX86mTZuO2CvhcrkYOnQof/zxB9dddx0nnngi+fn5fPHFF+zZs4fk5ORa/Ywci1AoxKhRoxg0aBDPPfcc3377Lc8//zwdO3bk1ltvrfPP3YABA+jQoQMffvgh11xzTaX7Zs+eTUJCAqNGjQLglltu4eOPP+b222+ne/fuFBQU8OOPP/LHH39w4oknHtPrOfz9ZDKZ2LBhQ53ek//4xz/Q6XTce++9lJSU8Mwzz3DVVVfxyy+/1HjduXPncumllzJ27Fjeeust9Hr9McUf9VTRYs2fP1/V6/WqXq9XBw8erN5///3qvHnzVL/fX+m8LVu2qDqdTr3ooovUUChU6b5wOFzxf4/HU+UaN998s2qz2dSysrKKtnPPPVfNysqqcu7y5ctVQH377berXKNz587qqFGjqlyvffv26plnnlnR9thjj6mAOm7cuFp9Dd5++20VUPv371/pdT/zzDMqoH7++ecVbcOGDVOHDRtWcfziiy+qgPree+9VtPn9fnXw4MGqw+FQnU6nqqqqmpeXpwLqY489VquYduzYoQLqs88+W6l9zJgxqslkUrdt21bRtm/fPjUmJkY97bTTqrymU089VQ0Gg0e93vTp01VAXbt2baX27t27q6effnrF8d13360C6g8//FDRVlpaqrZv315t165dxXvjrbfeUgH1hRdeqHKtI71f/H6/2rNnz0rXVFVVBVRAXbFiRUXbzp07VYvFol500UUVbddcc02176vy98ShsrKy1GuuuabiuKysrMp7e8eOHarZbFYff/zxirZFixapgNqhQ4cq8Zd/Hf/4449Kryk5ObnStarT0O/9rKwsFVCXLl1a0TZv3jwVUK1Wq7pz584qr2PRokVHjPnRRx9VAXXOnDlV7iuPtbY/I+Vf18OvWf6zcOjX5ZprrlGBSt8XVVXVfv36qf379684ruvP3eTJk1Wj0agWFhZWtPl8PjU+Pl697rrrKtri4uLUiRMn1uo5j+ZI76e6vie7deum+ny+ivaXXnqpys/1sGHD1B49eqiqqqqffPKJajQa1RtvvLHKdVoaGRprwc4880yWLVvGBRdcwO+//84zzzzDqFGjaN26NV988UXFeZ999hnhcJhHH320yl+Ih/aWWK3Wiv+XlpaSn5/P0KFD8Xg8bNy48ZjjXL16NVu2bOHKK6+koKCA/Px88vPzcbvdnHHGGXz//feEw+FKj7nlllvqdI2bbroJo9FYcXzrrbdiMBj46quvanzMV199RXp6OuPGjatoMxqN3HnnnbhcLpYsWVKnGI4kFAoxf/58xowZQ4cOHSraMzIyuPLKK/nxxx9xOp2VHnPjjTfW6i+8iy++GIPBwOzZsyva1q1bx4YNGxg7dmxF21dffcVJJ51UaajR4XBw00038eeff7JhwwYAPvnkE5KTk7njjjuqXKum90tRURElJSUMHTq02iGGwYMH079//4rjtm3bcuGFFzJv3rxKQ6/Hymw2V7y3Q6EQBQUFOBwOunbtWm0811xzTaX4AS6//HIsFgszZ86saJs3bx75+flcffXVxxRXfb73u3fvzuDBgyuOy3sQTz/9dNq2bVulffv27UeM7ZNPPqFPnz4VPYGHKv8+N+TPyOGvc+jQoUeN+UjGjh1LIBBgzpw5FW3z58+nuLi40s9BfHw8v/zyC/v27Tvmax2uuvdTXd+TEyZMwGQyVRyX9yxW9zX54IMPGDt2LDfffDPTp09v8fORWvarFwwcOJA5c+ZQVFTEr7/+yuTJkyktLeXSSy+t+GDbtm0bOp2O7t27H/G51q9fz0UXXURcXByxsbGkpKRUfACUlJQcc4xbtmwBtF8WKSkplW5vvvkmPp+vyvO3b9++Ttfo3LlzpWOHw0FGRgZ//vlnjY/ZuXMnnTt3rvJLpFu3bhX315e8vDw8Hg9du3atcl+3bt0Ih8Ps3r27UnttvwbJycmcccYZfPjhhxVts2fPxmAwVAwpgPZ6arp++f2gvV+6du2KwXDkkfe5c+dy8sknY7FYSExMrBjOqO69cvj3B6BLly54PJ6KyeXHIxwO83//93907twZs9lMcnIyKSkprFmzptp4qvvaxsfHc/755/P+++9XtM2cOZPWrVtz+umnH1Nc9fnePzTZAYiLiwMgMzOz2vajzZHbtm3bUYclG+pnxGKxVMwBKpeQkFCreX016dOnDyeccEKlPwhmz55NcnJype/fM888w7p168jMzOSkk05iypQpx5WAQfXfs7q+Jw///iYkJABVv487duzg6quv5pJLLuHll18+6vy5lkDmCAkATCYTAwcOZODAgXTp0oUJEybw0UcfVZk3UpPi4mKGDRtGbGwsjz/+OB07dsRisbBy5UoeeOCBKn+11kX5Y5999ln69u1b7Tnl82jKHf7XVUtUl6/BFVdcwYQJE1i9ejV9+/blww8/5IwzziA5OblBYvvhhx+44IILOO2003j11VfJyMjAaDTy9ttvV0ok6qKmX+i16TF66qmn+Nvf/sZ1113HE088QWJiIjqdjrvvvrva925NX9vx48fz0UcfsXTpUnr16sUXX3zBbbfddsx/cdfne7+m3sGa2tVazK2rL3X93jXUXJaxY8fy5JNPkp+fT0xMDF988QXjxo2rlNRffvnlDB06lE8//ZT58+fz7LPP8vTTTzNnzhxGjx59TNet7ntW1/dkbb+PGRkZZGRk8NVXX7FixQqpUYYkQqIa5T8Y+/fvB6Bjx46Ew2E2bNhQ4y/jxYsXU1BQwJw5czjttNMq2nfs2FHl3Jp+6dXUXj65NjY2lpEjR9b6ddTFli1bGDFiRMWxy+Vi//79nHPOOTU+JisrizVr1hAOhyt90JUPA5av/qqPv7hSUlKw2Wxs2rSpyn0bN25Ep9NV+cu+LsaMGcPNN99c8dfw5s2bmTx5cqVzsrKyarx++f2gfb9++eUXAoFApeHGQ33yySdYLBbmzZuH2WyuaH/77berPb+8Z+RQmzdvxmazVfQMJCQkUFxcXOW82vQ6fPzxx4wYMYL//Oc/ldqLi4vrlAyeffbZpKSkMHPmTAYNGoTH4+Evf/nLUR8Xyff+serYsSPr1q074jm1/Rkp7704/Pt3PL2qx/JzN3bsWKZOnconn3xCWloaTqeTK664osp5GRkZ3Hbbbdx2223k5uZy4okn8uSTTx5zIlSd+npPHs5isTB37lxOP/10zj77bJYsWUKPHj2ON9yoJkNjLdiiRYuq/auvfF5M+TDImDFj0Ol0PP7441X+Eil/fPlfI4c+n9/v59VXX63y/Ha7vdqu3fJ6J4f/Muzfvz8dO3bkueeew+VyVXlcfQyNvPHGGwQCgYrj1157jWAweMRfbOeccw7Z2dmVutKDwSAvv/wyDoeDYcOGAWCz2YCqr6su9Ho9Z511Fp9//nml4bqcnBzef/99Tj31VGJjY4/5+ePj4xk1ahQffvghs2bNwmQyMWbMmErnnHPOOfz6668sW7asos3tdvPGG2/Qrl27iqHTSy65hPz8/ErL7ssd+n5RFKXSX/x//vlnjVuJLFu2rNK8iN27d/P5559z1llnVbz3OnbsSElJCWvWrKk4b//+/Xz66adHff16vb7Kz8JHH33E3r17j/rYQxkMBsaNG8eHH37IjBkz6NWrV40rlQ4Vyff+sbrkkkv4/fffq/36ln8ta/szkpWVhV6vr1ISobrfH7V1LD933bp1o1evXsyePZvZs2eTkZFR6Q+7UChU5XdXamoqrVq1wufzVbTl5+ezceNGPB7PMcdfX+/J6sTFxTFv3ryKpf+HlrVoiaRHqAW744478Hg8XHTRRZxwwgn4/X6WLl3K7NmzadeuHRMmTACgU6dOPPzwwzzxxBMMHTqUiy++GLPZzPLly2nVqhXTpk1jyJAhJCQkcM0113DnnXeiKAr//e9/q020+vfvz+zZs5k0aRIDBw7E4XBw/vnn07FjR+Lj43n99deJiYnBbrczaNAg2rdvz5tvvsno0aPp0aMHEyZMoHXr1uzdu5dFixYRGxvL//73v+P6Wvj9fs4444yKZcOvvvoqp556KhdccEGNj7npppuYPn061157Lb/99hvt2rXj448/5qeffuLFF18kJiYG0Lq9u3fvzuzZs+nSpQuJiYn07Nmzzsu+//73v7NgwQJOPfVUbrvtNgwGA9OnT8fn8/HMM88c1+sH7a/hq6++mldffZVRo0ZVqS/14IMP8sEHHzB69GjuvPNOEhMTeeedd9ixYweffPJJxV/848eP591332XSpEn8+uuvDB06FLfbzbfffsttt93GhRdeyLnnnssLL7zA2WefzZVXXklubi7/+te/6NSpU6VEplzPnj0ZNWpUpeXzQKX6R1dccQUPPPAAF110EXfeeScej4fXXnuNLl26HLXGy3nnncfjjz/OhAkTGDJkCGvXrmXmzJmVJqbX1vjx4/nnP//JokWLePrpp2v1mEi+94/Vfffdx8cff8xll13GddddR//+/SksLOSLL77g9ddfp0+fPrX+GYmLi+Oyyy6rmLPSsWNH5s6de1yFXY/1527s2LE8+uijWCwWrr/++ko9WaWlpbRp04ZLL72UPn364HA4+Pbbb1m+fDnPP/98xXmvvPIKU6dOZdGiRce8P2F9vierk5ycXPH7ZOTIkfz444+0bt26Xp476kRquZqIvK+//lq97rrr1BNOOEF1OByqyWRSO3XqpN5xxx1qTk5OlfPfeusttV+/fqrZbFYTEhLUYcOGqQsWLKi4/6efflJPPvlk1Wq1qq1atapYjs9hy2JdLpd65ZVXqvHx8SpQacnz559/rnbv3l01GAxVls2uWrVKvfjii9WkpCTVbDarWVlZ6uWXX64uXLiw4pzyJcR5eXm1+hqULzVfsmSJetNNN6kJCQmqw+FQr7rqKrWgoKDSuYcvn1dVVc3JyVEnTJigJicnqyaTSe3Vq1eVJdCqqqpLly5V+/fvr5pMpqMu6a1p+byqqurKlSvVUaNGqQ6HQ7XZbOqIESMqLYk+9DUtX768Vl+Dck6nU7VarVWWOx9q27Zt6qWXXqrGx8erFotFPemkk9S5c+dWOc/j8agPP/yw2r59e9VoNKrp6enqpZdeWmnp/3/+8x+1c+fOqtlsVk844QT17bffrnapO6BOnDhRfe+99yrO79evX7XLu+fPn6/27NlTNZlMateuXdX33nuv1svn//rXv6oZGRmq1WpVTznlFHXZsmVVvuflS5U/+uijI34te/Tooep0OnXPnj1HPO9QDfnez8rKUs8999wq7eVf20Md6f13uIKCAvX2229XW7durZpMJrVNmzbqNddco+bn51ecU9ufkby8PPWSSy5RbTabmpCQoN58883qunXrql0+b7fbqzy+uu9zXX7uym3ZsqWiZMOPP/5Y6T6fz6fed999ap8+fdSYmBjVbrerffr0UV999dVqYzlaCYIjvZ+O9z1ZXemBQ5fPl9u6dauakZGhduvWrda/N5sbRVUbcUacEE3MjBkzmDBhAsuXL5dJg6Le9OvXj8TERBYuXBjpUIQQRyFzhIQQoh6tWLGC1atXM378+EiHIoSoBZkjJIQQ9WDdunX89ttvPP/882RkZFQqwieEaLqkR0gIIerBxx9/zIQJEwgEAnzwwQdYLJZIhySEqAWZIySEEEKIFkt6hIQQQgjRYkkiJIQQQogWSyZLH0U4HGbfvn3ExMTI5nRCCCFElFBVldLSUlq1anXE/f4kETqKffv2HdceTkIIIYSInN27d9OmTZsa75dE6CjKS8Dv3r37uPZyEkIIIUTjcTqdZGZmVnyO10QSoaMoHw6LjY2VREgIIYSIMkeb1iKTpYUQQgjRYkkiJIQQQogWSxIhIYQQQrRYUZMIvfbaa/Tu3btirs7gwYP5+uuvazx/xowZKIpS6SYl74UQQghxqKiZLN2mTRv+8Y9/0LlzZ1RV5Z133uHCCy9k1apV9OjRo9rHxMbGsmnTpopjqQMkhBBCiENFTSJ0/vnnVzp+8sknee211/j5559rTIQURSE9Pb0xwhNCCCFEFIqaobFDhUIhZs2ahdvtZvDgwTWe53K5yMrKIjMzkwsvvJD169c3YpRCCCGEaOqipkcIYO3atQwePJiysjIcDgeffvop3bt3r/bcrl278tZbb9G7d29KSkp47rnnGDJkCOvXrz9ihUmfz4fP56s4djqd9f46hBBCCNE0KKqqqpEOorb8fj+7du2ipKSEjz/+mDfffJMlS5bUmAwdKhAI0K1bN8aNG8cTTzxR43lTpkxh6tSpVdpLSkqkoKIQQggRJZxOJ3FxcUf9/I6qROhwI0eOpGPHjkyfPr1W51922WUYDAY++OCDGs+prkcoMzNTEiEhhBAiitQ2EYqqobHDhcPhSknLkYRCIdauXcs555xzxPPMZjNms7k+wquRLxCiwO0nEApj1OtIspswG/UNek0hhBBCVBU1idDkyZMZPXo0bdu2pbS0lPfff5/Fixczb948AMaPH0/r1q2ZNm0aAI8//jgnn3wynTp1ori4mGeffZadO3dyww03RPJlUODysT3PhdsXIhRW0esU7GY9HVIcJDkaNgETQgghRGVRkwjl5uYyfvx49u/fT1xcHL1792bevHmceeaZAOzatQud7uAiuKKiIm688Uays7NJSEigf//+LF26tFbziRqKLxBie56LEm+woi0YVinxBtme58JhNkjPkBBCCNGIonqOUGOo7Rhjbewr9rI5u5RgWPuSp3/xEbZdO9h+618xGA10SY+hVby1PsKuNRmmE0II0Ry1iDlC0SYQChM6kASZ8nLo9vj96L1e4lf8zPpnXiWQYm/UeGSYTgghREsXlQUVo5VRr0Ov07b58Kek8ceU5whabSQuX8qgS87AseS7Rovl0GG6YFhFpfIwnS8QarRYhBBCiEiRRKgRJdlN2M0Hh52yz7uEXz+aT2mX7pgKC0i89EJ4+GEIBo/wLPWjwO3H7as+2XH7tOEyIYQQormTRKgRmY3asFOc1YBBp6AA/o6d2fTZPMquvxFFVeGpp+CMMyAQaNBYDh2mO1worBIIhY/peX2BEPuKvewscLOv2Cs9S0IIIZo0mSPUyJIcZhxmQ9UJym++AWeNhBtugFNOAaOxQeMoH6YLVpMM6XUKRn3dc2SZcySEECLaSCIUAWajvvrVYZdfDiedBIfuhZaTA4mJ9Z4YlQ/THbqUv5zdrCfJbqrT80lpACGEENFIhsaamnbtwHAgP/X74fzz4bTTYOfOer1MdcN0Bp1CnNVAx1RHnZMWmXMkhBAiGkmPUFO2YQNs3gwlJdC3L8yYARdeWG9PX+Mw3TH03DTUnCMhhBCiIUmPUFPWty+sWgUDB0JxMYwZA3ffrfUU1ZPyYbqsJDut4q3HPHx1aGmAwx3rnCMhhBCiocmnU1PXvj38+CNMmqQdv/SSNpl6+/bIxnWYw0sDHOpY5hwJIYQQjUESoWhgMsHzz8MXX0BCAqxYATfeGOmoKqnvOUdCCCFEY5A5QtHk/PNh9Wq4+WZ45ZVIR1NFfc45EkIIIRqDJELRpm1b+Prrym1vvAEjRkDnzpGJ6RA1lgYQQgghmiAZGot2CxfCLbfAiSfCrFmRjkYIIYSIKpIIRbsTToChQ8HlgnHjtGEzrzfSUQkhhBBRQRKhaNe6tdYr9MgjoCjaMNmgQbBxY6QjE0IIIZo8SYSaA4MBnngC5s+H1FRYuxb694f33490ZEIIIUSTJolQczJypLaq7PTTweMBny/SEQkhhBBNmqwaa24yMrSeoS++0CpRlwsGD+5hJoQQQghAeoSaJ70eLrpImzMEUFAAPXrAW2+BWv1+YJHmC4TYV+xlZ4GbfcVefIHqN3AVQggh6pN0EbQE//qXtnnr9dfDd9/B66+DwxHpqCoUuHxsz3Ph9oUIhVX0OgW7WatUneQwVznfFwhJ0UYhhBD1QhKhluCRR8BohL/9DWbOhOXL4cMPoU+fSEeGLxBie56LEm+woi0YVinxBtme58JhNlRKcuqaNAkhhBBHIkNjLYFOB5Mnw+LF2nL7zZu1JfbTp0d8qKzA7cftq34YzO3Ten7KHZo0BcMqKpWTpkOH02SoTQghRG1IItSSnHqqtqrsnHO0FWW33AL//GdEQwqEwoTC1SdjobBKIBSuOK5t0lTg8rFmTzGbs0vZmuNic3Ypa/YUU+CSVXRCCCEqk0SopUlOhv/9D559Frp0gWuuiWg4Rr0OvU6p9j69TsGoP/gWrU3SVJdeIyGEEEISoZZIp4N774U1ayA+XmtTVW0z10YeKkuym7Cbq5/obDfrSbKbKo6PljSZDLo6DbUJIYQQkgi1ZOZDJhe/8YY2ZHbZZVBc3HghGLWJznFWAwadggIYdApxVgMdUx2VJkofLWlKtJnqNNQmhBBCyKoxoQmFtJVln3wCv/0Gs2fDSSc1yqWTHGYcZsNRl8SXJ03VrRorT5rKe42C1SRDhw+1CSGEEJIICc1tt8HAgTB2LOzYoU2sfvppuPvug4UZG5DZqKdVvPWo5x0taSrvNTp0OX658qE2qUMkhBCinKKqTbTUcBPhdDqJi4ujpKSE2NjYSIfT8IqL4YYbtJ4hgPPPhxkzIDExklHVSU21hjqlOQiHOWIdIkmShBCieajt57ckQkfR4hIh0CZMv/Ya3HOPtkfZ4sUwdGiko6qT6hIagDV7iqvtLUp2GMmIt7KrwFPtsJvddPShOyGEEE1HbT+/ZWhMVKUo2lDZ4MHw669RlwRB9UNt+4q9Na4oC4RU1u51wiF/FgTDKt5AmB15LvwhFV8gLNWshRCimYmamaOvvfYavXv3JjY2ltjYWAYPHszXX399xMd89NFHnHDCCVgsFnr16sVXX33VSNE2E/36wc03HzzeuFFbVZaXF7mYjkNNK8qMegWXL0hRNUvrTQaFzTludhd6pS6REEI0Q1GTCLVp04Z//OMf/Pbbb6xYsYLTTz+dCy+8kPXr11d7/tKlSxk3bhzXX389q1atYsyYMYwZM4Z169Y1cuTNhKrCtdfCxx9D377www+RjqjOaqpDpFMULUFSDz9fwesPUeDyEQxXXXYvdYmEECL6RfUcocTERJ599lmuv/76KveNHTsWt9vN3LlzK9pOPvlk+vbty+uvv17ra7TIOUI1WbMGLr8cNm3SijI+/ri2h5kuOvJpXyBU7Rwho17BbNBRWhZEOWSFnNmgwxsIsWG/k1ZxVhIPKe4IoACd0hxkJdkbI3whhBB1UNvP7+j4BDtMKBRi1qxZuN1uBg8eXO05y5YtY+TIkZXaRo0axbJly4743D6fD6fTWekmDujdG1asgL/8BcJhbVf7s8+GnJxIR1YrNRVvtJm0FWXxNmOl88OqilGvEGM2EGOpOp1O6hIJIUT0i6rJ0mvXrmXw4MGUlZXhcDj49NNP6d69e7XnZmdnk5aWVqktLS2N7OzsI15j2rRpTJ06td5ibnYcDnjnHRgxAiZOhAULtKGyJUu0vcuauCPVIVJQKi2tV1VItptQFAhWMxXo8C1AhBBCRJ+oSoS6du3K6tWrKSkp4eOPP+aaa65hyZIlNSZDx2Ly5MlMmjSp4tjpdJKZmVlvz98sKApMmACDBmlDZXFx0L59pKOqtZqKN9aUJLl8wSNWsxZCCBG9oioRMplMdOrUCYD+/fuzfPlyXnrpJaZPn17l3PT0dHIOG7LJyckhPT39iNcwm82YzbIkula6d9eW1zud2vYcAH4/FBbCUb7OTVV1SZLZqK/VFiBCCCGiT1RPcAiHw/h8vmrvGzx4MAsXLqzUtmDBghrnFIljZLNVTnomT9bmEs2fH7mYGkB5gpSVZKdVvFWSICGEaCaipkdo8uTJjB49mrZt21JaWsr777/P4sWLmTdvHgDjx4+ndevWTJs2DYC77rqLYcOG8fzzz3Puuecya9YsVqxYwRtvvBHJl9G8eb2wcKFWZ+jss7WkaOpUMETN26xOZDsOIYSIflHTI5Sbm8v48ePp2rUrZ5xxBsuXL2fevHmceeaZAOzatYv9+/dXnD9kyBDef/993njjDfr06cPHH3/MZ599Rs+ePSP1Epo/qxWWLdOKMKoqPPUUnH467NkT6cjqXYHLx5o9xWzOLmVrjovN2aWs2VNMgav6HkohhBBNU1TXEWoMUkfoGM2aBTfdBKWlkJQE774L55wT6ajqRU31iADirAZ6t4mXniEhhIiwZl1HSESBK66AlSu1bToKCmDcOCgqinRU9aLA7a9xzzKpNi2EENGleU7eEE1Dp06wdCncdx+ceiokJEQ6onpR055lAKGwSiBUdTsOIYQQTZMkQqJhWSzw8suV2xYt0pbcX3hhZGI6TuV7lgWrSYak2rQQQkQX+Y0tGldurjZMNmYM3HOPVncoyiTZTdjN1c8BkmrTQggRXSQREo0rPh6uvFL7/4svakNmO3ZEMqI6q2nPsjirQapNCyFElJFVY0chq8YayBdfwLXXahOo4+LgP/+BSy6JdFR1Ul0dIUBqCwkhRBNQ289vSYSOQhKhBrRrl7a6bNky7XjiRHjpJdBHZ+JQ4PJVuydZhxQHSQ7ZtkUIIRqTLJ8XTV/bttqu9fffrx0XF4MuOt+SvkCI7XkuSrxBgmEVFQiGVUq82oatvkD1y+2FEEJElqwaE5FlNMLTT8OZZ2q72SuK1h4IHNzINQrUprZQdTveCyGEiKzo/PNbND8jR0JMjPZ/VdXmC918s7Z/WRSQ2kJCCBGdpEdIND1Ll8LcuVpC9PPP8OGH0LVrpKM6IqktJIQQ0Ul+O4um55RTYN48SEmBNWugf394771IR3VEUltICCGikyRComk680z4/XcYMQLcbvjLX+C668DjiXRk1ZLaQkIIEZ1kaEw0XRkZsGABPPEEPP44vP22Vpl67txIR1atJIcZh9kgdYSEECKKSCIkmja9HqZMgdNOgwkT4G9/i3RER2Q26mV1mBBCRBEZGhPR4fTTYcsWbYl9uUWLwOWKXExCCCGinvQIiehhOmTC8Zo1cM450K4dzJ4NvXtHLKwjOXQbDrNBh8NkwOkLVnssQ2lCCNH4JBES0cnjgaQk2LhR6yV66SW48caDBRmbgEO33NDrFPQ6yC4pw2TQEWc1VTq2Gg2yJYcQQkSADI2J6HTyybB6NYweDWVlWvHFK68EpzPSkQFVt9wwGhR25HvYnu9hd6EXnVL52B8KEwyrePwhdhd6ZEsOIYRoJJIIieiVnKytIHvmGW1S9axZWs2hVasiHVmlLTeMegWvP0Sh26/dqUCRx19xXBYMEwiFcVj0hFWVfJeP7fluSYaEEKIRSCIkoptOB/fdBz/8oG3iunVrk1hef+iWGzpFIRBS8R/YZkOHgj8Urji2GrSq1Nty3azb62TNnhLW7S1hzZ5iCly+iL0GIYRoCWSOkGgeBg/WeoJefhkeeijS0VTaciOsqhj1Cia9Dl8wTBgVk15XcRxvN7GvuIwSbwDQtuRQVSp2rneYDTKBWgghGoj0CInmIzERHntMGyYDbe7QRRfB8uWNHsqhW24EQipWk57E8m02VEiwmUi0m7AYdSiouH2BisdaDDpiLAaMeq0nqdDjb/T4hRCipZBESDRff/87fPaZtnfZiy9qm7g2ksO33AgEVdon2+iQbCMzyUpYLT+2E2M1Ela1niC7SU9Wso0Eu5GwquL2BSlw+WS+kBBCNBAZGhPN1733wh9/wJw5cM89WgHGt9/Weo4aweFbbpgNOnq3jq9SR2h3sRe3L4SqQrLDhFGvY1uum0K3n5Cq4vIF8fpDsqxeCCEagPQIieYrPh4+/hheeUUrxvjFF9CvHyxb1mghlG+5kZVkJz3OisNqrHLcIdlOuyQbiXYTNrOeXYUe9peU4QuGMet1WIz6ivlC0jMkhBD1SxIh0bwpCkycqCU/HTvCrl3avmWzZkU6sgrlw2iJdiM+f5git79imCw93oJRr5P5QkII0UAkERItw4knwsqVMHYsxMTAkCGRjqiSJIeZHq3iSHCYSHKYaB1vpV2ynSS7uaK+kMwXEkKI+idzhETLERsLH3yg9Qq1bXuwfedOyMqKXFwHmI16kh1mMuKsBMMqJoMOBSrmC5mNOgx6BVdZkFirkXibSfYmE0KI4yQ9QqJlUZTKSc+XX0KnTvDUUxAORy6uAw5ddm8yKBXzhQw6hUSbid2FXpb/WcSKPwvZWeBm1a4iNuwrYV+xV3qKhBDiGEgiJFq2efMgGISHH4azz4bc3IiGU9N8obRYM86yALmlPgw6BZNBx7q9Tpb/WcT3m/NYL5WohRDimERNIjRt2jQGDhxITEwMqampjBkzhk2bNh3xMTNmzEBRlEo3i8XSSBGLqPDSS/Cf/4DVCgsWQJ8+2jL7CDp8vlD7JDuxVmPF3mVxNiO7C73sKnTjC4bxBsIUeQKyskwIIY5B1CRCS5YsYeLEifz8888sWLCAQCDAWWedhdvtPuLjYmNj2b9/f8Vt586djRSxiAqKAtddp1Wf7t4dsrNh5EiYOhVCkUsoDp0vlOQwEVbBHwpjMeoIhVTySn3oFe3HNxxWCR4Y1nP7QhS4ZWWZEELUVtRMlv7mm28qHc+YMYPU1FR+++03TjvttBofpygK6enpDR2eiHY9esCvv8Idd2hFF6dM0fYvO+usiIVUPl/I4w9V7FVm0OkIqiqqqmI1aYmQTqdg0Gn/D4VVAqHIz3USQohoETWJ0OFKSkoASDxKlWCXy0VWVhbhcJgTTzyRp556ih49etR4vs/nw+c7OM/C6XTWT8Ci6bPb4a23YMQIbQPXCCZBcHC+0PY8F6AlRiVlAWxGHalxZpQDHbrle5OBtk2HUR81Hb1CCBFxiqo24gZM9SQcDnPBBRdQXFzMjz/+WON5y5YtY8uWLfTu3ZuSkhKee+45vv/+e9avX0+bNm2qfcyUKVOYOnVqlfaSkhJiY2Pr7TWIKJKTo80juv9+MDT+3w6+QIgijx+nN8CuIi92k54deW5ySn1YDDrS4y0k2kzoFAW7WU+PVnGypF4I0eI5nU7i4uKO+vkdlYnQrbfeytdff82PP/5YY0JTnUAgQLdu3Rg3bhxPPPFEtedU1yOUmZkpiVBLpapaz9C332oVqd9/H1q3jlg4hydFRp2CxaijzB9Gp1PIiLeQmWCTREgI0eLVNhGKuj7022+/nblz57Jo0aI6JUEARqORfv36sXXr1hrPMZvNxMbGVrqJFkxR4PrrweGA77+Hvn3hsPlqjcls1JMeZ6VLeixDOyXTJtFGsTdAoceP2xdkT6FXltELIUQdRE0ipKoqt99+O59++infffcd7du3r/NzhEIh1q5dS0ZGRgNEKJqtK67Qtufo2xfy82H0aHjwQQgEIh0ZhS5t9ViMxYhBryMYVmUZvRBC1EHUJEITJ07kvffe4/333ycmJobs7Gyys7Pxer0V54wfP57JkydXHD/++OPMnz+f7du3s3LlSq6++mp27tzJDTfcEImXIKJZ587axq0TJ2rHTz8Nw4fD/v0RC6nA7a+oLXSo8g1ai7yyjF4IIY4malaNvfbaawAMHz68Uvvbb7/NtddeC8CuXbvQ6Q7mdkVFRdx4441kZ2eTkJBA//79Wbp0Kd27d2+ssEVzYrHAK69oq8quv17rHXI4IhZOIBQmFD44xc9k0GEyKIRCKooCews9uMuC2MwGEm2yJ5kQQlQnKidLN6baTrYSLcz27eDxQM+e2rGqalt1GI2NFsK+Yi+bs0srbdCaW+ojFA6zq9CDzaQnI85Cgt1EjMWIxaDDbjbgDYQx6RWsJoNs2iqEaLZq+/kdNT1CQjQpHTpUPn7pJW1n+1mz4Bjmrx2L8oKLJd4gJoPCtlw3igJ7irx4fEHSYy3sKymjoNRPWSiMzajHWRY4UIzRQPtkO2mxFrISbSQ4zI0SsxBCNDVRM0dIiCbL7YZ//EOrTN2vH8yZ0yiXPXyDVq8vSFgFpydAu2Qb+0rK8AdVthW4cZeF2JLnYu0eJyXeIA6zgbJACI8/xB/ZTjZnO2UHeyFEiySJkBDHy26HX36Bk0+GkhK45BJtqw5fwy9hP3SD1uQYMw6zntQ4M3pFh8sXhAOJkaqE2V/kxWHVk2Q3ke/ys7+kjEUbc/lyTTb/W7uflTsLZem9EKLFkURIiPqQlaXVGbrvPu34lVdgyBA4Qs2q+lK+QWuyw0yc1YjDbCCoqoTDKmFUQqoKqoI3GKJVrNZT5A0E2VngYUteKSXeAMVuP7lOH6VlQXYWuKVnSAjRYkgiJER9MRrhmWdg7lxIStJqDw0aBI2wX5026VmH1aTXagopCjqdgg4F84HVZOmxFowGcJYF0Cs69pd40aMjzmqgfZId04E5RF6fVr1aCCFaApksLUR9O/dcWL0axo2DCy+ERlhtWD5faHehm9bxFkq9QQrcfowGhYw4K2aDno4pDgJhFQXQKdry+7QEC60TrOS6fPiDYVrHW/D6Q/hC2vyh1vFWWVUmhGjWJBESoiG0aQOLFsEhda3YsAH0eujatUEumeQw4zAbKPL4CYVVWifayHV6CalQ7A5gNugo9QUp9foxm3Skxlpom2ijyBPAVRakVZyF0rIQ/mAYty9IXqmPvGQ7nVIdJDksDRKzEEJEmiRCQjSUQ3eqd7m0SdS7d8P06XDVVQ1yyfK9yABaJ4Av1UGRx08gFKYsEMZsUIizGtDpFOIsJlRUdhR4aB1vxeMPYTbq2FfsQ6+DRLsZbyBMrrOMLumxtEuyS++QEKLZkURIiMbg9UJ6OmzcCFdfrfUW/fOfYLM16GUPTYzK2c1Gdhe6SXWY2VXoxWbUYdCDQa8nt8SHNxAiM9HG/pIyNuY4Mep0/FnoZUBWAl3SYkiSmkNCiGZEJksL0RhSUuDbb+Gxx7Qd7f/zHzjpJG24rJElOcx0y4gjM8nGCRkO+mYmkJVox2rQ4/EHaZ2gJUFb81z4/CoWg55AMExOiU82cxVCNDuSCAnRWPR6mDJFS4jS02H9ehgwAGbMaPRQzEY9abFWOqbE0C7ZRpJD22rDoNeBCvuLvSTbTfRoFUNGvBW7yYBy4LeFrCgTQjQnkggJ0dhOP11bVTZypDZk9u67EA5HJBSzUU9Wkp0Eq7GiBpGKisOsp3Oag1yXn3V7i1m3t5gdeW72FHpxlQUjEqsQQjQEmSMkRCSkpcG8efDCC9rEaV3k/iYpX21mMxvwBkO4vAG6t4plT3EZOwvctI630j7ZjsNiwBcKkVNaRnqsBYe18TaYFUKIhiKJkBCRotPBvfdWbvvrX7Xl9TfeqM0laiRmo552yQ6sJj27Cz3syPewcb+Ldkk2MhNtBEIqLl8QV5lKIBjGHwzTs3UcKTGyrF4IEd0kERKiqVi8WOshAm1V2fTpjVKM8VBpsVbirSYUReHPAhcpDjMuXwgdkF1cdmD3eshz+fD6Qwxon0BqjPWozyuEEE2VzBESoqk47TR4+mltUvWsWdC/P6xa1ehhmI16MuKsDMxKJNlhIdaip6QsSL7Lh81koEOqgxiLEWdZgC05LlzeQKPHKIQQ9UUSISGaCp0O7r9f27w1M1PbsPXkk+HVV0FVGzWUJLuJeJsJu1mPougo9gaIt5lIj7Pg84fwB0Nkl/jYtL+UVbuLKHCVNWp8QghRXyQREqKpGTJE6wk6/3zw+2HiRLj55kYNwWzU0zbRRqzFiE5R8AdCpMVZKPOHMBoUdheVsSPfRYHLz5bcUpZuzWdTtlNqDAkhos4xzREKBAJkZ2fj8XhISUkhMTGxvuMSomVLSoLPP4cXX4QHHtA2cm1kiQ4zJr0OZ1mAdsk27CY9Rh3sLioj3+WjQ7KdArePXYUeHBY9zrIgXn+Itok2EqX6tBAiStS6R6i0tJTXXnuNYcOGERsbS7t27ejWrRspKSlkZWVx4403snz58oaMVYiWRVHgnntg2zZtF/tyO3Y02lCZw2qkZ+s4TkiPJcZiIBCC7BKvtlmr20+JN0DPVjH0zUzAqNexr9jLzgI3bpk3JISIErVKhF544QXatWvH22+/zciRI/nss89YvXo1mzdvZtmyZTz22GMEg0HOOusszj77bLZs2dLQcQvRcmRmHvz/nj1aNeqLL4aioka5fEqMhQHtE8hMtBFrMRBvNRJjNhBWVfpmxoFOxy87CliyMZddBW72FntZtqOAFTsK2FXgluEyIUSTpqjq0f+0HDduHI888gg9evQ44nk+n4+3334bk8nEddddV29BRpLT6SQuLo6SkhJiG3kpsxBVfPopXHGFNncoK0tbXXbyyY1yaZc3wPKdhazdU4zZYMDjD2A1GVi7t4RCt5+T2iViMerYkuOi0O2jfYqd7q3iyEq00Sk1RobLhBCNqraf37VKhFoySYREk7NyJVx+uTZkZjDAtGkwaVKjVKfOKy3jl20FeIJhnF4/+S4fy3cU0S09lpRYM5tySilw+eiXmUDbRBsGnYLdbKBVgpXOKQ6pRi2EaDS1/fw+5t+cW7duZd68eXi9XgAknxKikZx4opYMjR0LwSDcdx9ccAHk5zf4pVNiLAzskEjHFDspDjMKCnEWA4l2Ey5fEK8vSL/MeCwmPRv2FbMxx8maPcX8vruIVbuLyC+VZfZCiKalzolQQUEBI0eOpEuXLpxzzjns378fgOuvv56//vWv9R6gEKIasbHwwQfw+utgNsOXX8IzzzTKpdNirfRsFUentBjaJFhpk2jDZtITCodJjdVWmu0qcBMIg6sshMcXYnu+iw37S1i9q4hcp7dR4hRCiNqocyJ0zz33YDAY2LVrFzabraJ97NixfPPNN/UanBDiCBRFqy/0yy/a5OkpUxrt0majnk4pDnq0jqNTagxWk55Uh4WMOCslZUEMOoVgWEWngDsQJBiE/cU+1u51smxbITklkgwJIZqGOtcRmj9/PvPmzaNNmzaV2jt37szOnTvrLTAhRC316QOffHLwOBzWkqLbb4fU1Aa7bHnRxVAoTL4rQFkgSJHHj8sXxK/XYVChLBBC0SnkunzkOX2ghNme78blCzKyeyppsbJPmRAisurcI+R2uyv1BJUrLCzEbJZVIUJE3NNPwxNPQN++sGRJg14q0W6mZ+t4+mTGcUJ6DB1THLRPspPmMKMqYDToKPb42ZbrwhMIEmsxYdLrKHD52J4nS+uFEJFX50Ro6NChvPvuuxXHiqIQDod55plnGDFiRL0GJ4Q4BhdcAN26wf79cPrp8PjjEGq4hMNs1JMeZ6VrRhz9MhPomhFDVoqdjDgLZqNCXqmflBgz/TLj6d82kVM7J9OvbTxmg8If2U5+3VHAql1FUnNICBERdR4ae+aZZzjjjDNYsWIFfr+f+++/n/Xr11NYWMhPP/3UEDEKIeqiRw9YvhzuuAPefhsee0zbyPW99yA9vUEv7bAa6ZDiwGHSY0DB4wtiN+vpmOogHA6j00OR24fT68cbCLGzwEORx4/DbKBDioPurWLp1TqeJKk5JIRoJHXuEerZsyebN2/m1FNP5cILL8TtdnPxxRezatUqOnbs2BAxCiHqym6Ht96Cd98Fmw0WLtTmEn3/fYNfOtFupn1KDH3axtElLYYB7RLw+IMYdDpUFYo9AXJK/SzfUcTynUV4/GEUFEq9AbbmuNicUyo9Q0KIRnNMm67GxcXx8MMP13csQoj69pe/wMCBWgHGzZu1ZfeNwGzU0zbJgcWoR6/Tkev0kewwE1ZVgmGV0rIg3kCIUzsmk5lgI4yK3WQg3mZEAYo8ftLjZCK1EKLh1SoRWrNmTa2fsHfv3scczJFMmzaNOXPmsHHjRqxWK0OGDOHpp5+ma9euR3zcRx99xN/+9jf+/PNPOnfuzNNPP80555zTIDEK0SSdcIK2xP7nn7UJ1OV8Pq0GUQNKjbXSMdVOdmkZNqMepzdAOKxiNevo2SoWk0HPxhwnCmDU60h2mGmXbCct1tKgcQkhRLlaJUJ9+/ZFUZSjVo9WFIVQA03KXLJkCRMnTmTgwIEEg0EeeughzjrrLDZs2IDdbq/2MUuXLmXcuHFMmzaN8847j/fff58xY8awcuVKevbs2SBxCtEkWa1w6GKGFStgzBh48004++wGvXSi3UzXVAe+YBidolLqVUixmMlz+tie78ag15Hv8pFd4iXGbKSkzI/VpCfGqifZIb1CQoiGVau9xupSHygrK+u4AqqtvLw8UlNTWbJkCaeddlq154wdOxa3283cuXMr2k4++WT69u3L66+/XqvryF5jolm65BKYM0f7/wMPaMvtjQ2zD5gvEGLNnmJCqkqxO0CpL4hRBxuzS8l2lpFf6ifXVcbArAROyIglwWpEVRQSbEbaJdlJibFgNuobJDYhRPNV28/vWvUINVZyUxclJSUAJCYm1njOsmXLmDRpUqW2UaNG8dlnn9X4GJ/Ph8/nqzh2Op3HF6gQTdHMmXDvvfCvf2l1h374QdvJPjOz3i9lNurpkOJgd6GbJIcJu0lPWTBEcowZjz/IvmIvQzomkx5rpsDlZ+2eEpLsZjqlOQiEVHbku4mxGEl2mCQpEkLUu2OaLA2wYcMGdu3ahd/vr9R+wQUXHHdQRxMOh7n77rs55ZRTjjjElZ2dTVpaWqW2tLQ0srOza3zMtGnTmDp1ar3FKkSTZLHAK6/A8OFw/fWwdKk2f2jGDDj//Hq/XJLDjMNsoMjjR1VVijwBAEIhFZNBT6zFiMsXYk+RlzibgRirgfzSMnblu3H5QyTajbRJsJGZZKVdkoNEuyyvF0LUjzonQtu3b+eiiy5i7dq1leYNKYoC0GBzhA41ceJE1q1bx48//ljvzz158uRKvUhOp5PMBvgrWYgm4dJLtd3sx47V5g1dcAF8913l+UT1pLzwIkCiPQSqSiAEer2CqyxIaVkQo17BpNcRCIVx+kPsKfRiNum0bTuCKmaDnnizD7vJID1DQoh6Uec6QnfddRft27cnNzcXm83G+vXr+f777xkwYACLFy9ugBAru/3225k7dy6LFi2qst/Z4dLT08nJyanUlpOTQ/oRisqZzWZiY2Mr3YRo1jp0gB9/hLvugnPOgWHDGvySZqOe9HgrHZOtJNu03h2DDmwmA2FVxazXsc9ZhqKA1x8m2WGiXZINtz/A5jw3W3NLcXsDDR6nEKL5q3OP0LJly/juu+9ITk5Gp9Oh0+k49dRTmTZtGnfeeSerVq1qiDhRVZU77riDTz/9lMWLF9O+ffujPmbw4MEsXLiQu+++u6JtwYIFDB48uEFiFCJqmc3w4osQDILuwN9HbrdWgHH06Aa5ZKLdjN1kIIiCLxTCatLj8QchDEFVRaeqlIXDtEu0kR5r4fe9xegUSHFYyCstI8dZRodkOxnxNukdEkIcszr3CIVCIWJiYgBITk5m3759gDahetOmTfUb3SEmTpzIe++9x/vvv09MTAzZ2dlkZ2fj9Xorzhk/fjyTJ0+uOL7rrrv45ptveP7559m4cSNTpkxhxYoV3H777Q0WpxBRzXDI30YTJ2o9RHfeqdUcagBmo55OKQ66ZsSSmWgn2WHGbjZiMxkw6HUY9DrS4qxsyi7FGwij1+nZW+QhGA7jD4bYX+zj9z1FrNhRwB/7S9hX5JGq1EKIOqlzj1DPnj35/fffad++PYMGDeKZZ57BZDLxxhtv0KFDh4aIEYDXXnsNgOHDh1dqf/vtt7n22msB2LVrFzrdwdxuyJAhvP/++zzyyCM89NBDdO7cmc8++0xqCAlxNOEwpKZq/3/5ZW0y9ezZ0ADb6JiNetom2lBQibUYyHaWoVcUWsVZcQdCeAJB3P4gsVYTRW4/XdNjCIcVduR7CONBB9hNevR6HSkOM60TrLSKt5Bgl6KMQoijq1UdoUPNmzevYn+xrVu3ct5557F582aSkpKYPXs2p59+ekPFGhFSR0i0aF9+CddcAwUFEBOjFWC8/PIGuZQvEKLE66fI48cdCLK3sIxcp49QOMz2PDf+UJi2iVYURaHArW3UatTpcPtClAUCdExzkGgzoaIQazGSlWSjTYIMmwnRUtX287vOiVB1CgsLSUhIqFg51pxIIiRavD17YNw4bUI1wC23wP/9n7YEv4H4AiFynV427C+l1BdgR54HXzBESoyZ/SVe/EGV9DgLJd4AxQd6iXyBMDvy3YRVlZRYM93SY8lKstEuyU5SjPQOCdHS1Pbzu85zhEpKSigsLKzUlpiYSFFRkRQfFKI5atMGFi2Chx4CRYFPPoGioga9pNmoJzPJQd/MeNrG22kdbyXRZjqwtB6sJj0GnUKBy0/reCtuf4jVe4owGHR0TY/hhPQY/KEwe4q87CxwU+jyHv2iQogWqc6J0BVXXMGsWbOqtH/44YdcccUV9RKUEKKJMRjgySfhm2/ggw8gI6NRLpsWZ6Vjmo0erWNpm2jDYTFiMSqYDQpGg45EuxGLSc+eIg9JdhOdku1YTQZ+31XCrgI3gVCYPws8rN5dwoZ9xbhkyb0Q4jB1ToR++eUXRlRTbG348OH88ssv9RKUEKKJOussOOOMg8effAI33AAeT4NdMslh5YT0WHpmxtM5zUHHZAd2k4EYk4GMOCtGvQ4FhfQ4K2ajjo3ZTgx6hdRYK9tz3ewp9vLHfierd5ewfGcB+4rdDRarECL61HnVmM/nIxgMVmkPBAKVlrILIZo5pxNuvFEbJvvlF/jwQ+jWrUEuZTbqaZdkxxdrwW4ysC23FINOR6zFiDsQonWchSS7iRJvELcvSOcUB3uLPViNevYXe/H4g9iMXoo8VlQVdIpCepytQWIVQkSXOvcInXTSSbzxxhtV2l9//XX69+9fL0EJIaJAbCx8/DGkp8O6dTBgALzzToNe0mzU0zkthqGdUuicHkOHFBtd0hx0zogh3m7GaFRIsptw+UPYzQZyS30EQyqt4230bBNHr1bxGPQ6NmWX8vuuIvYXS90hIVq6OvcI/f3vf2fkyJH8/vvvnHGgi3zhwoUsX76c+fPn13uAQogm7PTTYfVquPpq+PZbuPZaba+yV18Fu73BLmu3GrFbjYC2wsxhNpDvLCMUUgkEwljNOpzeEHoFUuMt2I16zHodW3NdqKi0irUQDKvku8qwmAykxVjITJSl9kK0RMe0fH716tU8++yzrF69GqvVSu/evZk8eTKdO3duiBgjSpbPC1ELoRD84x/w6KNaMcZu3WD58gZNhg5X4vazo9DF1hw3oFLk9uPyhSj1BcmIs6CqYUJhiLeZ8PiC5Lp8FLr9WA062iTa6dU6jq4ZMbKzvRDNRG0/v+vcIwTQt29fZs6ceczBCSGaGb0eHn4Yhg7Vag6ddVajJkEAcXYTWYodg6JQ5PFjNxnYW+LB5Q8SUlWCIRW7yYDT62d/iY/Ve4rZU+ghLdaMLxBGr1Mw6hXsbWRneyFakjonQitXrsRoNNKrVy8APv/8c95++226d+/OlClTMJlM9R6kECJKnHYa/P67VoW6XE4O2GyV2xpIgs2MzajNDfIFAgRVFWdZEItBR0in4AuFCYZUNmY78QfD/OXkLNLiLCiKQozZgKqqlHj9pBqtDR6rEKJpqPNk6ZtvvpnNmzcDsH37dsaOHYvNZuOjjz7i/vvvr/cAhRBRJjlZ280etN3sL7tMm0i9enWjXN5s1JOZaCMz0UH7ZDtpDjM2kx6H2YDVoMcXCgMqF/bJwGrWk+PUdrLfV+xlZ6GHLdml5LvKGiVWIUTk1TkR2rx5M3379gXgo48+YtiwYbz//vvMmDGDTz75pL7jE0JEs9274c8/YfNmOPlkeO01OP5dfWrFbNTTOTWGjmkxJNhNOMxGHBY9Jp1C/7aJADg9QQIhlWJPgHX7nGzMLuXPAjeb9jnJK5VyIEK0BHVOhFRVJRwOA/Dtt99yzjnnAJCZmUl+fn79RieEiG7t28OqVXD++eDzwW23wdixUFLSKJc3G/V0SYshPdZCeryZ1BgLHVIcpMdZCIRVFB24fEEK3T4yE6zYTQY8/hCFngAFLr8srReiBahzIjRgwAD+/ve/89///pclS5Zw7rnnArBjxw7S0tLqPUAhRJRLSoLPP4cXXtC26vjoIzjxRFixolEun2g30yUtlqxEG5mJVjqlaYmQ5cCS+jJ/iLZJNvJdPv4scLE110W+q4ycEi85JQ1XMVsI0TTUORF68cUXWblyJbfffjsPP/wwnTp1AuDjjz9myJAh9R6gEKIZUBS45x746Sdo1w62b4ebbmrUYbLkGCvtkmNol+SgVbyF9Bgzep1CeryFglIfIRX6tk3gzB7pJNotOMtC7CqSZEiI5u6Y6ghVp6ysDL1ej9ForI+nazKkjpAQ9ay4GG6/XdvNvnv3iITg8gZYv7+EfcVe9pVovT+dU2OItRlZv89JnrMMk15H2yQbbZMc9GwVQ/uUhl/1JoSoPw1aR6g6Foulvp5KCNGcxcfDe+9Vbps+Hfr2hUGDGiUEh9VI20RtrzGvX1tSn2g3sWpPMUXuAGmxVqwmPYqi4A8GySv1YTcbSI2VZfVCNDf1lggJIcQxWboUJk7Uhs/+8Q+YNEn7fwPLiLeh14E/GMZq0uH0BQkc2MA1DJgNCq0TrBh1ekp9QXbkuyktC9ImQbbiEKI5qfMcISGEqFc9esAll2g1h+69Fy64AAoKGuXSqbE2umXEkBpjxqCDeLsZRVFwmPTEWU2UeoPklnrZsNfJlpxSdhe62Vngptjja5T4hBANTxIhIURkxcXBrFnw+utaIca5c7Vhsp9+apTLJ8dYSYu1kuowE2c1otcrWIx6gqEQZYEwuwq9mI0KBr0Oly9EXqmXnJIyXN5Ao8QnhGhYkggJISJPUeDmm+GXX6BLF9izB4YNg+efb5TLZybayEyw0yrOSrzFgD8UxmoykFvqJclhwmoyoAe8/hD7Snzsd3rZVeSmUCpQCxH16pQIbdiwgdtuu41+/fqRkZFBRkYG/fr147bbbmPDhg0NFaMQoqXo00erL3TVVdqO9o20CMNs1JMUYyIz0UJSjNYz5A2EsBj1OIx6DIqOIm+AIo8fVYX80gC7C71sy3WR65QK1EJEs1pPlv76668ZM2YMJ554IhdeeGFF8cScnBwWLFjAiSeeyOeff86oUaMaLFghRAsQEwP//S9cey2cccbB9rKyBk2M4mxm9IoOry+MWR/A7Q/iLgtiNRso8vgJhVXirSaKvX78gTClZQYUwO4uw6hAQoysKBMiGtW6jlCfPn248MILefzxx6u9f8qUKcyZM4c1a9bUa4CRJnWEhGgCSkrgpJPg6qu1+kP6hlu1le8qY2+hh9KyIHkuH/5QmDynjySHmdKyAKGwSkaclTYJVuIsBsqCYYq9AYx6HQl2E63jrDiszauemhDRqLaf37UeGtu8eTNXXXVVjfePGzeOLVu21C1KIYSojVmztI1bH30URo2C7OwGu1Syw0KbRCut4qykOCzYjQZsJj26Ayv602OtxFoN+IIhtuS6Wb6rkPV7S9iSW0q208uOAhc5TqlGLUS0qHUi1K5dO7788ssa7//yyy/Jysqql6CEEKKSm2+Gd94Bmw0WLtRWlS1c2GCXS3JYaZ1oo12SlXiHkZQYMygK8TYj/nCIYEilxBNgT7EHnz9M2yQ7nVJj8PpDFHn8FLuDFLll7pAQ0aDWc4Qef/xxrrzyShYvXszIkSMrzRFauHAh33zzDe+//36DBSqEaOHGj9eGxy67DNatgzPPhEcegccea5ChMrNRT5skB4oOFMDmD+H2hfH6A5QFwvjD2s70WYl2FAU27HOSmWDBoOjYW+yh0K0n0e6jdbxdhsqEaMLqtNfY0qVL+ec//8myZcvIPtA1nZ6ezuDBg7nrrrsYPHhwgwUaKTJHSIgmxuuFu+6Cf/9bO37oIXjyyQa95J5CF4WuADmlZZSUBbAZDTjLAigKJNpM7Mh30zrBhqKqlHgDGA06/EEVgx6SHGa6pcfSKsHWoDEKISqr7ed3vW262lxJIiREE/XBB/D44/D995CS0uCXyyv1sD3HTY7Lh82oJxRWCakKxR4/Bj04TAYKPH4CQZWVu4vYklNKWoyZTmkxdEy2c3LHZDqmysatQjSWRt90VQghGtW4cdowmeGQX2MffwxjxlRuqycpMTbCYYWyUBi9omAyKHj8IYJhA6FwGE8wRDissimnFIfZwF8Gt8eoA4fZiM2kw+kN4PEGsMkwmRBNSr1Vlv7jjz/o0KFDfT2dEEIc3aEJz7vvaonR8OGwe3eDXC4tzkrHlBjsZgOxVhNWk54EmwGbyYBOUfAFwyQ7TPRpHY9KGL1Owe0PAmA3G/AHgw0SlxDi2NVbIuT3+9m5c2d9PV21vv/+e84//3xatWqFoih89tlnRzx/8eLFKIpS5ZbdgEtvhRAR4nBAbKy2R1nfvnCEVa7Ho02ija6pMcRbjWTEWUmwm3CY9VgMOmKtRlrFWwmqYQxos6y7pjvomOJADav8vtfJ8u357M4vxeP1N0h8Qoi6qXX/8aRJk454f15e3nEHczRut5s+ffpw3XXXcfHFF9f6cZs2bao0PpiamtoQ4QkhIunii7UtOsaOhd9+g/POg7/+FZ56Ckymer2UzWqsGOLaV+TBatCjVxT0Oh0uX4BQSEWnU2h7YIL05lwXeaU+uqQ5SLab0Ov15Lp85Gc7CYXBatITZzWQHmfDbGy4YpFCiKpqnQi99NJL9O3bt8YJRy6Xq96Cqsno0aMZPXp0nR+XmppKfHx8/QckhGhaOnbUeoQeeABeeknbtPXHH7WCjO3aNcglWyXY8HgDGPQ6TG4/YVXFWRbEeiChKfEE2F/ipXNqDAa9gssfotTpp8jjA6DIHSDeasDjMBEIqSTHmIm3mRskViFEVbVOhDp16sQ999zD1VdfXe39q1evpn///vUWWH3q27cvPp+Pnj17MmXKFE455ZQaz/X5fPh8vopjp9PZGCEKIeqL2QwvvqjNFZowQdvRfuPGBkuEQOshyrIaUVUX/kAInaKgVxS8wRD7S8rIiLXgD4bQKRDUhSl0+QiEVXKcZcTbjBR4/NrN7UcBrEaD9AwJ0UhqPUdowIAB/PbbbzXerygKTW0lfkZGBq+//jqffPIJn3zyCZmZmQwfPpyVK1fW+Jhp06YRFxdXccvMzGzEiIUQ9WbMGFi9GqZPh7PPbpRLZsQfnDNkNigoKsRZjOj1CsGwSiAcJs8VwGoykOsqw24x8tvOYt74fjsLNuTg9gbwh8KUeGT+kBCNpdZ1hLKzs/H5fE1mGw1FUfj0008ZM2ZMnR43bNgw2rZty3//+99q76+uRygzM1PqCAnRHOzYATfeqCVHHTs2yCUKXGXsKfJQ6g0SQsWoUwiFVMKqitmowxcMU+QJEgiGWLGzmNKyIDef1p6UGAtef5A9RV6sJj3psWaS7CbsVhkmE+JY1HsdofT09HoJLNJOOukkfvzxxxrvN5vNmM3yi0eIZum227Q9yk48Ed58U1tuX8+SHBbCqso+ygiFw+gBNyECAZWQCjaTkUBIZX9ZELMerhzWHotBx458N/uLvXTJcJBss+ALaYmS1aQnI9ZMgt2Iw2qp93iFaOnqbfl8tFi9ejUZGRmRDkMIEQlvvAGnnAJOJ1x+uZYYlZXV+2VSYqxkJlqIMRvQ6XWkxlgwGfQY9TrsZh0JNhM6BU7umIxB0SZQ7yny0D7FjlmvZ1eRh5W7ikmNNZFkN7E938NP24r5cUseeaX1H68QLVmdy68mJCSgKEqVdkVRsFgsdOrUiWuvvZYJEybUS4CHcrlcbN26teJ4x44drF69msTERNq2bcvkyZPZu3cv7777LgAvvvgi7du3p0ePHpSVlfHmm2/y3XffMX/+/HqPTQgRBTIzYfFiePRRmDYNXnsNli2DDz+Ezp3r9VKJdit2kwmXzw+qiqpCWTCEqiqkOEwEU2ygaAUXc0r9JDnMgEowFGZPkYfOaQ5KvCEWbtzPoo05ZCXaOL9PBnEWPYFAiFaJ9nqNV4iWqs6J0KOPPsqTTz7J6NGjOemkkwD49ddf+eabb5g4cSI7duzg1ltvJRgMcuONN9ZrsCtWrGDEiBEVx+W1ja655hpmzJjB/v372bVrV8X9fr+fv/71r+zduxebzUbv3r359ttvKz2HEKKFMRi02kLDhsFf/qJNqD7xRPjuOxg4sF4vZTbqMRutACTFQLHHh8cXwqyH9skxZDvL8AVDOEx6DAcmVO8pKSPRbsKgKCz4I4f9xWU8f1kfEuwmnJ4A3kAITyBMbqmXGIMBq2zZIcRxqfOmq5dccglnnnkmt9xyS6X26dOnM3/+fD755BNefvll3njjDdauXVuvwUaCbLoqRDO2bx9ceSW4XFr9oUaeH7hhXzHusgA6nY5gOIxBp+1fZtApOH0hftmez4V9WwPg9oWItxmItZhwlwW1JCqkEm810iUthnh7/RaNFCLaNdju8w6Hg9WrV9OpU6dK7Vu3bqVv3764XC62bdtG7969cbvdxxZ9EyKJkBDNXDAIhYVQXnE+GISdOxtsVdmhClw+8pxl6HQKJR4fYcBi1DZx3VXoxaRXyEq04SwLYrdo1atznD4Wbcpje14pA9smMLB9IskxZuKsRtLjZbhMiHK1/fyu82TpxMRE/ve//1Vp/9///kdiYiKgbYURExNT16cWQojGZzAcTIIApk6F3r21TVwbWJLDTIzFiEKYJIcZo15PjMkAqkqrODNJDhMFHj96nYJOVSh0B1izp4RB7RN49pK+XDqwLXFWE/tKfKzbV8rqXUW4vIEGj1uI5qTOc4T+9re/ceutt7Jo0aKKOULLly/nq6++4vXXXwdgwYIFDBs2rH4jFUKIhhYKwa+/gscD11wDixbBK6+AveF6Wlon2nB69ITCIRQUguEwCXYzBp1CkSeAjiA6RcXpC+D2BemTGUebeCv57jJ2FnpZsimXQDDMye2TSHMYyCstwx8Okmi3NljMQjQndR4aA/jpp5945ZVX2LRpEwBdu3bljjvuYMiQIfUeYKTJ0JgQLUwopK0oe+wxCIehWzdtVVnPno1y+RKPj7Aawh8Mk1caREVFVVVKfUHCYYixGjHqYEe+h33FXga1TyQjzkIwBIVuH/5QmDirCavJQKzFgEMmU4sWqsHmCLU0kggJ0UItWaJNpN63DywWrWfouuugmvIhDSW3xIurLIjLFyAYDle0u/whvL4QrRIsOMwG3L4Q3kCYeKueMAr7ig+sRjMb6ZxqJy3O1mgxC9FUNGgiFAqF+Oyzz/jjjz8A6NGjBxdccAF6ffPbJFASISFasLw8GD8evvkGrFbYtEmrRdSI9hZ68IeCqKqCLxgiEApT4PaTYDVh1CsYDTpKy4KYDTqKPEG+25TLqj8LOLlDIiO6ppIWa8VuNpASJ0NlomVpsERo69atnHPOOezdu5euXbsCsGnTJjIzM/nyyy/p2AgrLRqTJEJCtHDhMDz7LKSlwbXXRiQEt8eH0xektCyIJxBEAVRAQesdijUb8YfCfLcxl4x4C2d0TUOvQKlP27usLKgSazXQJS3mQOFGIZq/BkuEzjnnHFRVZebMmRWrxAoKCrj66qvR6XR8+eWXxxd5EyOJkBCiiqVLYc0auPnmRh0qKywto8Dtw2LU4yzTVocFg2GCYZUCTwB/MESPVrEEQyq5pX4Wbswlu9jDwHYJ9MuMJ9lhxm42kBAjvUOi+WuwRMhut/Pzzz/Tq1evSu2///47p5xyCi6X69gibqIkERJCVFJcDL16wZ492n5lb7wBcXGNdvl8p4dAKEShJ0QwFMZi1OH2hSh0B8iIM2M06HD5QqzZU0z3jFg6JNlQFHD7w+zId1MWVLGb9XRJc5AaKwmRaL4arI6Q2WymtLS0SrvL5cJkksqmQohmLi4O7rlHqz/04YfQvz/89lujXT451kacxUKyw4TZoMdq1GPUKaTEGAmjUloWoCwQol9mHGmxJtyBENsLvLz3y27+uWgzf+wrwqqHgD9IYam30eIWoqmqcyJ03nnncdNNN/HLL7+gqtqyzp9//plbbrmFCy64oCFiFEKIpkNRYNIk+PFHyMqCbdtgyBB4+WVopEW4NquRtDgbMRYjKiqJDjNxVhMGnQ5QsJsNmA0GfEEVXzDM0m0FdE2389q4/vzl5CxSYq1szfewfGcJP27JI9cpCZFoueqcCP3zn/+kY8eODB48GIvFgsVi4ZRTTqFTp0689NJLDRGjEEI0PYMGwapVMGYM+P1w551wySXgbbykonWijQSbmRiLgg5QwypWkw69AoFwCF8wRE6pjx4ZDgZkJeANVu4d2pHnJN6iAzVIqUuSIdEy1bmydHx8PJ9//jlbtmxh48aNAHTr1q3K3mNCCNHsJSTAnDlajaF779V6iyyWRg0h1qatAjMbQyhOL4oCXn+IYFhFReukaptoxxfU6hAt3VZAnzYx3Dw0C51OCzmsQlFZiJV7cmQOkWhxpKDiUchkaSFEraxcCR06QHy8duz1arvZ6+rc8X7MSj0+3H4/zjIVfzCkJToohMIqKirF3iBqOEyHFAcAOkUhFA6T7fSzYEMum3KKOb1LKoM7JpHo0GPWG4ixSTIkolNtP79r1SM0adKkWl/4hRdeqPW5QgjRbJx44sH/q6pWc8jthnfegaSkRgkhxmbGZDSg0/lxekGnoC219wYIo6Cq0Cbehv9A75Bep+ALhlm9u5gRXZO4dVgWyoG8rcwP2cVl5O1yEgqrJDnMdE5xyJYdotmpVSK0atWqWj2Z0oj1NIQQosnauBE+/xx8PujbF2bNglNOaZRLm416Uo1W1LBKIBwiHAoTDqugg/QYM4FQmDCgVxTK/CGc/hAntYsnJcZCWUAbICjy+tme52XJphyWbi8kHFTp0SaWUzolc3q3NLKSGm4TWiEamwyNHYUMjQkhjsnq1VqdoS1bQK+Hv/8d7r+/UYfKPB4/nlAQrz9MIBTGZjLg8gVQVa03yB8M4w6ESLCZ0B/yd+z6faX87/d9fL0+h6sGteGCXhmkx1mxGMGmN2AwGLBKz5Bo4mTT1XoiiZAQ4piVlsItt8D772vHZ58N774LKSmNGobL48NZ5gcVSn2hih6h6vrwC91+Vu4pxqBTuKhvK4wGKs7zBaHQFSDbWYYvpBJvM9IlNYZ4u9SQE01PvRZUvOWWW9izZ0+tLjx79mxmzpxZuyiFEKI5i4mB996DN9/UVpN98w2ce26j1Rsq57CZSYqxYTIbsFsMmA06DHoFu8WA1WzAoFcqbgFVpXfrOM7rlUEwpOIPaPOFcpx+1u0t5b1fd/HDtjwSLDraJZrQh4N4GrFkgBD1rVZzhFJSUujRowennHIK559/PgMGDKBVq1ZYLBaKiorYsGEDP/74I7NmzaJVq1a88cYbDR23EEJEB0WB66/X6g6NG6dt4BqB+ZRmox6z0Yrb46OkLEAYbZQu6A8TDB1MzIyKQmqchWBYJayq6FVtknVOaQC3P8AdIzpSPirmC8K+0vIeohLpIRJRqdZDYzk5Obz55pvMmjWLDRs2VLovJiaGkSNHcsMNN3D22Wc3SKCRIkNjQoh6Ewpp84XKLVig7VuWnt6oYfgCIbwBP6qqEgxDmT9M+MBHgc0MTreWBBl0OsKqSrHHj06vkBFnxXAghyvy+tlT5OO7Tbl8vyUPlzdI1/QYhnRK4qzu6XRJl9+XIrIadI5QUVERu3btwuv1kpycTMeOHZvtijFJhIQQDWLjRhgwABwOmDkTzjgjYqEUuT34/FoBRrtZR4k3RCgcxqjXEqH9Th/JdhMmw8HZFH8WePnf7/uYvWIPVw1qw2X9WpMRb8akBwMGDAY9Vqv0DInIqdc6QodLSEggISHhmIMTQogWT1GgfXtYtw7OPBP+9jd49NHKPUaNJMFuw6MLEFADKKgEgqpWhPHA8FiS1YSiKgQOLK8vdPvYW+ThhDQ7Kx8ZgVGnTagOAqXeMNvznJQFVWwmrUJ1WpwUZRRNl6waOwrpERJCNBiPB+66S5tMDTB8uNY71KpVRMPKKfHiD4a0OUI6BbtJjzcQBrSPi73FZegVhcwEKwpaTucLhtnv9DF/fS6LN2ezp8hHu0Qbp3VN4ewe6fRpK388i8bVoD1CQggh6oHNBv/+N4wYATffDIsXawUY33sPzjorYmGlxVnJLfZUJEKKquIPhCsWu+lUSIox4w9rbeWJ0Lz1ubzxw3aGdEzkkXO60TnVjsUAegx4vV6sVukZEk2PJEJCCBFpV16pzRe6/HL4/Xf47ruIJkIAqfE2XB4vIVQUwGrWEwhpW3NkpVjwlKn4/do2HmXBEPucPsJqkKX3DyfWcrBGkQIU+0Js2eumLFgiw2WiyZFESAghmoIuXeDnn7Wd7O+6K9LRAOA4ZMPVWKDA6UFVwWJQCBpVwmoYnaIQVMPEWgxcd0oHzHoFf0h7jKJok6rnrdOGy3YW+WiXYGPYCSmM6p5OvywZLhORV+da76effjrFxcVV2p1OJ6effnp9xCSEEC2TxQL33gvGA4V6/H6tAOPcuZGN64CkWBtWo4KqqvgDIXx+Fd+BOkRxViOgUhYKUxbUbt5AmHnrcnn9h+1szPHw0Ogu/Gd8PyaemkmnZAteKcQomoA69wgtXrwYv99fpb2srIwffvihXoISQgiB1jv01Vfa7a9/hWnTDiZJEWI/0EvkCXgJhkOoKiRYDQT9KoGAiu7AmJhOga35HhZvzubNv/RjcPv4w7b0UPGrUFDoYluuW1aZiYipdSK0Zs2aiv9v2LCB7OzsiuNQKMQ333xD69at6zc6IYRoySZOhJ074Z//hOefhx9/hNmzISsr0pGRFmetGCoz6RV8BtDptaEyAJ2i4A2GuO+srpyYFY8/rA2VlU+uLvOr7HOWMf/AsNmuA8Nmp52Qwtnd0+krw2aikdQ6Eerbty+KoqAoSrVDYFarlZdffrlegxNCiBbNbIaXXtKW1V93Hfzyi7aqbMYMuPDCCAenDZV5DkyoDoRCBIJU6hGyGvR0TLUfWHpfWSCkMm9dLtN/2F7RtiGnlM15pShARrxFeoZEo6j1HKEdO3awbds2VFXl119/ZceOHRW3vXv34nQ6ue666xoyVr7//nvOP/98WrVqhaIofPbZZ0d9zOLFiznxxBMxm8106tSJGTNmNGiMQghR7y66CFatgpNOguJiGDNGS5CaAJvNSozNhtlkwmhU0BkU9AYFRa/Qr7UdJayihtRK/7rLQvyZ72HJZm1k4aJ+GXx9x2DWPnwaqyafzm2ntSNWilKLRlLrHqGsA12x4XDVzL6xuN1u+vTpw3XXXcfFF1981PN37NjBueeeyy233MLMmTNZuHAhN9xwAxkZGYwaNaoRIhZCiHrSrh388AM89JBWe+jccyMdUSVJDjPF7hCh0ME9ZY0GBXN57SEFwqr2ry8cwB0IMaJrKrNubk/lnEclABR4w2zbmSNzh0SDO6bK0lu2bGHRokXk5uZWSYweffTRegvuSBRF4dNPP2XMmDE1nvPAAw/w5Zdfsm7duoq2K664guLiYr755ptaXUcqSwshmpycHEhLO3j8xx/QrVvk4jmEx+MjTKhiYnSJV5tQfWgi5PKHURRIijFWDKWBVnPI45O5Q6J+NFhl6X//+9/ceuutJCcnk56eXmmzVUVRGi0Rqo1ly5YxcuTISm2jRo3i7rvvjkxAQghRHw5NghYu1PYqu/VWbUK1xRK5uACbzVzp2BXwEgqHUNESHRWwW7T91Lz+I88dumFoFlcPbEuSVQ96HQYFqVAt6l2dE6G///3vPPnkkzzwwAMNEU+9ys7OJu3QXxhAWloaTqezxh8mn8+Hz+erOHY6nQ0epxBCHLPfftOWYr36KixdCh9+CJ07RzqqCmlxVopcHuDgqjGDAu5ACA7JgxQFnGVh9pWU0SXNxvpHR1RZbq+qUOwL86sMmYl6VOdEqKioiMsuu6whYmkSpk2bxtSpUyMdhhBC1M7990Pv3vCXv8Dq1XDiifDGGzBuXKQjq5Dg0FaXgQqKNnpgNlT++FEUMAYDtE600LdtzKE5EgoQCEuVatEw6lxZ+rLLLmP+/PkNEUu9S09PJycnp1JbTk4OsbGxNXatTp48mZKSkorb7t27GyNUIYQ4dmefrSVBp50GLpe2d9lNN0ETqtxss1mx2WzYbDasViv+YBCfv/ItEFSxGPS4fWFKD7k5fVWrVHv9If7IKeX1JduZtyGbnJKm81pFdKlzj1CnTp3429/+xs8//0yvXr0wHlbl9M4776y34I7X4MGD+eqrryq1LViwgMGDB9f4GLPZjNlsrvF+IYRoklq31uYLPf44/P3v2sqyM86AsWMjHVm1FJ2BkBqq1Jbk0BMIqZWGxBQF/IGDVaqrs2RjHqd0TJYhMnFM6rxqrH379jU/maKwffv2Gu8/Xi6Xi61btwLQr18/XnjhBUaMGEFiYiJt27Zl8uTJ7N27l3fffRfQls/37NmTiRMnct111/Hdd99x55138uWXX9Z6+bysGhNCRJ1vv4Uvv4QXXji4lr0Jcno8HPoBZFIUfOHKIZcnQit2FXHXrNX4gipndU/lgbO6kGo3ojMcqGQNMolaVFLbz+9jWj4fKYsXL2bEiBFV2q+55hpmzJjBtddey59//snixYsrPeaee+5hw4YNtGnThr/97W9ce+21tb6mJEJCiKhXWAhPPQVTp4LdHuloKqmYO3RAqT9UY4/Q6t2FTDglq9qhDE8QClwBsp1l+EIq8TYjXVJjiLdLZcaWqlkmQpEgiZAQIupddBF89plWa+jDD6Fnz0hHVKOcEm25/eEsRh0mHRVJ0qHJ0l6nnz1FPr7blMv3W/JweYN0TY9hSKckzuqeTpd0+d3dEjVYHaGjbaPx1ltv1fUphRBCNKS77tL2KfvjD22bjpdf1vYua4LDZmlxVpweT+VeIcAdgLJq9iwDyCkN8M36bGav2FPRtnR7IdlOH0adjtQYi/QMiRrVedVYUVFRpVtubi7fffcdc+bMobi4uAFCFEIIcVyGD9dWlY0apa0ku+EGbbl9aWmkI6tWrM2GHgU9oEf7oDLoQpj0Kia9itlw8La/qIy9RR6Wbsuv8jx5pT5+31PMjnx3Y78EEUXq3CP06aefVmkLh8PceuutdOzYsV6CEkIIUc9SU+Grr+CZZ+CRR2DmTFi+HP73P+jSJdLRVWGzVZ74HDqkFMChvUVhxYfHHyKnVCuE+9/r+tM3w47eYKDp9XeJpqje5ght2rSJ4cOHs3///vp4uiZD5ggJIZqdH3/UCi7q9dqu9glNvxhhQWkp5Z9Wh47o/ZnnY2u+mwFt48g4wpb13hAUe4LsKfJSFlSJtRrokhZDkkPKpTRXDTZHqCbbtm0jGAzW19MJIYRoKKeeqg2V7d9/MAlSVfB4mtyqsnJhjARCVT9jMhIsdE63HfHDLNsVYH+Jn4Ubc1m8OZfCUj+dUh2c0jmJUT0y6N4qruECF01enROhSZMmVTpWVZX9+/fz5Zdfcs0119RbYEIIIRpQUpJ2K/fvf2vDZrNnQ//+kYurBikxFjyH1B06dNir2Ff9JOqK+70h5m/I5p1luyraVuwqZneRF52iIy3WIj1DLVidE6FVq1ZVOtbpdKSkpPD8888fdUWZEEKIJigYhP/7P9i2DYYMgeeeg9tvb3Krymw2Gx5PGZV2awUsBrXaZfUKsKPAz96SMr7fnFfl+ZzeAL/tLGRwhyRJhFqwOidCixYtaog4hBBCRIrBoO1cf911Wr2hO++ERYvgP/9pcvOHbDZLlTbV6602EQJAF8DrD7KvpAyAObecROckKwaDvkHjFNHjmCdL5+XlsWnTJgC6du1KSkpKvQbWVMhkaSFEi6GqWo2he++FQADatYNZs2DQoEhHdkSFpaVH7BHaXVJG/8w44i1HrxhT7AuzJcdNWVDFZtLTJc0he5hFqQarLO12u7njjjt49913CYe17km9Xs/48eN5+eWXsdlsxxd5EyOJkBCixVmxQtusdft2MJthxw7IyIh0VDWqqRp1uXjz0RMgFdic52XeulwWb85mZ5GPdgk2hp2Qwqju6fTLalo9Y+LoGmzV2KRJk1iyZAn/+9//OOWUUwD48ccfufPOO/nrX//Ka6+9duxRCyGEiLwBA2DlSq3wYq9eTToJAq0atcfjqfH+o02mLjdvXS6v/3Bw4/A/ckr5I0crOtkq3iI9Q81UnXuEkpOT+fjjjxk+fHil9kWLFnH55ZeTl1d1Qlo0kx4hIUSLparaTXegR2XzZsjP1yZUN0GHb+BaLhgKHbG4ok+F9fs8TPt6AxtzqiZU3dJieOjcbgzt0jyngDRXDdYj5PF4SEtLq9Kempp6xIxcCCFElFGUgyvHvF647DJYvx6efBLuu+9ggtREHF6Nupz3kKrUNfEGQ+woqP68P4s8ePw1D72J6Fbnd/HgwYN57LHHKCsrq2jzer1MnTqVwYMH12twQgghmohQCHr00P598EE47zyIkhGAQDBI8Ag3XyCI1aCnfVL1iVS7BBs2k6wya67q3CP00ksvMWrUKNq0aUOfPn0A+P3337FYLMybN6/eAxRCCNEEOBza/mQjRmjL67/+Gvr2hQ8+gNNOi3R0R+QNG444mRqgc5qd4V3S2ZiznUfO7crYvmkYDPW2+YJowo5p+bzH42HmzJls3LgRgG7dunHVVVdhtTa/iWQyR0gIIQ6zdi1cfjls3KgNj02dCpMna3uXNVFHm7qhAmVBsNYy95Fl9k1fgy2fb2kkERJCiGq4XDBxIrz7LgweDEuWgNEY6aiOqKbJ1HXhV2FHviyzjwYNNll62rRppKWlVdlO46233iIvL48HHnig7tEKIYSILg4HvPMOnHEGDBvW5JMgqHkyNRy9FtGhZJl981LnydLTp0/nhBNOqNLeo0cPXn/99XoJSgghRJQYPx6ysg4eP/ggPPaYNqk6ith0QWIM6hFvSlhlS46bxZuzq32OJRvz2JzjauTIxfGqc49QdnY2GdUU10pJSWH//v31EpQQQogotGYNPP209v/vv9cmV7dqFdmYaqk2E6PNyDL75qjOPUKZmZn89NNPVdp/+uknWkXJG14IIUQD6N0b3nsP7HZYvFhbVRYlq4mPtLy+Ypm9X5bZN0d1ToRuvPFG7r77bt5++2127tzJzp07eeutt7jnnnu48cYbGyJGIYQQ0eKqq7TtOfr00eoMnX02PPQQBIORjuyIPGEDpUHliDdVp1Qss6/OsBNS6JLmaOTIxfGq86oxVVV58MEH+ec//4nf7wfAYrHwwAMP8OijjzZIkJEkq8aEEOIYeL0waRKUzx0dPRq+/PJgpeomqDa7Ixy6auzifmlkxFgwGA6+puZYRiZaNfjyeZfLxR9//IHVaqVz586YzeZjDrYpk0RICCGOw4cfapu3vvEGXHFFpKM5qvpYYl8dqTvU+KSOUD2RREgIIY5TXh6kHLJh6aZN0KFDVCy5r05d99XclCd1hyKhweoICSGEEHVyaBKUkwPDh0O7djBrVuWl91FAqzcUrtNjpO5Q09a0tg4WQgjRvG3apM0f+vlnbVXZ559HOqI6qU29ofKb1SB1h6KBJEJCCCEaz2mnwapVcNJJUFwMY8bA3XfDgcU3TZ3BYKj1zWwwSN2hKCCJkBBCiMbVvj388IO2qgzgpZfglFNg+/YjP64JqE29oYq6Q0GpOxQNJBESQgjR+EwmeP55+OILSEiAFSvgH/+IdFRHVZt6Q+U3b1DqDkUDmSwthBAics4/H1avhocf1hKjJi4tzlrnVWOjeqYCVFk1NrpXukyUbgJk+fxRyPJ5IYRoZKqqbd56ww3QuXOko6lWXesNSR2hxlfbz++oGxr717/+Rbt27bBYLAwaNIhff/21xnNnzJiBoiiVbhaLpRGjFUIIUWevvgrPPAMnnqgtsW+CbDYrNput1rdWCQ6GnZDGqJ7pDO2SIklQExJVidDs2bOZNGkSjz32GCtXrqRPnz6MGjWK3NzcGh8TGxvL/v37K247d+5sxIiFEELU2ZgxMHQouFwwbhzcfLO25F6IBhBVQ2ODBg1i4MCBvPLKKwCEw2EyMzO54447ePDBB6ucP2PGDO6++26Ki4uP+ZoyNCaEEBEQDMLUqfDkk9pQWa9e2nYdJ5wQ6ciOybFu3SFDaseu2Q2N+f1+fvvtN0aOHFnRptPpGDlyJMuWLavxcS6Xi6ysLDIzM7nwwgtZv379Ea/j8/lwOp2VbkIIIRqZwQBPPAHz5kFqKqxdCwMGwCefRDqyOtMmV9c9CdqU5+XdpbuZ9vUG7v5wNU99+QdvL/2TVTuL6j/IFixqVo3l5+cTCoVIS0ur1J6WlsbGjRurfUzXrl1566236N27NyUlJTz33HMMGTKE9evX06ZNm2ofM23aNKZOnVrv8QshhDgGZ56prSq76iqt9lDr1pGOqE6OZUuOcrI1R+OImh6hYzF48GDGjx9P3759GTZsGHPmzCElJYXp06fX+JjJkydTUlJScdu9e3cjRiyEEKKKjAxYsACWLIGTTz7Y7mr621PUZUuOQ2+yNUfjiZoeoeTkZPR6PTk5OZXac3JySE+vvljV4YxGI/369WPr1q01nmM2mzGbzccVqxBCiHqm18OQIQeP167VNm999lmYMAEUJWKhHYnBcGwfs7I1R+OJmh4hk8lE//79WbhwYUVbOBxm4cKFDB48uFbPEQqFWLt2LRkZGQ0VphBCiMbw+utQWAjXXw/jxzfZ3qG6bMlx6E225mg8UZMIAUyaNIl///vfvPPOO/zxxx/ceuutuN1uJkyYAMD48eOZPHlyxfmPP/448+fPZ/v27axcuZKrr76anTt3csMNN0TqJQghhKgPL78MTz2l9RS99x707w+//x7pqKqoy5Ych95ka47GEzVDYwBjx44lLy+PRx99lOzsbPr27cs333xTMYF6165d6HQHc7uioiJuvPFGsrOzSUhIoH///ixdupTu3btH6iUIIYSoDzodTJ6s1Ru64grYvBkGDdI2cL3ppiYzVHYsW3KUk605GkdU1RGKBKkjJIQQTVx+PlxzDXz1lXY8ezZcfnlkYzqM1BFqfLX9/I6qHiEhhBCiiuRk+N//4IUXtNVll1wS6YiqsNmOLXmx2aBVggyDNaSomiMkhBBCVEung3vvha+/1uYNAZSVafOHZOBDHIEkQkIIIZqPQ+aJMmkS/OUvcOmlcBxbLYnmTRIhIYQQzVPXrmA0wpw50K8f/PprpCMSTZAkQkIIIZqnu+6Cn36C9u3hzz/h1FPh//5PhspEJZIICSGEaL4GDoSVK7UJ1IGANlw2ZoxWjFEIZNWYEEKI5i4+Hj76CF57De65B77/HkpLITEx0pEd1bEuu6+OLMWvniRCQgghmj9Fgdtug8GDITcXsrIO3qeqTaYA46GOtRBjdTbleZm3LrdKccZR3dPpl5VQb9eJRpIICSGEaDn69at8/OWX8OqrMGMGpKREJKTq5JR4CYXD9fZ889bl8voP2yuO/8gp5Y+cUgBaxVtadM+QJEJCCCFaJr8fbrkF9uyBvn3hgw/gtNMiHRUANl2w3mbxrtzrYfHm7GrvW7Ixj1M6JrfoREgmSwshhGiZTCatR6hrV9i3D0aMgCefhHrsiTlWBoOh3m7eYIgdBd5qr/NnkQePP9TIr65pkURICCFEy9W7N6xYoRVeDIfhkUfg7LMhJyeiYQWDwXq7WQ162idV3+PTLsGGzaRv5FfXtEgiJIQQomVzOOCdd+Ctt8Bq1fYr69tXm1QdIZ6wgdKgUi+3zml2hndJr/Y6w05IoUtay97LTOYICSGEEIoCEybAoEFw2WUwZAikpkYsnLQ4a72uGhvVU3sth68aG90rvUXPDwJQVFVKbB6J0+kkLi6OkpISYmNjIx2OEEKIhubxaImR9UCCkJsLoRBkZEQgFKkjdKxq+/ktPUJCCCHEoWy2g/8Ph+Hqq+H337Wd7M88s5FDqb9ExWaDVgktexisOjJHSAghhKhJfr42cTo3F0aN0iZTB4ORjkrUI0mEhBBCiJqkpsLPP8PNN2sVqJ98Ek4/Xas9JJoFSYSEEEKII7Fa4fXXtYKLMTHwww/aqrKvvop0ZKIeSCIkhBBC1MYVV2g72Z94IhQUaBu4BgKRjkocJ0mEhBBCiNrq1AmWLoW77oJZs8BojHRE4jjJqjEhhBCiLsxmePHFym3//rc2n+jCCyMSkjh20iMkhBBCHI81a2DiRBgzRhsu8/sjHZGoA0mEhBBCiONxwglw++3a/198EU49FXbsiGhIovYkERJCCCGOh8kEL7wAn38OCQmwfDn06wdz5kQ6MlELkggJIYQQ9eGCC2DVKhg8GEpK4JJLYNKkSEcljkISISGEEKK+ZGXBkiVw//3acQT2JxN1I6vGhBBCiPpkNMLTT8NFF8FJJx1sLy3VCjKKJkV6hIQQQoiGcPLJoDvwMet2w6BBcMst4PVGNi5RiSRCQgghREObPx82boTp07UEadOmSEckDpBESAghhGhoF10E8+ZpRRfXrIH+/eG99yIdlSAKE6F//etftGvXDovFwqBBg/j111+PeP5HH33ECSecgMVioVevXnwlm+QJIYSIhDPPhNWrYcQIbajsL3+B667T/t8IPN4AHo+nydz2FblYsjGHeeuy+WFzHjklkRkyjKpEaPbs2UyaNInHHnuMlStX0qdPH0aNGkVubm615y9dupRx48Zx/fXXs2rVKsaMGcOYMWNYt25dI0cuhBBCoK0iW7AApkwBRYG334a7727wy3o8HlCbzgaxm/K8vLt0N9O+3sDdH67mqS//4O2lf7JqZ1Gjx6Koqqo2+lWP0aBBgxg4cCCvvPIKAOFwmMzMTO644w4efPDBKuePHTsWt9vN3LlzK9pOPvlk+vbty+uvv16razqdTuLi4igpKSE2NrZ+XogQQgixaJGWBH39NbRq1WCXcXkDOMt8Dfb8x+Ldpbt5/YftVdpvGdaBCUPakRZnPe5r1PbzO2qWz/v9fn777TcmT55c0abT6Rg5ciTLli2r9jHLli1j0mHFrEaNGsVnn31W43V8Ph8+38E3jNPpPL7AhRBCiOqMGKENlSnKwbb33tP2LHM46u0yarCMmCb0ab9yr4fFm7OrvW/JxjxO6ZhcL4lQbTWhL82R5efnEwqFSEtLq9SelpbGxo0bq31MdnZ2tednZ1f/DQCYNm0aU6dOPf6AhRBCiKM5NAn6+GNt3lDXrvDhh9C7d71cwmBoWh/13mCIHQXVzwf6s8iDxx9q1Hiiao5QY5g8eTIlJSUVt927d0c6JCGEEC1BWhq0bq0trR80CN54A+ph9kowGGxSN6tBT/uk6nt82iXYsJn0x/2a66JppYlHkJycjF6vJycnp1J7Tk4O6enp1T4mPT29TucDmM1mzGbz8QcshBBC1MXQodpQ2fjx2ryhm2/W5hFNnw7HMUdVMVia1Byhzml2hndJZ2NO1TlCw05IoUta/Q0L1kbU9AiZTCb69+/PwoULK9rC4TALFy5k8ODB1T5m8ODBlc4HWLBgQY3nCyGEEBGVnAxz58Izz4BeD7NmaTWHVq065qd0WI3Em3VN6jaqZyq3DO3ACWk2rCY93dJiuGVYB0b3Sm/U+UEQRT1CAJMmTeKaa65hwIABnHTSSbz44ou43W4mTJgAwPjx42ndujXTpk0D4K677mLYsGE8//zznHvuucyaNYsVK1bwxhtvRPJlCCGEEDXT6eC+++DUU+GKK2DrVti5E/r1O+antNlseLyBJrOEvmuKlbQhmQzumEhZUMVm0tMlzdHoSRBEWSI0duxY8vLyePTRR8nOzqZv37588803FROid+3ahU53sJNryJAhvP/++zzyyCM89NBDdO7cmc8++4yePXtG6iUIIYQQtTN4sNYT9Nln2kqycqpaeZJ1LdmsRsBYX9EdN5sNWiU07jBYdaKqjlAkSB0hIYQQTcaePXDxxfCvf8HAgZGOpkmr7ed31MwREkIIIVq8Bx+E5cvhlFPgxRfrZVVZSyeJkBBCCBEtXnlF6xEKBOCee7Qhs8LCSEcV1SQREkIIIaJFfLxWePGVV8Bkgi++0CZR17DDgjg6SYSEEEKIaKIoMHGilvx07Ai7dsFpp8E330Q6sqgkiZAQQggRjU48EVauhLFjtW05Tjst0hFFJUmEhBBCiGgVGwsffADff6+tRwcIh7UK1aJWJBESQgghopmiQGLiweOnn4YBA+Cpp7SkSByRJEJCCCFEc6GqsHkzhELw8MMwejTk5kY6qiZNEiEhhBCiuVAUeOst+M9/wGqF+fOhb19YvDjSkTVZkggJIYQQzYmiwHXXaYUXu3eH/fvhjDNg6lStp0hUIomQEEII0Rz16AG//goTJmhzhZ56Shs2E5VE1aarQgghhKgDu10bKhsxAsrKoFu3SEfU5EgiJIQQQjR3f/lL5ePffoPPP4dHHwVDy04FWvarF0IIIVqasjKtCOO2bdok6g8+gNatIx1VxMgcISGEEKIlsVjg738HhwN++EFbVfb115GOKmIkERJCCCFamiuu0Lbn6NcP8vPhnHPggQe0Xe1bGEmEhBBCiJaoc2dYulTbwBXgmWdg2DAoKopsXI1MEiEhhBCipbJY4JVX4KOPtH3L4uMhLi7SUTUqmSwthBBCtHSXXqrtZh8TA7oDfSReL+j1YDJFNrYGJj1CQgghhIAOHSAl5eDxxIkwdCjs2BG5mBqBJEJCCCGEqGzvXvjsM60ydb9+MGdOpCNqMJIICSGEEKKy1q1h1So4+WQoKYFLLoE77gCfL9KR1TtJhIQQQghRVVYWfP893HefdvzKKzBkCGzdGtm46pkkQkIIIYSontGoLaufOxeSkrTaQ2efDcFgpCOrN5IICSGEEOLIzj0XVq+G007Teoaa0f5kzeeVCCGEEKLhtGmj7U2mKAfbFiyAtm2ha9eIhXW8pEdICCGEELVzaBK0Ywdcdhn07w8zZ0YupuMkiZAQQggh6s5i0ZbWu91w9dVwww3g8UQ6qjqTREgIIYQQdZeRAd9+C489pvUU/ec/cNJJsGFDpCOrE0mEhBBCCHFs9HqYMkVLiNLTYf16GDgQZsyIdGS1JomQEEIIIY7P6adrq8pGjtSGx1aujHREtRY1iVBhYSFXXXUVsbGxxMfHc/311+NyuY74mOHDh6MoSqXbLbfc0kgRCyGEEC1IWhp88w1Mnw7PPnuwXVUjF1MtRE0idNVVV7F+/XoWLFjA3Llz+f7777npppuO+rgbb7yR/fv3V9yeeeaZRohWCCGEaIH0erjpJjCbteNgEM45B/797yabEEVFHaE//viDb775huXLlzNgwAAAXn75Zc455xyee+45WrVqVeNjbTYb6enpjRWqEEIIIcq9/77WS/TNN7BokdZbFBMT6agqiYoeoWXLlhEfH1+RBAGMHDkSnU7HL7/8csTHzpw5k+TkZHr27MnkyZPxROHSPiGEECIqXX01PP201lP0wQdw4onaZq5NSFT0CGVnZ5OamlqpzWAwkJiYSHZ2do2Pu/LKK8nKyqJVq1asWbOGBx54gE2bNjFnzpwaH+Pz+fAdsruu0+k8/hcghBBCtEQ6Hdx/P5x6KlxxhbZh68knw//9H9x6a+UCjZEKMZIXf/DBB6tMZj78tnHjxmN+/ptuuolRo0bRq1cvrrrqKt59910+/fRTtm3bVuNjpk2bRlxcXMUtMzPzmK8vhBBCCLRd61etgvPPB78fJk6EBx+MdFQAKKoaudlLeXl5FBQUHPGcDh068N577/HXv/6VoqKiivZgMIjFYuGjjz7ioosuqtX13G43DoeDb775hlGjRlV7TnU9QpmZmZSUlBAbG1ur6wghhBCiGqoKL74IU6fCTz9Bjx4Ndimn00lcXNxRP78jOjSWkpJCSkrKUc8bPHgwxcXF/Pbbb/Tv3x+A7777jnA4zKBBg2p9vdWrVwOQkZFR4zlmsxlz+Wx3IYQQQtQfRYF77oEJEyA+PtLRwP+3d+9BUZVvHMC/i7K7iIAXRHEkU0BAkgyNQjM0L9t4GZnRzHKIpjQ1CClLcaqxpulnZumUkdoNiJrwit0jFBElTQTXEJUU0DQVxtEQRRHh+f3RcMbFXWCJZTns9zOzf3DO857zPPt22sdz3gWoZLF0UFAQHn30UcybNw8HDhxAbm4uYmNjMXv2bOUbY3///TcCAwNx4MABAEBJSQneeust5Ofn49SpU/juu+/w1FNP4eGHH0ZISIg9yyEiInJsHaQJAlTSCAH/fvsrMDAQ48ePx+TJk/HQQw/hk08+UfbX1taiuLhY+VaYVqvFjh07MGnSJAQGBmLx4sWYMWMGvv/+e3uVQERERB2MXdcIqUFLnzESERFRx9HSz2/V3BEiIiIiamtshIiIiMhhsREiIiIih8VGiIiIiBwWGyEiIiJyWGyEiIiIyGGxESIiIiKHxUaIiIiIHBYbISIiInJYbISIiIjIYbERIiIiIofFRoiIiIgcFhshIiIiclhd7Z1ARyciAP79K7ZERESkDg2f2w2f45awEWpGVVUVAMDHx8fOmRAREZG1qqqq4OHhYXG/RpprlRxcfX09zp07Bzc3N2g0mjY77pUrV+Dj44MzZ87A3d29zY7bkbDGzoE1dg6dvcbOXh/AGq0lIqiqqkL//v3h5GR5JRDvCDXDyckJAwYMsNnx3d3dO+1/0A1YY+fAGjuHzl5jZ68PYI3WaOpOUAMuliYiIiKHxUaIiIiIHBYbITvR6XRYvnw5dDqdvVOxGdbYObDGzqGz19jZ6wNYo61wsTQRERE5LN4RIiIiIofFRoiIiIgcFhshIiIiclhshIiIiMhhsRGyocTERNx9993Q6/V44IEHcODAgSbjN2/ejMDAQOj1egwbNgw//fRTO2XaetbUmJycDI1GY/LS6/XtmK31cnJyMG3aNPTv3x8ajQbbt29vdkx2djZCQ0Oh0+ng5+eH5ORkm+fZWtbWl52dfcccajQaXLhwoX0SboUVK1bg/vvvh5ubG7y8vBAZGYni4uJmx6npemxNjWq7HtetW4eQkBDlF+2Fh4fj559/bnKMmubQ2vrUNn/mvPPOO9BoNIiPj28yztbzyEbIRjZu3IiXXnoJy5cvR0FBAe69914YDAZUVFSYjf/tt9/wxBNP4Nlnn8WhQ4cQGRmJyMhIHDlypJ0zbzlrawT+/W2h58+fV16nT59ux4ytd+3aNdx7771ITExsUXxZWRmmTJmCcePGwWg0Ij4+HnPnzkVGRoaNM20da+trUFxcbDKPXl5eNsrwv9u9ezdiYmKwf/9+ZGZmora2FpMmTcK1a9csjlHb9diaGgF1XY8DBgzAO++8g/z8fBw8eBCPPPIIpk+fjqKiIrPxaptDa+sD1DV/jeXl5WHDhg0ICQlpMq5d5lHIJsLCwiQmJkb5ua6uTvr37y8rVqwwGz9r1iyZMmWKybYHHnhA5s+fb9M8/wtra0xKShIPD492yq7tAZD09PQmY5YsWSLBwcEm2x5//HExGAw2zKxttKS+Xbt2CQC5fPlyu+RkCxUVFQJAdu/ebTFGjdfj7VpSo9qvRxGRnj17ymeffWZ2n9rnUKTp+tQ8f1VVVeLv7y+ZmZkSEREhixYtshjbHvPIO0I2cPPmTeTn52PChAnKNicnJ0yYMAH79u0zO2bfvn0m8QBgMBgsxttba2oEgKtXr2LgwIHw8fFp9l87aqS2eWyt4cOHw9vbGxMnTkRubq6907FKZWUlAKBXr14WY9Q+jy2pEVDv9VhXV4e0tDRcu3YN4eHhZmPUPIctqQ9Q7/zFxMRgypQpd8yPOe0xj2yEbODixYuoq6tD3759Tbb37dvX4lqKCxcuWBVvb62pMSAgAF988QW+/fZbfPXVV6ivr8eoUaNw9uzZ9ki5XViaxytXruD69et2yqrteHt7Y/369di6dSu2bt0KHx8fjB07FgUFBfZOrUXq6+sRHx+P0aNH45577rEYp7br8XYtrVGN12NhYSG6d+8OnU6HBQsWID09HUOHDjUbq8Y5tKY+Nc4fAKSlpaGgoAArVqxoUXx7zCP/+jy1m/DwcJN/3YwaNQpBQUHYsGED3nrrLTtmRi0VEBCAgIAA5edRo0ahpKQEa9asQWpqqh0za5mYmBgcOXIEe/futXcqNtPSGtV4PQYEBMBoNKKyshJbtmxBdHQ0du/ebbFZUBtr6lPj/J05cwaLFi1CZmZmh1rYzUbIBjw9PdGlSxeUl5ebbC8vL0e/fv3MjunXr59V8fbWmhobc3Z2xn333YeTJ0/aIkW7sDSP7u7ucHFxsVNWthUWFqaKxiI2NhY//PADcnJyMGDAgCZj1XY9NrCmxsbUcD1qtVr4+fkBAEaMGIG8vDx88MEH2LBhwx2xapxDa+prTA3zl5+fj4qKCoSGhirb6urqkJOTg48++gg1NTXo0qWLyZj2mEc+GrMBrVaLESNGYOfOncq2+vp67Ny50+Lz3vDwcJN4AMjMzGzy+bA9tabGxurq6lBYWAhvb29bpdnu1DaPbcFoNHboORQRxMbGIj09HVlZWRg0aFCzY9Q2j62psTE1Xo/19fWoqakxu09tc2hOU/U1pob5Gz9+PAoLC2E0GpXXyJEjMWfOHBiNxjuaIKCd5rHNll2TibS0NNHpdJKcnCxHjx6V5557Tnr06CEXLlwQEZGoqChJSEhQ4nNzc6Vr167y3nvvybFjx2T58uXi7OwshYWF9iqhWdbW+Oabb0pGRoaUlJRIfn6+zJ49W/R6vRQVFdmrhGZVVVXJoUOH5NChQwJAVq9eLYcOHZLTp0+LiEhCQoJERUUp8aWlpdKtWzd55ZVX5NixY5KYmChdunSRX375xV4lNMna+tasWSPbt2+XEydOSGFhoSxatEicnJxkx44d9iqhWQsXLhQPDw/Jzs6W8+fPK6/q6molRu3XY2tqVNv1mJCQILt375aysjL5448/JCEhQTQajfz6668iov45tLY+tc2fJY2/NWaPeWQjZENr166Vu+66S7RarYSFhcn+/fuVfRERERIdHW0Sv2nTJhkyZIhotVoJDg6WH3/8sZ0ztp41NcbHxyuxffv2lcmTJ0tBQYEdsm65hq+LN3411BUdHS0RERF3jBk+fLhotVoZPHiwJCUltXveLWVtfStXrhRfX1/R6/XSq1cvGTt2rGRlZdkn+RYyVx8Ak3lR+/XYmhrVdj0+88wzMnDgQNFqtdKnTx8ZP3680iSIqH8Ora1PbfNnSeNGyB7zqBERabv7S0RERETqwTVCRERE5LDYCBEREZHDYiNEREREDouNEBERETksNkJERETksNgIERERkcNiI0REREQOi40QEdnc008/jcjISKvHff7555g0adJ/Pk5npNFosH379hbHJyQk4IUXXrBdQkQqxUaIiDqkGzdu4PXXX8fy5cttfp6nn34aw4YNQ9euXS02WtnZ2QgNDYVOp4Ofnx+Sk5Ntmldbe/nll5GSkoLS0lJ7p0LUobARIiKzbt68adfzb9myBe7u7hg9erRNz1NXVwcXFxfExcVhwoQJZmPKysowZcoUjBs3DkajEfHx8Zg7dy4yMjJsmltb8vT0hMFgwLp16+ydClGHwkaIiAAAY8eORWxsLOLj45UPTQBYvXo1hg0bBldXV/j4+OD555/H1atXlXHJycno0aMHMjIyEBQUhO7du+PRRx/F+fPnLZ4rLy8Pffr0wcqVKy3GpKWlYdq0aU3mXFNTg7i4OHh5eUGv1+Ohhx5CXl6eScx3330Hf39/6PV6jBs3DikpKdBoNPjnn38AAK6urli3bh3mzZuHfv36mT3P+vXrMWjQILz//vsICgpCbGwsZs6ciTVr1ljM7fTp05g2bRp69uwJV1dXBAcH46efflL2FxUVYerUqXB3d4ebmxvGjBmDkpIS5f2ZOHEiPD094eHhgYiICBQUFDT5Xpw5cwazZs1Cjx490KtXL0yfPh2nTp0yiZk2bRrS0tKaPA6Ro2EjRESKlJQUaLVa5ObmYv369QAAJycnfPjhhygqKkJKSgqysrKwZMkSk3HV1dV47733kJqaipycHPz11194+eWXzZ4jKysLEydOxNtvv42lS5dazGXv3r0YOXJkk/kuWbIEW7duRUpKCgoKCuDn5weDwYBLly4B+PdOzsyZMxEZGYnDhw9j/vz5ePXVV615SwAA+/btu+NukcFgwL59+yyOiYmJQU1NDXJyclBYWIiVK1eie/fuAIC///4bDz/8MHQ6HbKyspCfn49nnnkGt27dAgBUVVUhOjoae/fuxf79++Hv74/JkyejqqrK7Llqa2thMBjg5uaGPXv2IDc3V2lIb7+zFxYWhrNnz97RIBE5tDb9E65EpFoRERFy3333NRu3efNm6d27t/JzUlKSAJCTJ08q2xITE6Vv377Kz9HR0TJ9+nTZtm2bdO/eXdLS0po8x+XLlwWA5OTkmGxvOI6IyNWrV8XZ2Vm+/vprZf/Nmzelf//+8u6774qIyNKlS+Wee+4xOcarr74qAOTy5ct3nPf249/O399f/ve//5ls+/HHHwWAVFdXm61h2LBh8sYbb5jdt2zZMhk0aJDcvHnT7P7G6urqxM3NTb7//ntlGwBJT08XEZHU1FQJCAiQ+vp6ZX9NTY24uLhIRkaGsq2yslIASHZ2dovOS+QIutqzCSOijmXEiBF3bNuxYwdWrFiB48eP48qVK7h16xZu3LiB6upqdOvWDQDQrVs3+Pr6KmO8vb1RUVFhcpzff/8dP/zwA7Zs2dLsN7+uX78OANDr9RZjSkpKUFtba7KGyNnZGWFhYTh27BgAoLi4GPfff7/JuLCwsCbP3Vbi4uKwcOFC/Prrr5gwYQJmzJiBkJAQAIDRaMSYMWPg7Oxsdmx5eTlee+01ZGdno6KiAnV1daiursZff/1lNv7w4cM4efIk3NzcTLbfuHFDedwGAC4uLgD+vYNHRP/iozEiUri6upr8fOrUKUydOhUhISHYunUr8vPzkZiYCMB0MXXjD3SNRgMRMdnm6+uLwMBAfPHFF6itrW0yj969e0Oj0eDy5cv/pZw2069fP5SXl5tsKy8vh7u7u9JcNDZ37lyUlpYiKioKhYWFGDlyJNauXQsAFsc0iI6OhtFoxAcffIDffvsNRqMRvXv3triA/erVqxgxYgSMRqPJ688//8STTz6pxDU8MuzTp0+Layfq7NgIEZFF+fn5qK+vx/vvv48HH3wQQ4YMwblz51p1LE9PT2RlZeHkyZOYNWtWk82QVqvF0KFDcfToUYsxvr6+ynqmBrW1tcjLy8PQoUMBAAEBATh48KDJuMaLqVsiPDwcO3fuNNmWmZmJ8PDwJsf5+PhgwYIF2LZtGxYvXoxPP/0UABASEoI9e/ZYfA9yc3MRFxeHyZMnIzg4GDqdDhcvXrR4ntDQUJw4cQJeXl7w8/MzeXl4eChxR44cgbOzM4KDg1taOlGnx0aIiCzy8/NDbW0t1q5di9LSUqSmpiqLqFvDy8sLWVlZOH78OJ544gllcbA5BoMBe/futbjf1dUVCxcuxCuvvIJffvkFR48exbx581BdXY1nn30WADB//nwcP34cS5cuxZ9//olNmzYpv/9Ho9Eoxzp69CiMRiMuXbqEyspK5Y5KgwULFqC0tBRLlizB8ePH8fHHH2PTpk148cUXLeYXHx+PjIwMlJWVoaCgALt27UJQUBAAIDY2FleuXMHs2bNx8OBBnDhxAqmpqSguLgYA+Pv7IzU1FceOHcPvv/+OOXPmNHkXac6cOfD09MT06dOxZ88elJWVITs7G3FxcTh79qwSt2fPHowZM6bZO1JEDsXei5SIqGOIiIiQRYsW3bF99erV4u3tLS4uLmIwGOTLL780WWyclJQkHh4eJmPS09Pl9v+9NF6EfO7cORkyZIjMmjVLbt26ZTafoqIicXFxkX/++cfica5fvy4vvPCCeHp6ik6nk9GjR8uBAwdMjvPtt9+Kn5+f6HQ6GTt2rKxbt04AyPXr15WYgQMHCoA7XrfbtWuXDB8+XLRarQwePFiSkpLM5t0gNjZWfH19RafTSZ8+fSQqKkouXryo7D98+LBMmjRJunXrJm5ubjJmzBgpKSkREZGCggIZOXKk6PV68ff3l82bN8vAgQNlzZo1ynjctlhaROT8+fPy1FNPKe/F4MGDZd68eVJZWanEBAQEyDfffNNk3kSORiPS6EE+EVEH8dhjjyE0NBTLli1rs2O+/fbbWL9+Pc6cOdNmx1SDn3/+GYsXL8Yff/yBrl35PRmiBnw0RkQd1qpVq5TfvdNaH3/8MfLy8pRHe6tWrUJ0dHQbZage165dQ1JSEpsgokZ4R4iIOrUXX3wRGzduxKVLl3DXXXchKioKy5YtY0NARADYCBEREZED46MxIiIiclhshIiIiMhhsREiIiIih8VGiIiIiBwWGyEiIiJyWGyEiIiIyGGxESIiIiKHxUaIiIiIHBYbISIiInJY/wecD6hhPBH0rwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "plot_ranked_vocab_counts(fig, ax, corpus_small, zipf=True)\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I additionally passed the argument `zipf=True` which compares the corpus' distribution to the distribution predicted by [Zipf's law](https://en.wikipedia.org/wiki/Zipf%27s_law)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Text processing: transforming documents and tokens\n",
    "\n",
    "So far we haven't modified anything in our corpus, we only investigated its contents. This will change now as we will apply several text processing methods to the contents of our corpus. But before we do that, I want you pay attention to an important detail about how a `Corpus` object behaves when it is modified."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aside: A `Corpus` object as \"state machine\"\n",
    "\n",
    "A `Corpus` object is implemented as a \"state machine\", i.e. its contents (the documents) and behavior can change when you apply a function to it. For instance, let's suppose we want to turn all tokens in a corpus to lowercase tokens. We do that by applying [to_lowercase](api.rst#tmtoolkit.corpus.to_lowercase):\n",
    "\n",
    "\n",
    "```python\n",
    "corp = Corpus({\n",
    "    \"doc1\": \"Hello world!\",\n",
    "    \"doc2\": \"Another example\"\n",
    "}, language='en')\n",
    "\n",
    "doc_tokens(corp)\n",
    "\n",
    "# Out:\n",
    "# {\n",
    "#   'doc1': ['Hello', 'world', '!'],\n",
    "#   'doc2': ['Another', 'example']\n",
    "# }\n",
    "\n",
    "to_lowercase(corp)\n",
    "doc_tokens(corp)\n",
    "\n",
    "# Out:\n",
    "# {\n",
    "#   'doc1': ['hello', 'world', '!'],\n",
    "#   'doc2': ['another', 'example']\n",
    "# }\n",
    "```\n",
    "\n",
    "As you can see, the tokens \"inside\" `corp` are changed *in place*. It's important to see that after applying the function `to_lowercase`, the tokens in `corp` were transformed and **the original tokens from before calling this method are not available anymore.** In Python, assigning a *mutable* object to a variable binds the same object only to a different name, it doesn't copy it. Since a `Corpus` object is a mutable object (you can change its contents), when we simply assign such an object to a different variable (say `corp_original`) we essentially only have two names for the same object and by applying a function to one of these variable names, the values will be changed for *both* names.\n",
    "\n",
    "#### Copying `Corpus` objects\n",
    "\n",
    "What can we do about that? There are two ways: The first is to *copy* the object which can be done with the Python [copy](https://docs.python.org/3/library/copy.html#copy.copy) function. By this, we create another variable `corpus_orig` that points to a separate `Corpus` object. The second way is to apply the corpus transformation function, e.g. `to_lowercase`, but set the parameter `inplace=False`. This will then return a *modified copy* and retain the original input corpus. The `inplace` parameter is a common corpus function parameter that is available for all functions that modify a `Corpus` object in some way. By default, it is set to `True`.\n",
    "\n",
    "We start with the first way, copying a `Corpus` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.764708Z",
     "iopub.status.busy": "2023-05-03T13:22:48.764592Z",
     "iopub.status.idle": "2023-05-03T13:22:48.809398Z",
     "shell.execute_reply": "2023-05-03T13:22:48.808889Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Trump suggests Obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterates Govt support for Finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): President Trump says he has asked the Justice Depa...\n",
      "> NewsArticles-1119 (975 tokens): An amazing moment in history : Donald Trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland sees losses widening    Bai...\n",
      "> NewsArticles-1377 (774 tokens): Turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1185 (1271 tokens): For more than a week,-France - has been rocked by ...\n",
      "> NewsArticles-1263 (410 tokens): Russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-1353 (30 tokens): Islamic State battle : Fierce gunfight outside Mos...\n",
      "> NewsArticles-1387 (513 tokens): Protests after Anaheim policeman drags teen , fire...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 9223\n"
     ]
    }
   ],
   "source": [
    "from copy import copy\n",
    "\n",
    "\n",
    "corpus_orig = copy(corpus_small)\n",
    "print_summary(corpus_orig)   # same content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.811104Z",
     "iopub.status.busy": "2023-05-03T13:22:48.810960Z",
     "iopub.status.idle": "2023-05-03T13:22:48.813839Z",
     "shell.execute_reply": "2023-05-03T13:22:48.813473Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(140357545204400, 140355946502688)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# but the different IDs confirm that we have two different objects\n",
    "id(corpus_small), id(corpus_orig)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now apply `to_lowercase` to `corpus_small`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.815446Z",
     "iopub.status.busy": "2023-05-03T13:22:48.815346Z",
     "iopub.status.idle": "2023-05-03T13:22:48.841379Z",
     "shell.execute_reply": "2023-05-03T13:22:48.840975Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): trump suggests obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): minister reiterates govt support for finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): president trump says he has asked the justice depa...\n",
      "> NewsArticles-1119 (975 tokens): an amazing moment in history : donald trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): royal bank of scotland sees losses widening    bai...\n",
      "> NewsArticles-1377 (774 tokens): turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1185 (1271 tokens): for more than a week,-france - has been rocked by ...\n",
      "> NewsArticles-1263 (410 tokens): russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-1353 (30 tokens): islamic state battle : fierce gunfight outside mos...\n",
      "> NewsArticles-1387 (513 tokens): protests after anaheim policeman drags teen , fire...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 8369\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import to_lowercase\n",
    "\n",
    "to_lowercase(corpus_small)\n",
    "print_summary(corpus_small)  # all tokens are lowercase"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The tokens of the copied original corpus remain unchanged:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.843065Z",
     "iopub.status.busy": "2023-05-03T13:22:48.842906Z",
     "iopub.status.idle": "2023-05-03T13:22:48.845887Z",
     "shell.execute_reply": "2023-05-03T13:22:48.845521Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Trump suggests Obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterates Govt support for Finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): President Trump says he has asked the Justice Depa...\n",
      "> NewsArticles-1119 (975 tokens): An amazing moment in history : Donald Trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland sees losses widening    Bai...\n",
      "> NewsArticles-1377 (774 tokens): Turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1185 (1271 tokens): For more than a week,-France - has been rocked by ...\n",
      "> NewsArticles-1263 (410 tokens): Russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-1353 (30 tokens): Islamic State battle : Fierce gunfight outside Mos...\n",
      "> NewsArticles-1387 (513 tokens): Protests after Anaheim policeman drags teen , fire...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 9223\n"
     ]
    }
   ],
   "source": [
    "print_summary(corpus_orig)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this also uses up almost twice as much computer memory now. So you shouldn't create copies that often and also release unused memory by using `del`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.847554Z",
     "iopub.status.busy": "2023-05-03T13:22:48.847312Z",
     "iopub.status.idle": "2023-05-03T13:22:48.849511Z",
     "shell.execute_reply": "2023-05-03T13:22:48.849167Z"
    }
   },
   "outputs": [],
   "source": [
    "# removing the object again\n",
    "del corpus_small"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now to the second approach. We pass the `inplace=False` parameter and get back a transformed copy of `corpus_orig` as return value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.851004Z",
     "iopub.status.busy": "2023-05-03T13:22:48.850894Z",
     "iopub.status.idle": "2023-05-03T13:22:48.920708Z",
     "shell.execute_reply": "2023-05-03T13:22:48.920163Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): trump suggests obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): minister reiterates govt support for finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): president trump says he has asked the justice depa...\n",
      "> NewsArticles-1119 (975 tokens): an amazing moment in history : donald trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): royal bank of scotland sees losses widening    bai...\n",
      "> NewsArticles-1377 (774 tokens): turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1185 (1271 tokens): for more than a week,-france - has been rocked by ...\n",
      "> NewsArticles-1263 (410 tokens): russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-1353 (30 tokens): islamic state battle : fierce gunfight outside mos...\n",
      "> NewsArticles-1387 (513 tokens): protests after anaheim policeman drags teen , fire...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 8369\n"
     ]
    }
   ],
   "source": [
    "corpus_lowercase = to_lowercase(corpus_orig, inplace=False)\n",
    "print_summary(corpus_lowercase)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, the original corpus stays unchanged:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.922796Z",
     "iopub.status.busy": "2023-05-03T13:22:48.922677Z",
     "iopub.status.idle": "2023-05-03T13:22:48.925668Z",
     "shell.execute_reply": "2023-05-03T13:22:48.925260Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Trump suggests Obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterates Govt support for Finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): President Trump says he has asked the Justice Depa...\n",
      "> NewsArticles-1119 (975 tokens): An amazing moment in history : Donald Trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland sees losses widening    Bai...\n",
      "> NewsArticles-1377 (774 tokens): Turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1185 (1271 tokens): For more than a week,-France - has been rocked by ...\n",
      "> NewsArticles-1263 (410 tokens): Russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-1353 (30 tokens): Islamic State battle : Fierce gunfight outside Mos...\n",
      "> NewsArticles-1387 (513 tokens): Protests after Anaheim policeman drags teen , fire...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 9223\n"
     ]
    }
   ],
   "source": [
    "print_summary(corpus_orig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.927520Z",
     "iopub.status.busy": "2023-05-03T13:22:48.927370Z",
     "iopub.status.idle": "2023-05-03T13:22:48.929685Z",
     "shell.execute_reply": "2023-05-03T13:22:48.929331Z"
    }
   },
   "outputs": [],
   "source": [
    "del corpus_lowercase"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're now ready to apply some common text processing steps to our corpus."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lemmatization and token normalization\n",
    "\n",
    "Lemmatization brings a token, if it is a word, to its base form. The lemma is already found out during the tokenization process and is available in the `lemma` token attribute. However, when you want to further process the tokens on the base of the lemmata, you should use the [lemmatize](api.rst#tmtoolkit.corpus.lemmatize) corpus function. This function sets the lemmata as tokens and all further processing will happen using these lemmatized tokens:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:48.931538Z",
     "iopub.status.busy": "2023-05-03T13:22:48.931371Z",
     "iopub.status.idle": "2023-05-03T13:22:49.069662Z",
     "shell.execute_reply": "2023-05-03T13:22:49.069340Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>is_punct</th>\n",
       "      <th>is_stop</th>\n",
       "      <th>lemma</th>\n",
       "      <th>like_num</th>\n",
       "      <th>pos</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>President</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>President</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>Trump</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Trump</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>say</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>say</td>\n",
       "      <td>False</td>\n",
       "      <td>VERB</td>\n",
       "      <td>VBZ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>he</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>he</td>\n",
       "      <td>False</td>\n",
       "      <td>PRON</td>\n",
       "      <td>PRP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>have</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>have</td>\n",
       "      <td>False</td>\n",
       "      <td>AUX</td>\n",
       "      <td>VBZ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59593</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>282</td>\n",
       "      <td>priority</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>priority</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NNS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59594</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>283</td>\n",
       "      <td>for</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>for</td>\n",
       "      <td>False</td>\n",
       "      <td>ADP</td>\n",
       "      <td>IN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59595</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>284</td>\n",
       "      <td>the</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>the</td>\n",
       "      <td>False</td>\n",
       "      <td>DET</td>\n",
       "      <td>DT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59596</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>285</td>\n",
       "      <td>nation</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>nation</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59597</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>286</td>\n",
       "      <td>.</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>.</td>\n",
       "      <td>False</td>\n",
       "      <td>PUNCT</td>\n",
       "      <td>.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59598 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position      token  is_punct  is_stop      lemma  \\\n",
       "0      NewsArticles-1100         0  President     False    False  President   \n",
       "1      NewsArticles-1100         1      Trump     False    False      Trump   \n",
       "2      NewsArticles-1100         2        say     False    False        say   \n",
       "3      NewsArticles-1100         3         he     False     True         he   \n",
       "4      NewsArticles-1100         4       have     False     True       have   \n",
       "...                  ...       ...        ...       ...      ...        ...   \n",
       "59593   NewsArticles-960       282   priority     False    False   priority   \n",
       "59594   NewsArticles-960       283        for     False     True        for   \n",
       "59595   NewsArticles-960       284        the     False     True        the   \n",
       "59596   NewsArticles-960       285     nation     False    False     nation   \n",
       "59597   NewsArticles-960       286          .      True    False          .   \n",
       "\n",
       "       like_num    pos  tag  \n",
       "0         False  PROPN  NNP  \n",
       "1         False  PROPN  NNP  \n",
       "2         False   VERB  VBZ  \n",
       "3         False   PRON  PRP  \n",
       "4         False    AUX  VBZ  \n",
       "...         ...    ...  ...  \n",
       "59593     False   NOUN  NNS  \n",
       "59594     False    ADP   IN  \n",
       "59595     False    DET   DT  \n",
       "59596     False   NOUN   NN  \n",
       "59597     False  PUNCT    .  \n",
       "\n",
       "[59598 rows x 9 columns]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import lemmatize\n",
    "\n",
    "# we use `inplace=False` to generate a lemmatized copy `corpus_norm`\n",
    "# of the original data; all further steps will be applied to `corpus_norm`\n",
    "corpus_norm = lemmatize(corpus_orig, inplace=False)\n",
    "tokens_table(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the `lemma` column was copied over to the `token` column."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Stemming**\n",
    "    \n",
    "tmtoolkit doesn't support stemming directly, since lemmatization is generally accepted as a better approach to bring different word forms of one word to a common base form. However, you may install [NLTK](https://www.nltk.org/) and apply stemming by using the [stem](api.rst#tmtoolkit.corpus.stem) function.\n",
    "    \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on how you further want to analyze the data, it may be necessary to \"clean\" or \"normalize\" your tokens in different ways in order to remove noise from the corpus, such as punctuation tokens or numbers, upper/lowercase forms of the same word, etc. Note that this is usually not necessary when you work with some approaches such as word embeddings (word vectors).   \n",
    "\n",
    "If you want to remove certain characters in *all* tokens in your documents, you can use [remove_chars](api.rst#tmtoolkit.corpus.remove_chars) and pass it a sequence of characters to remove. There is also a shortcut [remove_punctuation](api.rst#tmtoolkit.corpus.remove_punctuation) which will remove all punctuation characters (all characters in [string.punction](https://docs.python.org/3/library/string.html#string.punctuation) by default) *in* tokens. This means, a token like \"vs.\" will be transformed to \"vs\" and a token \",\" will be transformed to an empty token \"\". It's useful to also remove empty tokens and we will do that in a later step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.071633Z",
     "iopub.status.busy": "2023-05-03T13:22:49.071467Z",
     "iopub.status.idle": "2023-05-03T13:22:49.090761Z",
     "shell.execute_reply": "2023-05-03T13:22:49.090322Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Trump suggest Obama be ' behind ' town hall protes...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterate Govt support for Finucane inqui...\n",
      "> NewsArticles-1100 (224 tokens): President Trump say he have ask the Justice Depart...\n",
      "> NewsArticles-1119 (975 tokens): an amazing moment in history : Donald Trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland see loss widen    bail - ou...\n",
      "> NewsArticles-1377 (774 tokens): Turkey - back rebel in ' near full control ' of Al...\n",
      "> NewsArticles-1185 (1271 tokens): for more than a week,-france - have be rock by - t...\n",
      "> NewsArticles-1263 (410 tokens): russian doctor use mobile field hospital to provid...\n",
      "> NewsArticles-1353 (30 tokens): islamic State battle : fierce gunfight outside Mos...\n",
      "> NewsArticles-1387 (513 tokens): protest after Anaheim policeman drag teen , fire g...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 7287\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import remove_chars\n",
    "\n",
    "# remove only full stops \".\"\n",
    "remove_chars(corpus_norm, ['.'])\n",
    "print_summary(corpus_norm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.092517Z",
     "iopub.status.busy": "2023-05-03T13:22:49.092280Z",
     "iopub.status.idle": "2023-05-03T13:22:49.112364Z",
     "shell.execute_reply": "2023-05-03T13:22:49.111877Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Trump suggest Obama be  behind  town hall protest ...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterate Govt support for Finucane inqui...\n",
      "> NewsArticles-1100 (224 tokens): President Trump say he have ask the Justice Depart...\n",
      "> NewsArticles-1119 (975 tokens): an amazing moment in history  Donald Trump s press...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland see loss widen  bail  out R...\n",
      "> NewsArticles-1377 (774 tokens): Turkey  back rebel in  near full control  of Al Ba...\n",
      "> NewsArticles-1185 (1271 tokens): for more than a weekfrance  have be rock by  tensi...\n",
      "> NewsArticles-1263 (410 tokens): russian doctor use mobile field hospital to provid...\n",
      "> NewsArticles-1353 (30 tokens): islamic State battle  fierce gunfight outside Mosu...\n",
      "> NewsArticles-1387 (513 tokens): protest after Anaheim policeman drag teen  fire gu...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 7203\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import remove_punctuation\n",
    "\n",
    "# remove all punctuation\n",
    "remove_punctuation(corpus_norm)\n",
    "print_summary(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how the vocabulary size (the number of unique token types in a corpus) was also reduced with each step. We can also confirm that our functions worked by comparing the set of characters used in the original corpus to the set of characters used at the current normalization step via [corpus_unique_chars](api.rst#tmtoolkit.corpus.corpus_unique_chars). We can see that there are no more punctuation characters in the latter set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.114190Z",
     "iopub.status.busy": "2023-05-03T13:22:49.114015Z",
     "iopub.status.idle": "2023-05-03T13:22:49.122185Z",
     "shell.execute_reply": "2023-05-03T13:22:49.121817Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['\\n',\n",
       " ' ',\n",
       " '!',\n",
       " '\"',\n",
       " '#',\n",
       " '$',\n",
       " '%',\n",
       " '&',\n",
       " \"'\",\n",
       " '(',\n",
       " ')',\n",
       " '*',\n",
       " '+',\n",
       " ',',\n",
       " '-',\n",
       " '.',\n",
       " '/',\n",
       " '0',\n",
       " '1',\n",
       " '2',\n",
       " ...]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import corpus_unique_chars\n",
    "\n",
    "# original corpus\n",
    "sorted(corpus_unique_chars(corpus_orig))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.123992Z",
     "iopub.status.busy": "2023-05-03T13:22:49.123752Z",
     "iopub.status.idle": "2023-05-03T13:22:49.130027Z",
     "shell.execute_reply": "2023-05-03T13:22:49.129664Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['0',\n",
       " '1',\n",
       " '2',\n",
       " '3',\n",
       " '4',\n",
       " '5',\n",
       " '6',\n",
       " '7',\n",
       " '8',\n",
       " '9',\n",
       " 'A',\n",
       " 'B',\n",
       " 'C',\n",
       " 'D',\n",
       " 'E',\n",
       " 'F',\n",
       " 'G',\n",
       " 'H',\n",
       " 'I',\n",
       " 'J',\n",
       " ...]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transformed corpus\n",
    "sorted(corpus_unique_chars(corpus_norm))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A common (but harsh) practice is to transform all tokens to lowercase forms, which can be done with [to_lowercase](api.rst#tmtoolkit.corpus.to_lowercase), as already shown before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.131688Z",
     "iopub.status.busy": "2023-05-03T13:22:49.131514Z",
     "iopub.status.idle": "2023-05-03T13:22:49.153682Z",
     "shell.execute_reply": "2023-05-03T13:22:49.153261Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): trump suggest obama be  behind  town hall protest ...\n",
      "> NewsArticles-1407 (202 tokens): minister reiterate govt support for finucane inqui...\n",
      "> NewsArticles-1100 (224 tokens): president trump say he have ask the justice depart...\n",
      "> NewsArticles-1119 (975 tokens): an amazing moment in history  donald trump s press...\n",
      "> NewsArticles-1472 (298 tokens): royal bank of scotland see loss widen  bail  out r...\n",
      "> NewsArticles-1377 (774 tokens): turkey  back rebel in  near full control  of al ba...\n",
      "> NewsArticles-1185 (1271 tokens): for more than a weekfrance  have be rock by  tensi...\n",
      "> NewsArticles-1263 (410 tokens): russian doctor use mobile field hospital to provid...\n",
      "> NewsArticles-1353 (30 tokens): islamic state battle  fierce gunfight outside mosu...\n",
      "> NewsArticles-1387 (513 tokens): protest after anaheim policeman drag teen  fire gu...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 6692\n"
     ]
    }
   ],
   "source": [
    "to_lowercase(corpus_norm)\n",
    "print_summary(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several ways on how to treat numbers in text. You may choose to leave them untreated, remove them completely or transform them to placeholders that only encode their magnitude. Number removal can be applied via [filter_clean_tokens](api.rst#tmtoolkit.corpus.filter_clean_tokens) which I will present later. Number transformation to magnitudes can be done via [numbers_to_magnitudes](api.rst#tmtoolkit.corpus.numbers_to_magnitudes) which I will show now. But first, let's get an overview about the numbers used in the corpus:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.155620Z",
     "iopub.status.busy": "2023-05-03T13:22:49.155351Z",
     "iopub.status.idle": "2023-05-03T13:22:49.204417Z",
     "shell.execute_reply": "2023-05-03T13:22:49.203794Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>like_num</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>288</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>64</td>\n",
       "      <td>fifteen</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>320</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>96</td>\n",
       "      <td>one</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>328</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>104</td>\n",
       "      <td>four</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>759</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>535</td>\n",
       "      <td>100</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>787</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>563</td>\n",
       "      <td>four</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59253</th>\n",
       "      <td>NewsArticles-901</td>\n",
       "      <td>856</td>\n",
       "      <td>85</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59256</th>\n",
       "      <td>NewsArticles-901</td>\n",
       "      <td>859</td>\n",
       "      <td>9</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59374</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>63</td>\n",
       "      <td>2021</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59400</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>89</td>\n",
       "      <td>2010</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59413</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>102</td>\n",
       "      <td>1550</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1139 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position    token  like_num\n",
       "288    NewsArticles-1119        64  fifteen      True\n",
       "320    NewsArticles-1119        96      one      True\n",
       "328    NewsArticles-1119       104     four      True\n",
       "759    NewsArticles-1119       535      100      True\n",
       "787    NewsArticles-1119       563     four      True\n",
       "...                  ...       ...      ...       ...\n",
       "59253   NewsArticles-901       856       85      True\n",
       "59256   NewsArticles-901       859        9      True\n",
       "59374   NewsArticles-960        63     2021      True\n",
       "59400   NewsArticles-960        89     2010      True\n",
       "59413   NewsArticles-960       102     1550      True\n",
       "\n",
       "[1139 rows x 4 columns]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table_norm_num = tokens_table(corpus_norm, with_attr='like_num')\n",
    "table_norm_num[table_norm_num.like_num]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see all tokens that were detected as \"number-like\" by SpaCy. Those that consist of digits can be converted to their respective magnitudes using the mentioned `numbers_to_magnitudes` function. This function has many options for customization, but by default a two digits number will be converted to \"10\", a three digits number to \"100\", a ten digits number to \"1,000,000,000\", etc. You may customize this output, e.g. so that all numbers are converted to the form \"NNN...\". You can further drop or keep signs, use thousands separators, etc. Depending on your research context, it may or may not make sense to reduce numbers in such a way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.206618Z",
     "iopub.status.busy": "2023-05-03T13:22:49.206320Z",
     "iopub.status.idle": "2023-05-03T13:22:49.270570Z",
     "shell.execute_reply": "2023-05-03T13:22:49.270114Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>like_num</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>288</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>64</td>\n",
       "      <td>fifteen</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>320</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>96</td>\n",
       "      <td>one</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>328</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>104</td>\n",
       "      <td>four</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>759</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>535</td>\n",
       "      <td>100</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>787</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>563</td>\n",
       "      <td>four</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59253</th>\n",
       "      <td>NewsArticles-901</td>\n",
       "      <td>856</td>\n",
       "      <td>10</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59256</th>\n",
       "      <td>NewsArticles-901</td>\n",
       "      <td>859</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59374</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>63</td>\n",
       "      <td>1000</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59400</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>89</td>\n",
       "      <td>1000</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59413</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>102</td>\n",
       "      <td>1000</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1139 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position    token  like_num\n",
       "288    NewsArticles-1119        64  fifteen      True\n",
       "320    NewsArticles-1119        96      one      True\n",
       "328    NewsArticles-1119       104     four      True\n",
       "759    NewsArticles-1119       535      100      True\n",
       "787    NewsArticles-1119       563     four      True\n",
       "...                  ...       ...      ...       ...\n",
       "59253   NewsArticles-901       856       10      True\n",
       "59256   NewsArticles-901       859        1      True\n",
       "59374   NewsArticles-960        63     1000      True\n",
       "59400   NewsArticles-960        89     1000      True\n",
       "59413   NewsArticles-960       102     1000      True\n",
       "\n",
       "[1139 rows x 4 columns]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import numbers_to_magnitudes\n",
    "\n",
    "numbers_to_magnitudes(corpus_norm)\n",
    "table_norm_num = tokens_table(corpus_norm, with_attr='like_num')\n",
    "table_norm_num[table_norm_num.like_num]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, all numbers with digits were converted to their respective magnitudes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function [filter_clean_tokens](api.rst#tmtoolkit.corpus.filter_clean_tokens) finally applies several steps that remove tokens that meet certain criteria. This includes removing:\n",
    "\n",
    "- punctuation tokens (i.e. all tokens with attribute `is_punct` set to `True`)\n",
    "- stopwords (very common words for the given language, i.e. all tokens with attribute `is_stop` set to `True`)\n",
    "- empty tokens (i.e. `''`)\n",
    "- tokens that are longer or shorter than a certain number of characters\n",
    "- \"number-like\" tokens\n",
    "\n",
    "This method has many parameters to tweak, so it's recommended to check out the documentation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.272445Z",
     "iopub.status.busy": "2023-05-03T13:22:49.272214Z",
     "iopub.status.idle": "2023-05-03T13:22:49.316318Z",
     "shell.execute_reply": "2023-05-03T13:22:49.315790Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (182 tokens): trump suggest obama town hall protest president do...\n",
      "> NewsArticles-1407 (105 tokens): minister reiterate govt support finucane inquiry m...\n",
      "> NewsArticles-1100 (96 tokens): president trump say ask justice department investi...\n",
      "> NewsArticles-1119 (404 tokens): amazing moment history donald trump press conferen...\n",
      "> NewsArticles-1472 (163 tokens): royal bank scotland see loss widen bail royal bank...\n",
      "> NewsArticles-1377 (398 tokens): turkey back rebel near control al bab turkey defen...\n",
      "> NewsArticles-1185 (578 tokens): weekfrance rock tension flare police officer alleg...\n",
      "> NewsArticles-1263 (202 tokens): russian doctor use mobile field hospital provide m...\n",
      "> NewsArticles-1353 (21 tokens): islamic state battle fierce gunfight outside mosul...\n",
      "> NewsArticles-1387 (253 tokens): protest anaheim policeman drag teen fire gun lapd ...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 28042 / vocabulary size: 6268\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import filter_clean_tokens\n",
    "\n",
    "# remove punct., stopwords, empty tokens (this is the default)\n",
    "# plus tokens shorter than 2 characters\n",
    "filter_clean_tokens(corpus_norm, remove_shorter_than=2)\n",
    "print_summary(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Due to the removal of several tokens in the previous steps, the overall number of tokens was almost halved as compared to the original corpus:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.318081Z",
     "iopub.status.busy": "2023-05-03T13:22:49.317809Z",
     "iopub.status.idle": "2023-05-03T13:22:49.320873Z",
     "shell.execute_reply": "2023-05-03T13:22:49.320533Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(59598, 28042)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import corpus_num_tokens\n",
    "\n",
    "corpus_num_tokens(corpus_orig), corpus_num_tokens(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also observe that the vocabulary got smaller after the processing steps, which, for large corpora, is also important in terms of computation time and memory consumption for later analyses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.322464Z",
     "iopub.status.busy": "2023-05-03T13:22:49.322279Z",
     "iopub.status.idle": "2023-05-03T13:22:49.325809Z",
     "shell.execute_reply": "2023-05-03T13:22:49.325289Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9223, 6268)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocabulary_size(corpus_orig), vocabulary_size(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "You can also apply custom token transform functions by using [transform_tokens](api.rst#tmtoolkit.corpus.transform_tokens) and passing it a function that should be applied to each token in each document (hence it must accept one string argument).\n",
    "\n",
    "First let's define such a function. Here we create a simple function that should return a token's \"shape\" in terms of the case of its characters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.327467Z",
     "iopub.status.busy": "2023-05-03T13:22:49.327300Z",
     "iopub.status.idle": "2023-05-03T13:22:49.330434Z",
     "shell.execute_reply": "2023-05-03T13:22:49.330072Z"
    },
    "pycharm": {
     "is_executing": false,
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('XX', 'XxxxxXxxx', 'xxxxx')"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def token_shape(t):\n",
    "    return ''.join(['X' if str.isupper(c) else 'x' for c in t])\n",
    "\n",
    "token_shape('EU'), token_shape('CamelCase'), token_shape('lower')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now apply this function to our documents (we will use the original documents here, because they were not transformed to lower case):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.331715Z",
     "iopub.status.busy": "2023-05-03T13:22:49.331606Z",
     "iopub.status.idle": "2023-05-03T13:22:49.413740Z",
     "shell.execute_reply": "2023-05-03T13:22:49.413181Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Xxxxx xxxxxxxx Xxxxx xxx x xxxxxx x xxxx xxxx xxxx...\n",
      "> NewsArticles-1407 (202 tokens): Xxxxxxxx xxxxxxxxxx Xxxx xxxxxxx xxx Xxxxxxxx xxxx...\n",
      "> NewsArticles-1100 (224 tokens): Xxxxxxxxx Xxxxx xxxx xx xxx xxxxx xxx Xxxxxxx Xxxx...\n",
      "> NewsArticles-1119 (975 tokens): Xx xxxxxxx xxxxxx xx xxxxxxx x Xxxxxx Xxxxx xx xxx...\n",
      "> NewsArticles-1472 (298 tokens): Xxxxx Xxxx xx Xxxxxxxx xxxx xxxxxx xxxxxxxx xx Xxx...\n",
      "> NewsArticles-1377 (774 tokens): Xxxxxx x xxxxxx xxxxxx xx x xxxx xxxx xxxxxxx x xx...\n",
      "> NewsArticles-1185 (1271 tokens): Xxx xxxx xxxx x xxxxxxXxxxxx x xxx xxxx xxxxxx xx ...\n",
      "> NewsArticles-1263 (410 tokens): Xxxxxxx xxxxxxx xxx xxxxxx xxxxx xxxxxxxx xx xxxxx...\n",
      "> NewsArticles-1353 (30 tokens): Xxxxxxx Xxxxx xxxxxx x Xxxxxx xxxxxxxx xxxxxxx Xxx...\n",
      "> NewsArticles-1387 (513 tokens): Xxxxxxxx xxxxx Xxxxxxx xxxxxxxxx xxxxx xxxx x xxxx...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59598 / vocabulary size: 176\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import transform_tokens\n",
    "\n",
    "corpus_shapes = transform_tokens(corpus_orig, func=token_shape, inplace=False)\n",
    "print_summary(corpus_shapes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.415562Z",
     "iopub.status.busy": "2023-05-03T13:22:49.415449Z",
     "iopub.status.idle": "2023-05-03T13:22:49.418146Z",
     "shell.execute_reply": "2023-05-03T13:22:49.417691Z"
    }
   },
   "outputs": [],
   "source": [
    "del corpus_shapes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several more token transforming functions available in tmtoolkit. These are listed in the [corpus module API](api.rst#tmtoolkit-corpus). There are for example functions to simplify or normalize unicode characters in tokens.\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Retokenization**\n",
    "    \n",
    "One important point to note is that although you may change a token's text via a transformation, its token attributes such as POS tag, lemma, etc. stay the same. This is because the SpaCy NLP pipeline is only run initially and in most of the cases this is fine. However, if you want to re-run the NLP pipeline after you've applied some transformations in order to re-tokenize and re-analyze the text so that token attributes are also updated, you should run the [corpus_retokenize](api.rst#tmtoolkit.corpus.corpus_retokenize) function.\n",
    "    \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Identifying and joining token collocations\n",
    "\n",
    "*Collocations* are tokens that occur together in a series frequently (i.e. more than would be expected by chance). Examples could be the collocations \"United\", \"States\" or \"Bank\", \"of\", \"America\". The tmtoolkit package provides functions for identifying and joining such series of tokens.\n",
    "\n",
    "For identifying collocations, you can use [corpus_collocations](api.rst#tmtoolkit.corpus.corpus_collocations). By default, it will produce a dataframe ranked by a collocation statistic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.419687Z",
     "iopub.status.busy": "2023-05-03T13:22:49.419583Z",
     "iopub.status.idle": "2023-05-03T13:22:49.679458Z",
     "shell.execute_reply": "2023-05-03T13:22:49.678811Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>collocation</th>\n",
       "      <th>statistic</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>successor soviet</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>anatolyevich sushchin</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>alexsey alexseyevich</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>alexseyevich belan</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>akehmet tokbergenov</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ancaster ontario</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>ontario toronto</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>marissa mayer</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>podesta internal</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>diminish thrive</td>\n",
       "      <td>10.212809</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             collocation  statistic\n",
       "0       successor soviet  10.212809\n",
       "1  anatolyevich sushchin  10.212809\n",
       "2   alexsey alexseyevich  10.212809\n",
       "3     alexseyevich belan  10.212809\n",
       "4    akehmet tokbergenov  10.212809\n",
       "5       ancaster ontario  10.212809\n",
       "6        ontario toronto  10.212809\n",
       "7          marissa mayer  10.212809\n",
       "8       podesta internal  10.212809\n",
       "9        diminish thrive  10.212809"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import corpus_collocations\n",
    "\n",
    "corpus_collocations(corpus_norm).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The default statistic is the *positive pointwise mutual information (PPMI)* measure implemented in the [ppmi](api.rst#tmtoolkit.tokenseq.ppmi) function. You can use a different statistic via the `statistic` argument. Here, we use the PMI³ statistic from the [tokenseq](api.rst#module-tmtoolkit.tokenseq) module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:49.681326Z",
     "iopub.status.busy": "2023-05-03T13:22:49.681179Z",
     "iopub.status.idle": "2023-05-03T13:22:50.362259Z",
     "shell.execute_reply": "2023-05-03T13:22:50.361804Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/markus/Development/tmtoolkit/tmtoolkit/tokenseq/_metrics.py:129: RuntimeWarning: divide by zero encountered in log\n",
      "  logxy = logfn(xy)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>collocation</th>\n",
       "      <th>statistic</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>le pen</td>\n",
       "      <td>-7.114798</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>white house</td>\n",
       "      <td>-7.941276</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>real estate</td>\n",
       "      <td>-8.004290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>donald trump</td>\n",
       "      <td>-8.043160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>kuala lumpur</td>\n",
       "      <td>-8.133367</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>united states</td>\n",
       "      <td>-8.187434</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>whatsapp tumblr</td>\n",
       "      <td>-8.266899</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>tumblr linkedin</td>\n",
       "      <td>-8.266899</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>digg reddit</td>\n",
       "      <td>-8.266899</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>reddit newsvine</td>\n",
       "      <td>-8.266899</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       collocation  statistic\n",
       "0           le pen  -7.114798\n",
       "1      white house  -7.941276\n",
       "2      real estate  -8.004290\n",
       "3     donald trump  -8.043160\n",
       "4     kuala lumpur  -8.133367\n",
       "5    united states  -8.187434\n",
       "6  whatsapp tumblr  -8.266899\n",
       "7  tumblr linkedin  -8.266899\n",
       "8      digg reddit  -8.266899\n",
       "9  reddit newsvine  -8.266899"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.tokenseq import pmi3\n",
    "\n",
    "corpus_collocations(corpus_norm, statistic=pmi3).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a few more parameters to `corpus_collocations`. For instance, you can specify a minimum number for the collocation to occur using `min_count` and a threshold value on the calculated statistic via `threshold` and produce a list output instead of a dataframe via `as_table=False`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:50.364282Z",
     "iopub.status.busy": "2023-05-03T13:22:50.364135Z",
     "iopub.status.idle": "2023-05-03T13:22:50.393900Z",
     "shell.execute_reply": "2023-05-03T13:22:50.393369Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/markus/Development/tmtoolkit/tmtoolkit/tokenseq/_metrics.py:129: RuntimeWarning: divide by zero encountered in log\n",
      "  logxy = logfn(xy)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[('white house', -4.1705337005796475),\n",
       " ('le pen', -4.1705337005796475),\n",
       " ('donald trump', -4.184123274560292),\n",
       " ('year old', -4.37305796469112),\n",
       " ('new york', -4.515374186871377),\n",
       " ('real estate', -4.575998808687811),\n",
       " ('president donald', -4.75002156266707),\n",
       " ('china daily', -4.783638173466056),\n",
       " ('far right', -4.863680881139593),\n",
       " ('prime minister', -4.900048525310467),\n",
       " ('north korean', -4.940257380836643),\n",
       " ('� �', -4.950692258129222),\n",
       " ('chuck berry', -4.950692258129222),\n",
       " ('body camera', -4.950692258129222)]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corpus_collocations(corpus_norm, statistic=pmi3, min_count=5,\n",
    "                    threshold=-5, as_table=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After identifying and investigating collocations, you may also choose to join some of them so that they form a single token. This can be done via [join_collocations_by_statistic](api.rst#tmtoolkit.corpus.join_collocations_by_statistic). Here, we specify the collocation statistic to use, we set a minimum threshold and also enable returning a set of actually joint tokens:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:50.395390Z",
     "iopub.status.busy": "2023-05-03T13:22:50.395249Z",
     "iopub.status.idle": "2023-05-03T13:22:50.433493Z",
     "shell.execute_reply": "2023-05-03T13:22:50.432940Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'body_camera',\n",
       " 'china_daily',\n",
       " 'chuck_berry',\n",
       " 'donald_trump',\n",
       " 'far_right',\n",
       " 'le_pen',\n",
       " 'new_york',\n",
       " 'north_korean',\n",
       " 'president_donald',\n",
       " 'prime_minister',\n",
       " 'real_estate',\n",
       " 'white_house',\n",
       " 'year_old',\n",
       " '�_�'}"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import join_collocations_by_statistic\n",
    "\n",
    "join_collocations_by_statistic(corpus_norm, statistic=pmi3, min_count=5,\n",
    "                               threshold=-5, return_joint_tokens=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These collocations were joint to form single tokens. We can check one of them to see in which documents this joint token appears by using the [find_documents](api.rst#tmtoolkit.corpus.find_documents) function. This function searches the documents for matches to one or more keywords or patterns. By default, it returns all documents with at least one match as dictionary that maps document labels to number of matches. Here, we use the common parameter `as_table` again to provide a tabular output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:50.435627Z",
     "iopub.status.busy": "2023-05-03T13:22:50.435497Z",
     "iopub.status.idle": "2023-05-03T13:22:50.451712Z",
     "shell.execute_reply": "2023-05-03T13:22:50.451342Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>n_matches</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2431</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2487</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1185</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1853</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  n_matches\n",
       "1  NewsArticles-2431          6\n",
       "0  NewsArticles-2487          4\n",
       "2  NewsArticles-1185          1\n",
       "3  NewsArticles-1853          1"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import find_documents\n",
    "\n",
    "find_documents(corpus_norm, 'far_right', as_table='-n_matches')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another option for joining collocations is using the [join_collocations_by_patterns](api.rst#tmtoolkit.corpus.join_collocations_by_patterns) function, which allows you to define a pattern of subsequent tokens that should be joint. Here, we want to join all subsequent tokens where the first token is \"north\" and the second is anything that starts with \"korea*\", i.e. matching for example \"north\", \"korea\" or \"north\", \"korean\". The pattern \"korea*\" is a *glob pattern* – details on pattern matching will be given in the next section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:50.453654Z",
     "iopub.status.busy": "2023-05-03T13:22:50.453503Z",
     "iopub.status.idle": "2023-05-03T13:22:50.483398Z",
     "shell.execute_reply": "2023-05-03T13:22:50.482905Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'north_korea', 'north_koreans'}"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import join_collocations_by_patterns\n",
    "\n",
    "join_collocations_by_patterns(corpus_norm, ['north', 'korea*'], match_type='glob', return_joint_tokens=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see, that three collocation patterns were joint. Again, we can find the documents that contain these patterns using a glob pattern:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:50.485386Z",
     "iopub.status.busy": "2023-05-03T13:22:50.485225Z",
     "iopub.status.idle": "2023-05-03T13:22:50.507275Z",
     "shell.execute_reply": "2023-05-03T13:22:50.506880Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>n_matches</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1587</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1860</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  n_matches\n",
       "0  NewsArticles-1587         18\n",
       "1  NewsArticles-1860         11"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "find_documents(corpus_norm, 'north_korea*', match_type='glob' , as_table='-n_matches')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing corpus statistics of the transformed corpus\n",
    "\n",
    "In order to see the effect of the corpus transformations on the distribution of tokens, we can again plot these transformations.\n",
    "\n",
    "Plotting the distribution of document lengths on the same scale as for the original corpus reveals that the documents are now much shorter and that the distribution is much less dispersed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:50.509292Z",
     "iopub.status.busy": "2023-05-03T13:22:50.509151Z",
     "iopub.status.idle": "2023-05-03T13:22:50.603556Z",
     "shell.execute_reply": "2023-05-03T13:22:50.603196Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1sAAAIjCAYAAAD1OgEdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABFtklEQVR4nO3deVhV5f7+8XsDMjiAogySAk6ZszmmllOUmtloWcdMzVLLMT2m5FGjLKxOSZk5NIiVaeZxOplThkPmkJqWDYqF5jGVcgCnUOH5/eHP/W0LKCIPm+H9uq596X7Ws9b6rA/TvllrLxzGGCMAAAAAQJ7ycHcBAAAAAFAUEbYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2ACALkZGR6tWrl7vLKPJeffVVVa1aVZ6enmrYsOFVr9+2bVu1bds2z+sqjnr16qXIyEh3l3FF8fHxcjgc2rJli7tLAYArImwBKPKu9OKsbdu2qlu37jXv5/PPP9dzzz13zdspLlasWKFnnnlGrVq10owZM/TSSy+5u6Qi5euvv9Zzzz2n48ePu7uUXHn77bcVHx/v7jIA4Jp4ubsAACiIdu3aJQ+Pq/t91Oeff67JkycTuHLoyy+/lIeHh9577z15e3u7u5wi5+uvv1ZMTIx69eqlsmXLurucq/b222+rQoUKnGEGUKhxZgsAsuDj46MSJUq4u4yrcurUKXeXcFWSk5Pl5+dH0AIAFFmELQDIwqXv2Tp37pxiYmJUo0YN+fr6qnz58rr55pu1cuVKSRfe7zJ58mRJksPhcD4uOnXqlIYPH67KlSvLx8dHNWvW1L///W8ZY1z2e+bMGQ0ePFgVKlRQmTJldNddd+nAgQNyOBwuZ8yee+45ORwO/fjjj/rHP/6hcuXK6eabb5Ykfffdd+rVq5eqVq0qX19fhYaG6rHHHtORI0dc9nVxG7t379YjjzyigIAABQUFacyYMTLGaP/+/br77rvl7++v0NBQvfbaaznq3fnz5/XCCy+oWrVq8vHxUWRkpJ599lmlpaU55zgcDs2YMUOnTp1y9upKl4xNnz5d1apVk5+fn5o1a6Z169ZlOS85OVl9+vRRSEiIfH191aBBA82cOTPTvIyMDL3xxhuqV6+efH19FRQUpI4dOzovN927d2+2dWX38biWXqalpWncuHGqXr26fHx8VLlyZT3zzDMufbu474EDB2rhwoWqW7eufHx8VKdOHS1btsylnhEjRkiSqlSp4uzx3r17L9vjrHoUFxenOnXqyNfXVyEhIerXr5+OHTvmMi8yMlJ33nmnvvrqKzVr1ky+vr6qWrWqPvjgg0zb/O6779SmTRv5+fmpUqVKGj9+vGbMmOFSX2RkpH744QetWbPGWful781LS0vTsGHDFBQUpFKlSunee+/VH3/84TJny5Yt6tChgypUqCA/Pz9VqVJFjz322FX1AACuBZcRAig2UlJS9Oeff2YaP3fu3BXXfe655xQbG6vHH39czZo1U2pqqrZs2aJt27bptttuU79+/fT7779r5cqV+vDDD13WNcborrvuUkJCgvr06aOGDRtq+fLlGjFihA4cOKCJEyc65/bq1Utz585Vjx49dNNNN2nNmjXq3LlztnU98MADqlGjhl566SVncFu5cqV+/fVX9e7dW6Ghofrhhx80ffp0/fDDD9q4caNLCJSkbt26qVatWpowYYKWLFmi8ePHKzAwUNOmTVP79u318ssva9asWfrnP/+ppk2bqnXr1pft1eOPP66ZM2eqa9euGj58uDZt2qTY2Fj99NNPWrBggSTpww8/1PTp07V582a9++67kqSWLVtmu8333ntP/fr1U8uWLTV06FD9+uuvuuuuuxQYGKjKlSs75505c0Zt27bVnj17NHDgQFWpUkWffvqpevXqpePHj2vIkCHOuX369FF8fLw6deqkxx9/XOfPn9e6deu0ceNGNWnS5LLHmJ3c9jIjI0N33XWXvvrqK/Xt21e1atXS999/r4kTJ2r37t1auHChy36++uorzZ8/X0899ZTKlCmjN998U/fff79+++03lS9fXvfdd592796t2bNna+LEiapQoYIkKSgo6KqOp1+/foqPj1fv3r01ePBgJSUl6a233tK3336r9evXu5z93bNnj7p27ao+ffqoZ8+eev/999WrVy81btxYderUkSQdOHBA7dq1k8PhUHR0tEqVKqV3331XPj4+LvuNi4vToEGDVLp0aY0ePVqSFBIS4jJn0KBBKleunMaNG6e9e/cqLi5OAwcO1CeffCLpQui+/fbbFRQUpFGjRqls2bLau3ev5s+ff1U9AIBrYgCgiJsxY4aRdNlHnTp1XNaJiIgwPXv2dD5v0KCB6dy582X3M2DAAJPVt9WFCxcaSWb8+PEu4127djUOh8Ps2bPHGGPM1q1bjSQzdOhQl3m9evUyksy4ceOcY+PGjTOSzMMPP5xpf6dPn840Nnv2bCPJrF27NtM2+vbt6xw7f/68qVSpknE4HGbChAnO8WPHjhk/Pz+XnmRl+/btRpJ5/PHHXcb/+c9/Gknmyy+/dI717NnTlCpV6rLbM8aYs2fPmuDgYNOwYUOTlpbmHJ8+fbqRZNq0aeMci4uLM5LMRx995LJ+ixYtTOnSpU1qaqoxxpgvv/zSSDKDBw/OtL+MjAxjjDFJSUlGkpkxY0amOdl9PHLbyw8//NB4eHiYdevWuexn6tSpRpJZv369y769vb2dnzfGGLNjxw4jyUyaNMk59uqrrxpJJikpKVP9WenZs6eJiIhwPl+3bp2RZGbNmuUyb9myZZnGIyIiMn1+JScnGx8fHzN8+HDn2KBBg4zD4TDffvutc+zIkSMmMDAwU6116tRx+dhedPHrOSoqyvmxMsaYp59+2nh6eprjx48bY4xZsGCBkWS++eabHB0/ANjAZYQAio3Jkydr5cqVmR7169e/4rply5bVDz/8oMTExKve7+effy5PT08NHjzYZXz48OEyxmjp0qWS5LwM7KmnnnKZN2jQoGy33b9//0xjfn5+zv//9ddf+vPPP3XTTTdJkrZt25Zp/uOPP+78v6enp5o0aSJjjPr06eMcL1u2rGrWrKlff/0121qkC8cqScOGDXMZHz58uCRpyZIll10/K1u2bFFycrL69+/v8v6uXr16KSAgINP+Q0ND9fDDDzvHSpQoocGDB+vkyZNas2aNJOk///mPHA6Hxo0bl2l/l575uxq57eWnn36qWrVq6YYbbtCff/7pfLRv316SlJCQ4LKfqKgoVatWzfm8fv368vf3v+LH52p8+umnCggI0G233eZSU+PGjVW6dOlMNdWuXVu33HKL83lQUFCm41y2bJlatGjhcpv/wMBAde/e/arr69u3r8vH6pZbblF6err27dsnSc6bgnz22Wc5OnsNADZwGSGAYqNZs2ZZXh5Wrly5LC8v/Lvnn39ed999t66//nrVrVtXHTt2VI8ePXIU1Pbt26ewsDCVKVPGZbxWrVrO5Rf/9fDwUJUqVVzmVa9ePdttXzpXko4ePaqYmBjNmTNHycnJLstSUlIyzQ8PD3d5HhAQIF9fX+elZ38fv/R9X5e6eAyX1hwaGqqyZcs6j/VqXFynRo0aLuMlSpRQ1apVM82tUaNGpjtJXtrrX375RWFhYQoMDLzqei4nt71MTEzUTz/9lO1lfpd+HC/dj3Th8/jS91Jdi8TERKWkpCg4ODjPatq3b59atGiRad7lPsezc+n+ypUrJ0nO/bVp00b333+/YmJiNHHiRLVt21b33HOP/vGPf2S6bBEAbCFsAUAOtG7dWr/88osWLVqkFStW6N1339XEiRM1depUl7MZ+e3vZ7EuevDBB/X1119rxIgRatiwoUqXLq2MjAx17NhRGRkZmeZ7enrmaExSpht6ZOdazg4VFNkdQ3p6erbr5LaXGRkZqlevnl5//fUs5/79fWk53ea1ysjIUHBwsGbNmpXl8kuDYX7UdDX7czgcmjdvnjZu3Kj//ve/Wr58uR577DG99tpr2rhxo0qXLm2lLgD4O8IWAORQYGCgevfurd69e+vkyZNq3bq1nnvuOWfYyu7FeUREhL744gudOHHC5ezWzz//7Fx+8d+MjAwlJSW5nMXZs2dPjms8duyYVq1apZiYGI0dO9Y5npvLH3Pj4jEkJiY6zyZJ0uHDh3X8+HHnsV7tNqULx3Dxsjrpwo1NkpKS1KBBA5e53333nTIyMlzObl3a62rVqmn58uU6evRotme3Lp4pufSPAufm7NyVVKtWTTt27NCtt96aZ0H1WrdTrVo1ffHFF2rVqlWWoT43IiIisvx8zmosr/pw00036aabbtKLL76ojz/+WN27d9ecOXPc+ksSAMUH79kCgBy49PK50qVLq3r16i635S5VqpSkzC/O77jjDqWnp+utt95yGZ84caIcDoc6deokSerQoYOkC3/M9e8mTZqU4zov/rb/0rMJcXFxOd7Gtbjjjjuy3N/FMzaXu7Nidpo0aaKgoCBNnTpVZ8+edY7Hx8dn2etDhw4570gnXbgV/aRJk1S6dGm1adNGknT//ffLGKOYmJhM+7vYO39/f1WoUEFr1651WX7pxycvPPjggzpw4IDeeeedTMvOnDmTq7+hlt3n49XUlJ6erhdeeCHTsvPnz+dqux06dNCGDRu0fft259jRo0ezPHtWqlSpXNcuXfjFw6VfBxffK3bp7fQBwBbObAFADtSuXVtt27ZV48aNFRgYqC1btmjevHkaOHCgc07jxo0lSYMHD1aHDh3k6emphx56SF26dFG7du00evRo7d27Vw0aNNCKFSu0aNEiDR061Hmjg8aNG+v+++9XXFycjhw54rz1++7duyXl7Df9/v7+at26tV555RWdO3dO1113nVasWKGkpCQLXcmsQYMG6tmzp6ZPn67jx4+rTZs22rx5s2bOnKl77rlH7dq1u+ptlihRQuPHj1e/fv3Uvn17devWTUlJSZoxY0am92z17dtX06ZNU69evbR161ZFRkZq3rx5Wr9+veLi4pxnFtu1a6cePXrozTffVGJiovMSy3Xr1qldu3bOj+vjjz+uCRMm6PHHH1eTJk20du1a58cjL/Xo0UNz585V//79lZCQoFatWik9PV0///yz5s6dq+XLl1/17egvfj6OHj1aDz30kEqUKKEuXbo4Q9iVtGnTRv369VNsbKy2b9+u22+/XSVKlFBiYqI+/fRTvfHGG+ratetV1fTMM8/oo48+0m233aZBgwY5b/0eHh6uo0ePunyON27cWFOmTNH48eNVvXp1BQcHu5zZvJKZM2fq7bff1r333qtq1arpxIkTeuedd+Tv7+/8pQAA2EbYAoAcGDx4sBYvXqwVK1YoLS1NERERGj9+vPMPx0rSfffdp0GDBmnOnDn66KOPZIzRQw89JA8PDy1evFhjx47VJ598ohkzZigyMlKvvvqq8y59F33wwQcKDQ3V7NmztWDBAkVFRemTTz5RzZo15evrm6NaP/74Yw0aNEiTJ0+WMUa33367li5dqrCwsDztSXbeffddVa1aVfHx8VqwYIFCQ0MVHR2d5Z3/cqpv375KT0/Xq6++qhEjRqhevXpavHixxowZ4zLPz89Pq1ev1qhRozRz5kylpqaqZs2amjFjhssfqZakGTNmqH79+nrvvfc0YsQIBQQEqEmTJi5/72vs2LH6448/NG/ePM2dO1edOnXS0qVLs71pRG55eHho4cKFmjhxoj744AMtWLBAJUuWVNWqVTVkyBBdf/31V73Npk2b6oUXXtDUqVO1bNky5yWqOQ1bkjR16lQ1btxY06ZN07PPPisvLy9FRkbqkUceUatWra66psqVKyshIUGDBw/WSy+9pKCgIA0YMEClSpXS4MGDXT7Hx44dq3379umVV17RiRMn1KZNm6sKWxeD/pw5c3T48GEFBASoWbNmmjVrVpY3lgEAGxzG1jtXAQB5Yvv27brxxhv10Ucf5eoW2UBBN3ToUE2bNk0nT57M9sYXAFAY8Z4tAChAzpw5k2ksLi5OHh4eat26tRsqAvLWpZ/jR44c0Ycffqibb76ZoAWgyOEyQgAoQF555RVt3bpV7dq1k5eXl5YuXaqlS5eqb9++mW7/DRRGLVq0UNu2bVWrVi0dPnxY7733nlJTUzNdEgoARQGXEQJAAbJy5UrFxMToxx9/1MmTJxUeHq4ePXpo9OjR8vLi92Mo/J599lnNmzdP//vf/+RwONSoUSONGzdOUVFR7i4NAPIcYQsAAAAALOA9WwAAAABgAWELAAAAACwo8m8AyMjI0O+//64yZcrk6A+CAgAAACiajDE6ceKEwsLC5OFh/7xTkQ9bv//+O3fwAgAAAOC0f/9+VapUyfp+inzYKlOmjKQLDfX393dzNQAAAADcJTU1VZUrV3ZmBNuKfNi6eOmgv78/YQsAAABAvr29iBtkAAAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYIFbw9batWvVpUsXhYWFyeFwaOHChdnO7d+/vxwOh+Li4vKtPgAAAADILbeGrVOnTqlBgwaaPHnyZectWLBAGzduVFhYWD5VBgAAAADXxsudO+/UqZM6dep02TkHDhzQoEGDtHz5cnXu3DmfKgMAAACAa+PWsHUlGRkZ6tGjh0aMGKE6derkaJ20tDSlpaU5n6emptoqDwAAAACyVaDD1ssvvywvLy8NHjw4x+vExsYqJibGYlXFQ+SoJde0/t4JnIUEAABA8VZg70a4detWvfHGG4qPj5fD4cjxetHR0UpJSXE+9u/fb7FKAAAAAMhagQ1b69atU3JyssLDw+Xl5SUvLy/t27dPw4cPV2RkZLbr+fj4yN/f3+UBAAAAAPmtwF5G2KNHD0VFRbmMdejQQT169FDv3r3dVBUAAAAA5Ixbw9bJkye1Z88e5/OkpCRt375dgYGBCg8PV/ny5V3mlyhRQqGhoapZs2Z+lwoAAAAAV8WtYWvLli1q166d8/mwYcMkST179lR8fLybqgIAAACAa+fWsNW2bVsZY3I8f+/evfaKAQAAAIA8VGBvkAEAAAAAhRlhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALvNxdALIXOWrJNa2/d0LnPKoEAAAAwNXizBYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYIFbw9batWvVpUsXhYWFyeFwaOHChc5l586d08iRI1WvXj2VKlVKYWFhevTRR/X777+7r2AAAAAAyCG3hq1Tp06pQYMGmjx5cqZlp0+f1rZt2zRmzBht27ZN8+fP165du3TXXXe5oVIAAAAAuDpe7tx5p06d1KlTpyyXBQQEaOXKlS5jb731lpo1a6bffvtN4eHh+VEiAAAAAOSKW8PW1UpJSZHD4VDZsmWznZOWlqa0tDTn89TU1HyoDAAAAABcFZobZPz1118aOXKkHn74Yfn7+2c7LzY2VgEBAc5H5cqV87FKAAAAALigUIStc+fO6cEHH5QxRlOmTLns3OjoaKWkpDgf+/fvz6cqAQAAAOD/FPjLCC8GrX379unLL7+87FktSfLx8ZGPj08+VQcAAAAAWSvQYeti0EpMTFRCQoLKly/v7pIAAAAAIEfcGrZOnjypPXv2OJ8nJSVp+/btCgwMVMWKFdW1a1dt27ZNn332mdLT03Xo0CFJUmBgoLy9vd1VNgAAAABckVvD1pYtW9SuXTvn82HDhkmSevbsqeeee06LFy+WJDVs2NBlvYSEBLVt2za/ygQAAACAq+bWsNW2bVsZY7JdfrllAAAAAFCQFYq7EQIAAABAYUPYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAscGvYWrt2rbp06aKwsDA5HA4tXLjQZbkxRmPHjlXFihXl5+enqKgoJSYmuqdYAAAAALgKbg1bp06dUoMGDTR58uQsl7/yyit68803NXXqVG3atEmlSpVShw4d9Ndff+VzpQAAAABwdbzcufNOnTqpU6dOWS4zxiguLk7/+te/dPfdd0uSPvjgA4WEhGjhwoV66KGHslwvLS1NaWlpzuepqal5XzgAAAAAXEGBfc9WUlKSDh06pKioKOdYQECAmjdvrg0bNmS7XmxsrAICApyPypUr50e5AAAAAOCiwIatQ4cOSZJCQkJcxkNCQpzLshIdHa2UlBTnY//+/VbrBAAAAICsuPUyQht8fHzk4+Pj7jIAAAAAFHMF9sxWaGioJOnw4cMu44cPH3YuAwAAAICCqsCGrSpVqig0NFSrVq1yjqWmpmrTpk1q0aKFGysDAAAAgCtz62WEJ0+e1J49e5zPk5KStH37dgUGBio8PFxDhw7V+PHjVaNGDVWpUkVjxoxRWFiY7rnnHvcVDQAAAAA54NawtWXLFrVr1875fNiwYZKknj17Kj4+Xs8884xOnTqlvn376vjx47r55pu1bNky+fr6uqtkAAAAAMgRt4attm3byhiT7XKHw6Hnn39ezz//fD5WBQAAAADXrsC+ZwsAAAAACjPCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWeLm7gKIuctQSd5fgFtdy3HsndM7DSgAAAAD34MwWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwp02EpPT9eYMWNUpUoV+fn5qVq1anrhhRdkjHF3aQAAAABwWV7uLuByXn75ZU2ZMkUzZ85UnTp1tGXLFvXu3VsBAQEaPHiwu8sDAAAAgGwV6LD19ddf6+6771bnzp0lSZGRkZo9e7Y2b97s5soAAAAA4PIK9GWELVu21KpVq7R7925J0o4dO/TVV1+pU6dO2a6Tlpam1NRUlwcAAAAA5LcCfWZr1KhRSk1N1Q033CBPT0+lp6frxRdfVPfu3bNdJzY2VjExMflYJfJa5Kgl17T+3gmd86iSq3cttbuzbgAAAOS9An1ma+7cuZo1a5Y+/vhjbdu2TTNnztS///1vzZw5M9t1oqOjlZKS4nzs378/HysGAAAAgAsK9JmtESNGaNSoUXrooYckSfXq1dO+ffsUGxurnj17ZrmOj4+PfHx88rNMAAAAAMikQJ/ZOn36tDw8XEv09PRURkaGmyoCAAAAgJzJVdhq3769jh8/nmk8NTVV7du3v9aanLp06aIXX3xRS5Ys0d69e7VgwQK9/vrruvfee/NsHwAAAABgQ64uI1y9erXOnj2bafyvv/7SunXrrrmoiyZNmqQxY8boqaeeUnJyssLCwtSvXz+NHTs2z/YBAAAAADZcVdj67rvvnP//8ccfdejQIefz9PR0LVu2TNddd12eFVemTBnFxcUpLi4uz7YJAAAAAPnhqsJWw4YN5XA45HA4srxc0M/PT5MmTcqz4gAAAACgsLqqsJWUlCRjjKpWrarNmzcrKCjIuczb21vBwcHy9PTM8yIBAAAAoLC5qrAVEREhSdwNEAAAAACuINd/ZysxMVEJCQlKTk7OFL64gQUAAACA4i5XYeudd97Rk08+qQoVKig0NFQOh8O5zOFwELYAAAAAFHu5Clvjx4/Xiy++qJEjR+Z1PQAAAABQJOTqjxofO3ZMDzzwQF7XAgAAAABFRq7C1gMPPKAVK1bkdS0AAAAAUGTk6jLC6tWra8yYMdq4caPq1aunEiVKuCwfPHhwnhQHAAAAAIVVrsLW9OnTVbp0aa1Zs0Zr1qxxWeZwOAhbAAAAAIq9XIWtpKSkvK4DAAAAAIqUXL1nCwAAAABwebk6s/XYY49ddvn777+fq2IAAAAAoKjIVdg6duyYy/Nz585p586dOn78uNq3b58nhQEAAABAYZarsLVgwYJMYxkZGXryySdVrVq1ay4KAAAAAAq7PHvPloeHh4YNG6aJEyfm1SYBAAAAoNDK0xtk/PLLLzp//nxebhIAAAAACqVcXUY4bNgwl+fGGB08eFBLlixRz54986QwAAAAACjMchW2vv32W5fnHh4eCgoK0muvvXbFOxUCAAAAQHGQq7CVkJCQ13UAAAAAQJGSq7B10R9//KFdu3ZJkmrWrKmgoKA8KQoAAAAACrtc3SDj1KlTeuyxx1SxYkW1bt1arVu3VlhYmPr06aPTp0/ndY0AAAAAUOjkKmwNGzZMa9as0X//+18dP35cx48f16JFi7RmzRoNHz48r2sEAAAAgEInV5cR/uc//9G8efPUtm1b59gdd9whPz8/Pfjgg5oyZUpe1QcAAAAAhVKuzmydPn1aISEhmcaDg4O5jBAAAAAAlMszWy1atNC4ceP0wQcfyNfXV5J05swZxcTEqEWLFnlaIJCfIkctcXcJAAAAKCJyFbbi4uLUsWNHVapUSQ0aNJAk7dixQz4+PlqxYkWeFggAAAAAhVGuwla9evWUmJioWbNm6eeff5YkPfzww+revbv8/PzytEAAAAAAKIxyFbZiY2MVEhKiJ554wmX8/fff1x9//KGRI0fmSXEAAAAAUFjl6gYZ06ZN0w033JBpvE6dOpo6deo1FwUAAAAAhV2uwtahQ4dUsWLFTONBQUE6ePDgNRcFAAAAAIVdrsJW5cqVtX79+kzj69evV1hY2DUXBQAAAACFXa7es/XEE09o6NChOnfunNq3by9JWrVqlZ555hkNHz48TwsEAAAAgMIoV2FrxIgROnLkiJ566imdPXtWkuTr66uRI0cqOjo6TwsEAAAAgMIoV2HL4XDo5Zdf1pgxY/TTTz/Jz89PNWrUkI+PT17XBwAAAACFUq7C1kWlS5dW06ZN86oWAAAAACgycnWDDAAAAADA5RG2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsKDAh60DBw7okUceUfny5eXn56d69eppy5Yt7i4LAAAAAC7Ly90FXM6xY8fUqlUrtWvXTkuXLlVQUJASExNVrlw5d5cGAAAAAJdVoMPWyy+/rMqVK2vGjBnOsSpVqrixIgAAAADImQJ9GeHixYvVpEkTPfDAAwoODtaNN96od95557LrpKWlKTU11eUBAAAAAPmtQJ/Z+vXXXzVlyhQNGzZMzz77rL755hsNHjxY3t7e6tmzZ5brxMbGKiYmJs9qiBy1JM+2ld8Kc+0AAABAYVegz2xlZGSoUaNGeumll3TjjTeqb9++euKJJzR16tRs14mOjlZKSorzsX///nysGAAAAAAuKNBhq2LFiqpdu7bLWK1atfTbb79lu46Pj4/8/f1dHgAAAACQ3wp02GrVqpV27drlMrZ7925FRES4qSIAAAAAyJkCHbaefvppbdy4US+99JL27Nmjjz/+WNOnT9eAAQPcXRoAAAAAXFaBDltNmzbVggULNHv2bNWtW1cvvPCC4uLi1L17d3eXBgAAAACXVaDvRihJd955p+688053lwEAAAAAV6VAn9kCAAAAgMKKsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALDAy90FALggctSSa1p/74TOeVQJAAAA8gJntgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhQqMLWhAkT5HA4NHToUHeXAgAAAACXVWjC1jfffKNp06apfv367i4FAAAAAK6oUIStkydPqnv37nrnnXdUrlw5d5cDAAAAAFdUKMLWgAED1LlzZ0VFRV1xblpamlJTU10eAAAAAJDfvNxdwJXMmTNH27Zt0zfffJOj+bGxsYqJibFcFQqyyFFL3F0CAAAAULDPbO3fv19DhgzRrFmz5Ovrm6N1oqOjlZKS4nzs37/fcpUAAAAAkFmBPrO1detWJScnq1GjRs6x9PR0rV27Vm+99ZbS0tLk6enpso6Pj498fHzyu1QAAAAAcFGgw9att96q77//3mWsd+/euuGGGzRy5MhMQQsAAAAACooCHbbKlCmjunXruoyVKlVK5cuXzzQOAAAAAAVJgX7PFgAAAAAUVgX6zFZWVq9e7e4SAAAAAOCKOLMFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwwMvdBQBwv8hRS65p/b0TOhfKfQMAANjEmS0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwIICHbZiY2PVtGlTlSlTRsHBwbrnnnu0a9cud5cFAAAAAFdUoMPWmjVrNGDAAG3cuFErV67UuXPndPvtt+vUqVPuLg0AAAAALsvL3QVczrJly1yex8fHKzg4WFu3blXr1q3dVBUAAAAAXFmBDluXSklJkSQFBgZmOyctLU1paWnO56mpqdbrAgAAAIBLFZqwlZGRoaFDh6pVq1aqW7dutvNiY2MVExOTj5UBBUPkqCXuLgEAAAB/U6Dfs/V3AwYM0M6dOzVnzpzLzouOjlZKSorzsX///nyqEAAAAAD+T6E4szVw4EB99tlnWrt2rSpVqnTZuT4+PvLx8cmnygAAAAAgawU6bBljNGjQIC1YsECrV69WlSpV3F0SAAAAAORIgQ5bAwYM0Mcff6xFixapTJkyOnTokCQpICBAfn5+bq4OAAAAALJXoN+zNWXKFKWkpKht27aqWLGi8/HJJ5+4uzQAAAAAuKwCfWbLGOPuEgAAAAAgVwr0mS0AAAAAKKwIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABY4OXuAgAUfpGjlri7BLe4luPeO6Fzod03AADIGc5sAQAAAIAFhC0AAAAAsICwBQAAAAAWELYAAAAAwALCFgAAAABYQNgCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgAAAAALCFsAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbAAAAAGABYQsAAAAALCBsAQAAAIAFhC0AAAAAsICwBQAAAAAWFIqwNXnyZEVGRsrX11fNmzfX5s2b3V0SAAAAAFxWgQ9bn3zyiYYNG6Zx48Zp27ZtatCggTp06KDk5GR3lwYAAAAA2SrwYev111/XE088od69e6t27dqaOnWqSpYsqffff9/dpQEAAABAtrzcXcDlnD17Vlu3blV0dLRzzMPDQ1FRUdqwYUOW66SlpSktLc35PCUlRZKUmpqaqxoy0k7naj0A+SO3X9t54Vq+P1xr3e7cNwAAhdXFn4HGmHzZX4EOW3/++afS09MVEhLiMh4SEqKff/45y3ViY2MVExOTabxy5cpWagTgXgFx7q4gd9xZd2HtGQAAeeXIkSMKCAiwvp8CHbZyIzo6WsOGDXM+z8jI0NGjR1W+fHk5HI4rrp+amqrKlStr//798vf3t1lqoUFPskZfMqMnmdGTrNGXzOhJZvQka/QlM3qSNfqSWUpKisLDwxUYGJgv+yvQYatChQry9PTU4cOHXcYPHz6s0NDQLNfx8fGRj4+Py1jZsmWvet/+/v58Ul6CnmSNvmRGTzKjJ1mjL5nRk8zoSdboS2b0JGv0JTMPj/y5dUWBvkGGt7e3GjdurFWrVjnHMjIytGrVKrVo0cKNlQEAAADA5RXoM1uSNGzYMPXs2VNNmjRRs2bNFBcXp1OnTql3797uLg0AAAAAslXgw1a3bt30xx9/aOzYsTp06JAaNmyoZcuWZbppRl7x8fHRuHHjMl2KWJzRk6zRl8zoSWb0JGv0JTN6khk9yRp9yYyeZI2+ZJbfPXGY/LrvIQAAAAAUIwX6PVsAAAAAUFgRtgAAAADAAsIWAAAAAFhA2AIAAAAACwhbfzN58mRFRkbK19dXzZs31+bNm91dkjWxsbFq2rSpypQpo+DgYN1zzz3atWuXy5y//vpLAwYMUPny5VW6dGndf//9mf7A9G+//abOnTurZMmSCg4O1ogRI3T+/Pn8PBRrJkyYIIfDoaFDhzrHimtPDhw4oEceeUTly5eXn5+f6tWrpy1btjiXG2M0duxYVaxYUX5+foqKilJiYqLLNo4eParu3bvL399fZcuWVZ8+fXTy5Mn8PpQ8kZ6erjFjxqhKlSry8/NTtWrV9MILL+jv9xsqDj1Zu3atunTporCwMDkcDi1cuNBleV714LvvvtMtt9wiX19fVa5cWa+88ortQ8u1y/Xk3LlzGjlypOrVq6dSpUopLCxMjz76qH7//XeXbRSnnlyqf//+cjgciouLcxkvaj2RctaXn376SXfddZcCAgJUqlQpNW3aVL/99ptzeVH7mXSlnpw8eVIDBw5UpUqV5Ofnp9q1a2vq1Kkuc4paT/Lz9drq1avVqFEj+fj4qHr16oqPj7d9eLlypZ4cPXpUgwYNUs2aNeXn56fw8HANHjxYKSkpLtvJt54YGGOMmTNnjvH29jbvv/+++eGHH8wTTzxhypYtaw4fPuzu0qzo0KGDmTFjhtm5c6fZvn27ueOOO0x4eLg5efKkc07//v1N5cqVzapVq8yWLVvMTTfdZFq2bOlcfv78eVO3bl0TFRVlvv32W/P555+bChUqmOjoaHccUp7avHmziYyMNPXr1zdDhgxxjhfHnhw9etRERESYXr16mU2bNplff/3VLF++3OzZs8c5Z8KECSYgIMAsXLjQ7Nixw9x1112mSpUq5syZM845HTt2NA0aNDAbN24069atM9WrVzcPP/ywOw7pmr344oumfPny5rPPPjNJSUnm008/NaVLlzZvvPGGc05x6Mnnn39uRo8ebebPn28kmQULFrgsz4sepKSkmJCQENO9e3ezc+dOM3v2bOPn52emTZuWX4d5VS7Xk+PHj5uoqCjzySefmJ9//tls2LDBNGvWzDRu3NhlG8WpJ383f/5806BBAxMWFmYmTpzosqyo9cSYK/dlz549JjAw0IwYMcJs27bN7NmzxyxatMjldUlR+5l0pZ488cQTplq1aiYhIcEkJSWZadOmGU9PT7No0SLnnKLWk/x6vfbrr7+akiVLmmHDhpkff/zRTJo0yXh6epply5bl6/HmxJV68v3335v77rvPLF682OzZs8esWrXK1KhRw9x///3ObeRnTwhb/1+zZs3MgAEDnM/T09NNWFiYiY2NdWNV+Sc5OdlIMmvWrDHGXHhRUKJECfPpp5865/z0009GktmwYYMx5sI3RQ8PD3Po0CHnnClTphh/f3+TlpaWvweQh06cOGFq1KhhVq5cadq0aeMMW8W1JyNHjjQ333xztsszMjJMaGioefXVV51jx48fNz4+Pmb27NnGGGN+/PFHI8l88803zjlLly41DofDHDhwwF7xlnTu3Nk89thjLmP33Xef6d69uzGmePbk0hdGedWDt99+25QrV87l62fkyJGmZs2alo/o2l0uWFy0efNmI8ns27fPGFN8e/K///3PXHfddWbnzp0mIiLCJWwV9Z4Yk3VfunXrZh555JFs1ynqP5Oy6kmdOnXM888/7zLWqFEjM3r0aGNM0e+JMfZerz3zzDOmTp06Lvvq1q2b6dChg+1DumaX9iQrc+fONd7e3ubcuXPGmPztCZcRSjp79qy2bt2qqKgo55iHh4eioqK0YcMGN1aWfy6eWg0MDJQkbd26VefOnXPpyQ033KDw8HBnTzZs2KB69eq5/IHpDh06KDU1VT/88EM+Vp+3BgwYoM6dO7scu1R8e7J48WI1adJEDzzwgIKDg3XjjTfqnXfecS5PSkrSoUOHXPoSEBCg5s2bu/SlbNmyatKkiXNOVFSUPDw8tGnTpvw7mDzSsmVLrVq1Srt375Yk7dixQ1999ZU6deokqXj25FJ51YMNGzaodevW8vb2ds7p0KGDdu3apWPHjuXT0diTkpIih8OhsmXLSiqePcnIyFCPHj00YsQI1alTJ9Py4tqTJUuW6Prrr1eHDh0UHBys5s2bu1xWVxx/JrVs2VKLFy/WgQMHZIxRQkKCdu/erdtvv11S8eiJrddrGzZsyPS6p0OHDoXidfClPclujr+/v7y8vCTlb08IW5L+/PNPpaenuzRckkJCQnTo0CE3VZV/MjIyNHToULVq1Up169aVJB06dEje3t7OFwAX/b0nhw4dyrJnF5cVRnPmzNG2bdsUGxubaVlx7cmvv/6qKVOmqEaNGlq+fLmefPJJDR48WDNnzpT0f8d1ua+fQ4cOKTg42GW5l5eXAgMDC2VfRo0apYceekg33HCDSpQooRtvvFFDhw5V9+7dJRXPnlwqr3pQFL+mLvrrr780cuRIPfzww/L395dUPHvy8ssvy8vLS4MHD85yeXHsSXJysk6ePKkJEyaoY8eOWrFihe69917dd999WrNmjaTi+TNp0qRJql27tipVqiRvb2917NhRkydPVuvWrSUV/Z7YfL2W3ZzU1FSdOXPGxuHkiax6cqk///xTL7zwgvr27escy8+eeOV4JoqsAQMGaOfOnfrqq6/cXYpb7d+/X0OGDNHKlSvl6+vr7nIKjIyMDDVp0kQvvfSSJOnGG2/Uzp07NXXqVPXs2dPN1bnH3LlzNWvWLH388ceqU6eOtm/frqFDhyosLKzY9gRX59y5c3rwwQdljNGUKVPcXY7bbN26VW+88Ya2bdsmh8Ph7nIKjIyMDEnS3XffraefflqS1LBhQ3399deaOnWq2rRp487y3GbSpEnauHGjFi9erIiICK1du1YDBgxQWFhYpjMQRRGv1zK7Uk9SU1PVuXNn1a5dW88991z+Fvf/cWZLUoUKFeTp6Znpzi2HDx9WaGiom6rKHwMHDtRnn32mhIQEVapUyTkeGhqqs2fP6vjx4y7z/96T0NDQLHt2cVlhs3XrViUnJ6tRo0by8vKSl5eX1qxZozfffFNeXl4KCQkpdj2RpIoVK6p27douY7Vq1XLeEevicV3u6yc0NFTJyckuy8+fP6+jR48Wyr6MGDHCeXarXr166tGjh55++mnnGdHi2JNL5VUPiuLX1MWgtW/fPq1cudJ5Vksqfj1Zt26dkpOTFR4e7vy+u2/fPg0fPlyRkZGSil9PpAuvS7y8vK74vbc4/Uw6c+aMnn32Wb3++uvq0qWL6tevr4EDB6pbt27697//Lalo98T267Xs5vj7+8vPzy+vDydPZNeTi06cOKGOHTuqTJkyWrBggUqUKOFclp89IWxJ8vb2VuPGjbVq1SrnWEZGhlatWqUWLVq4sTJ7jDEaOHCgFixYoC+//FJVqlRxWd64cWOVKFHCpSe7du3Sb7/95uxJixYt9P3337v8ELz4wuHSHxCFwa233qrvv/9e27dvdz6aNGmi7t27O/9f3HoiSa1atcp0m9ndu3crIiJCklSlShWFhoa69CU1NVWbNm1y6cvx48e1detW55wvv/xSGRkZat68eT4cRd46ffq0PDxcv316eno6fxtdHHtyqbzqQYsWLbR27VqdO3fOOWflypWqWbOmypUrl09Hk3cuBq3ExER98cUXKl++vMvy4taTHj166LvvvnP5vhsWFqYRI0Zo+fLlkopfT6QLr0uaNm162e+9xe3n9Llz53Tu3LnLfu8tij3Jr9drLVq0cNnGxTkF8XXwlXoiXfh5c/vtt8vb21uLFy/OdMVSvvbkqm6nUYTNmTPH+Pj4mPj4ePPjjz+avn37mrJly7rcpaQoefLJJ01AQIBZvXq1OXjwoPNx+vRp55z+/fub8PBw8+WXX5otW7aYFi1amBYtWjiXX7xt5u233262b99uli1bZoKCggrs7VNz4+93IzSmePZk8+bNxsvLy7z44osmMTHRzJo1y5QsWdJ89NFHzjkTJkwwZcuWNYsWLTLfffedufvuu7O8xfeNN95oNm3aZL766itTo0aNQnWb87/r2bOnue6665y3fp8/f76pUKGCeeaZZ5xzikNPTpw4Yb799lvz7bffGknm9ddfN99++63zznp50YPjx4+bkJAQ06NHD7Nz504zZ84cU7JkyQJ7S+/L9eTs2bPmrrvuMpUqVTLbt293+d7797ugFaeeZOXSuxEaU/R6YsyV+zJ//nxTokQJM336dJOYmOi87fS6deuc2yhqP5Ou1JM2bdqYOnXqmISEBPPrr7+aGTNmGF9fX/P22287t1HUepJfr9cu3uZ8xIgR5qeffjKTJ08usLd+v1JPUlJSTPPmzU29evXMnj17XOacP3/eGJO/PSFs/c2kSZNMeHi48fb2Ns2aNTMbN250d0nWSMryMWPGDOecM2fOmKeeesqUK1fOlCxZ0tx7773m4MGDLtvZu3ev6dSpk/Hz8zMVKlQww4cPd95Wsyi4NGwV157897//NXXr1jU+Pj7mhhtuMNOnT3dZnpGRYcaMGWNCQkKMj4+PufXWW82uXbtc5hw5csQ8/PDDpnTp0sbf39/07t3bnDhxIj8PI8+kpqaaIUOGmPDwcOPr62uqVq1qRo8e7fKCuTj0JCEhIcvvIz179jTG5F0PduzYYW6++Wbj4+NjrrvuOjNhwoT8OsSrdrmeJCUlZfu9NyEhwbmN4tSTrGQVtopaT4zJWV/ee+89U716dePr62saNGhgFi5c6LKNovYz6Uo9OXjwoOnVq5cJCwszvr6+pmbNmua1114zGRkZzm0UtZ7k5+u1hIQE07BhQ+Pt7W2qVq3qso+C5Eo9ye7zSJJJSkpybie/euL4/0UDAAAAAPIQ79kCAAAAAAsIWwAAAABgAWELAAAAACwgbAEAAACABYQtAAAAALCAsAUAAAAAFhC2AAAAAMACwhYAAAAAWEDYAgBIktq2bauhQ4e6u4wCJT4+XmXLlnV3GU6RkZGKi4tzdxkAgBwibAEAip3Vq1fL4XDo+PHj7i4lSwUt5AEAcoewBQAAAAAWELYAoBg6deqUHn30UZUuXVoVK1bUa6+9lmnOsWPH9Oijj6pcuXIqWbKkOnXqpMTERJc569evV9u2bVWyZEmVK1dOHTp00LFjxyRlfclbw4YN9dxzzzmfOxwOTZs2TXfeeadKliypWrVqacOGDdqzZ4/atm2rUqVKqWXLlvrll19ctrNo0SI1atRIvr6+qlq1qmJiYnT+/HmX7b777ru69957VbJkSdWoUUOLFy+WJO3du1ft2rWTJJUrV04Oh0O9evXKce+uZd8XLV68WDVq1JCvr6/atWunmTNnOs+0rV69Wr1791ZKSoocDoccDodLz06fPq3HHntMZcqUUXh4uKZPn+5cdvbsWQ0cOFAVK1aUr6+vIiIiFBsbm+NjAwDkLcIWABRDI0aM0Jo1a7Ro0SKtWLFCq1ev1rZt21zm9OrVS1u2bNHixYu1YcMGGWN0xx136Ny5c5Kk7du369Zbb1Xt2rW1YcMGffXVV+rSpYvS09OvqpYXXnhBjz76qLZv364bbrhB//jHP9SvXz9FR0dry5YtMsZo4MCBzvnr1q3To48+qiFDhujHH3/UtGnTFB8frxdffNFluzExMXrwwQf13Xff6Y477lD37t119OhRVa5cWf/5z38kSbt27dLBgwf1xhtv5KjWa923JCUlJalr16665557tGPHDvXr10+jR492rtuyZUvFxcXJ399fBw8e1MGDB/XPf/7Tufy1115TkyZN9O233+qpp57Sk08+qV27dkmS3nzzTS1evFhz587Vrl27NGvWLEVGRub8gwEAyFsGAFCsnDhxwnh7e5u5c+c6x44cOWL8/PzMkCFDjDHG7N6920gy69evd875888/jZ+fn3O9hx9+2LRq1Srb/URERJiJEye6jDVo0MCMGzfO+VyS+de//uV8vmHDBiPJvPfee86x2bNnG19fX+fzW2+91bz00ksu2/3www9NxYoVs93uyZMnjSSzdOlSY4wxCQkJRpI5duxYtvUbY8yMGTNMQEBAnu575MiRpm7dui7bGD16tEs9l+73ooiICPPII484n2dkZJjg4GAzZcoUY4wxgwYNMu3btzcZGRmXPS4AQP7wclvKAwC4xS+//KKzZ8+qefPmzrHAwEDVrFnT+fynn36Sl5eXy5zy5curZs2a+umnnyRdOLP1wAMPXHM99evXd/4/JCREklSvXj2Xsb/++kupqany9/fXjh07tH79epezSenp6frrr790+vRplSxZMtN2S5UqJX9/fyUnJ19TrXmx7127dqlp06Yu223WrFmOa/j7th0Oh0JDQ53b7tWrl2677TbVrFlTHTt21J133qnbb7/96g8UAJAnCFsAgFzx8/O77HIPDw8ZY1zGLl6C+HclSpRw/t/hcGQ7lpGRIUk6efKkYmJidN9992Xalq+vb5bbvbidi9vILXfuOyfbbtSokZKSkrR06VJ98cUXevDBBxUVFaV58+blyb4BAFeHsAUAxUy1atVUokQJbdq0SeHh4ZIu3Axj9+7datOmjSSpVq1aOn/+vDZt2qSWLVtKko4cOaJdu3apdu3aki6cYVm1apViYmKy3E9QUJAOHjzofJ6amqqkpKRrrr9Ro0batWuXqlevnutteHt7S9JVv78sL/Zds2ZNff755y5j33zzTab6rra2i/z9/dWtWzd169ZNXbt2VceOHXX06FEFBgbmumYAQO4QtgCgmCldurT69OmjESNGqHz58goODtbo0aPl4fF/90yqUaOG7r77bj3xxBOaNm2aypQpo1GjRum6667T3XffLUmKjo5WvXr19NRTT6l///7y9vZWQkKCHnjgAVWoUEHt27dXfHy8unTporJly2rs2LHy9PS85vrHjh2rO++8U+Hh4eratas8PDy0Y8cO7dy5U+PHj8/RNiIiIuRwOPTZZ5/pjjvukJ+fn0qXLp0v++7Xr59ef/11jRw5Un369NH27dsVHx8v6f/O4kVGRurkyZNatWqVGjRooJIlSzovUbyc119/XRUrVtSNN94oDw8PffrppwoNDeVvdgGAm3A3QgAohl599VXdcsst6tKli6KionTzzTercePGLnNmzJihxo0b684771SLFi1kjNHnn3/uvIzt+uuv14oVK7Rjxw41a9ZMLVq00KJFi+TldeH3eNHR0WrTpo3uvPNOde7cWffcc4+qVat2zbV36NBBn332mVasWKGmTZvqpptu0sSJExUREZHjbVx33XWKiYnRqFGjFBIS4nK3Q9v7rlKliubNm6f58+erfv36mjJlivNuhD4+PpIu3JGwf//+6tatm4KCgvTKK6/kaNtlypTRK6+8oiZNmqhp06bau3evPv/8c5cgDQDIPw5z6QX1AAAgX7344ouaOnWq9u/f7+5SAAB5iMsIAQDIZ2+//baaNm2q8uXLa/369Xr11VdzfHYNAFB4ELYAAMhniYmJGj9+vI4eParw8HANHz5c0dHR7i4LAJDHuIwQAAAAACzgHbMAAAAAYAFhCwAAAAAsIGwBAAAAgAWELQAAAACwgLAFAAAAABYQtgAAAADAAsIWAAAAAFhA2AIAAAAAC/4fKMkgG8QeYpoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10, 6))   # make the plot larger\n",
    "plot_doc_lengths_hist(fig, ax, corpus_norm, y_log=False, bins=20)  # use 20 bins\n",
    "ax.set_xticks(range(0, 2201, 200))    # set x axis ticks and range\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also observe that there's now a much stronger deviation from the distribution predicted by Zipf's law. This is also expected, since we removed many high frequent token types (stopwords)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:50.605432Z",
     "iopub.status.busy": "2023-05-03T13:22:50.605297Z",
     "iopub.status.idle": "2023-05-03T13:22:50.707267Z",
     "shell.execute_reply": "2023-05-03T13:22:50.706780Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHHCAYAAABTMjf2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACOr0lEQVR4nOzdd3xT9frA8c/JTpruXcCy956yBBFFxIEDcV0Qt+LEydWr4OLnutd7Xej1Kg4UxC0OUJYyFGWoCLI3dI+0SZp5fn8cGyhtoS1t0/G8X+RFc3JO8pyM5ul3PF9FVVUVIYQQQohmSBfuAIQQQgghwkUSISGEEEI0W5IICSGEEKLZkkRICCGEEM2WJEJCCCGEaLYkERJCCCFEsyWJkBBCCCGaLUmEhBBCCNFsSSIkhBBCiGZLEiEhqmjkyJGMHDmyXh5r+/btnHXWWURHR6MoCp9++mm9PG5DpCgKt956a63eZ+vWrbn66qtr9T6FaOhGjhxJ9+7dwx1GgyOJUDP3+++/c8kll5Ceno7FYqFFixaceeaZvPDCC3X2mO+99x7PP/98ue2HDh1ixowZbNy4sc4eOxxcLhczZsxg+fLlVT5m8uTJ/P777zzxxBO888479O/fv+4CFGHXVN/74VSTz51oniQRasZWr15N//79+fXXX7n++ut58cUXue6669DpdPz73/+us8c9XiI0c+bMJvdl4HK5mDlzZpV/IbvdbtasWcO1117LrbfeylVXXUXLli3rNkgRVk31vR9O1f3ciebLEO4ARPg88cQTREdH8/PPPxMTE1PmtqysrPAEVQecTicRERHhDqPKsrOzAcq9JiejsT0HjUlJSQkmkwmdruH9XSmve+PTkN9PTZU8083Yzp076datW4VfuElJSeW2vfvuuwwcOBCbzUZsbCynnXYaixcvDt3+2WefMW7cONLS0jCbzbRr147HHnuMQCAQ2mfkyJF8+eWX7N27F0VRUBSF1q1bs3z5cgYMGADAlClTQrfNmTMndOxPP/3E2WefTXR0NDabjREjRrBq1aoyMc6YMQNFUdi8eTNXXHEFsbGxDBs2rNLnYM6cOSiKwvfff8+NN95IfHw8UVFRTJo0ifz8/BM+h1lZWVx77bUkJydjsVjo1asXb731Vuj2PXv2kJiYCMDMmTND5zVjxowK72/GjBmkp6cDcO+994aen1IbNmxg7NixREVFYbfbOeOMM/jxxx8rPKcVK1Zwyy23kJSUVGmLUmZmJgaDgZkzZ5a7bevWrSiKwosvvhjatmvXLiZMmEBcXBw2m41TTz2VL7/8styxJSUlzJgxg44dO2KxWEhNTeWiiy5i586doX2effZZhgwZQnx8PFarlX79+vHhhx9WGCfA3Llz6dSpExaLhX79+vH999+Xuf3qq68u81yVKn1PHE9eXh733HMPPXr0wG63ExUVxdixY/n111/L7Ld8+XIURWHevHk89NBDtGjRApvNxsaNG1EUhX/961/l7nv16tUoisL7779f4WPXx3u/devWnHvuuSxfvpz+/ftjtVrp0aNHqLXk448/pkePHqHndsOGDcd9vkoVFBRw11130bp1a8xmMy1btmTSpEnk5OSE9jnRZ+To5/XY1ps9e/aUey6uvvpq7HY7Bw8eZPz48djtdhITE7nnnntCv2uq+7n75ZdfUBSlXFwAixYtQlEUFi5cCEBRURF33nln6JyTkpI488wzWb9+fZWes4rO+9j3k8PhqPZ78oMPPuCJJ56gZcuWWCwWzjjjDHbs2HHCGBYvXozNZuPyyy/H7/dX+xyaAmkRasbS09NZs2YNmzZtOuEAupkzZzJjxgyGDBnCo48+islk4qeffmLp0qWcddZZgPYFbLfbmTZtGna7naVLl/Lwww/jcDh45plnAHjwwQcpLCzkwIEDoS8Nu91Oly5dePTRR3n44Ye54YYbGD58OABDhgwBYOnSpYwdO5Z+/frxyCOPoNPpePPNNxk1ahQ//PADAwcOLBPvhAkT6NChA08++SSqqp7wubj11luJiYlhxowZbN26lVdeeYW9e/eGfslUxO12M3LkSHbs2MGtt95KmzZtWLBgAVdffTUFBQXccccdJCYm8sorr3DzzTdz4YUXctFFFwHQs2fPCu/zoosuIiYmhrvuuovLL7+cc845B7vdDsAff/zB8OHDiYqK4r777sNoNPLqq68ycuRIVqxYwaBBg8rc1y233EJiYiIPP/wwTqezwsdLTk5mxIgRfPDBBzzyyCNlbps/fz56vZ4JEyYAWtI0ZMgQXC4Xt99+O/Hx8bz11lucf/75fPjhh1x44YUABAIBzj33XJYsWcJll13GHXfcQVFREd9++y2bNm2iXbt2APz73//m/PPP58orr8Tr9TJv3jwmTJjAwoULGTduXJlYVqxYwfz587n99tsxm828/PLLnH322axdu7ZWBn/u2rWLTz/9lAkTJtCmTRsyMzN59dVXGTFiBJs3byYtLa3M/o899hgmk4l77rkHj8dD586dGTp0KHPnzuWuu+4qs+/cuXOJjIzkggsuqPCx6+u9v2PHDq644gpuvPFGrrrqKp599lnOO+88Zs+ezd///nduueUWAGbNmsWll17K1q1bj9sqUVxczPDhw9myZQvXXHMNffv2JScnh88//5wDBw6QkJBQpc9ITQQCAcaMGcOgQYN49tln+e6773juuedo164dN998c7U/d/3796dt27Z88MEHTJ48ucxt8+fPJzY2ljFjxgBw00038eGHH3LrrbfStWtXcnNzWblyJVu2bKFv3741Op9j308mk4nNmzdX6z35f//3f+h0Ou655x4KCwt5+umnufLKK/npp58qfdyFCxdyySWXMHHiRN544w30en2N4m/0VNFsLV68WNXr9aper1cHDx6s3nfffeqiRYtUr9dbZr/t27erOp1OvfDCC9VAIFDmtmAwGPrZ5XKVe4wbb7xRtdlsaklJSWjbuHHj1PT09HL7/vzzzyqgvvnmm+Ueo0OHDuqYMWPKPV6bNm3UM888M7TtkUceUQH18ssvr9Jz8Oabb6qA2q9fvzLn/fTTT6uA+tlnn4W2jRgxQh0xYkTo+vPPP68C6rvvvhva5vV61cGDB6t2u111OByqqqpqdna2CqiPPPJIlWLavXu3CqjPPPNMme3jx49XTSaTunPnztC2Q4cOqZGRkeppp51W7pyGDRum+v3+Ez7eq6++qgLq77//XmZ7165d1VGjRoWu33nnnSqg/vDDD6FtRUVFaps2bdTWrVuH3htvvPGGCqj//Oc/yz3W8d4vXq9X7d69e5nHVFVVBVRA/eWXX0Lb9u7dq1osFvXCCy8MbZs8eXKF76vS98TR0tPT1cmTJ4eul5SUlHtv7969WzWbzeqjjz4a2rZs2TIVUNu2bVsu/tLnccuWLWXOKSEhocxjVaSu3/vp6ekqoK5evTq0bdGiRSqgWq1Wde/eveXOY9myZceN+eGHH1YB9eOPPy53W2msVf2MlD6vxz5m6Wfh6Odl8uTJKlDmdVFVVe3Tp4/ar1+/0PXqfu6mT5+uGo1GNS8vL7TN4/GoMTEx6jXXXBPaFh0drU6dOrVK93kix3s/Vfc92aVLF9Xj8YS2//vf/y73uR4xYoTarVs3VVVV9aOPPlKNRqN6/fXXl3uc5ka6xpqxM888kzVr1nD++efz66+/8vTTTzNmzBhatGjB559/Htrv008/JRgM8vDDD5f7C/Ho1hKr1Rr6uaioiJycHIYPH47L5eLPP/+scZwbN25k+/btXHHFFeTm5pKTk0NOTg5Op5MzzjiD77//nmAwWOaYm266qVqPccMNN2A0GkPXb775ZgwGA1999VWlx3z11VekpKRw+eWXh7YZjUZuv/12iouLWbFiRbViOJ5AIMDixYsZP348bdu2DW1PTU3liiuuYOXKlTgcjjLHXH/99VX6C++iiy7CYDAwf/780LZNmzaxefNmJk6cGNr21VdfMXDgwDJdjXa7nRtuuIE9e/awefNmAD766CMSEhK47bbbyj1WZe+X/Px8CgsLGT58eIVdDIMHD6Zfv36h66eccgoXXHABixYtKtP1WlNmszn03g4EAuTm5mK32+nUqVOF8UyePLlM/ACXXnopFouFuXPnhrYtWrSInJwcrrrqqhrFVZvv/a5duzJ48ODQ9dIWxFGjRnHKKaeU275r167jxvbRRx/Rq1evUEvg0Upf57r8jBx7nsOHDz9hzMczceJEfD4fH3/8cWjb4sWLKSgoKPM5iImJ4aeffuLQoUM1fqxjVfR+qu57csqUKZhMptD10pbFip6T999/n4kTJ3LjjTfy6quvNvvxSM377AUDBgzg448/Jj8/n7Vr1zJ9+nSKioq45JJLQl9sO3fuRKfT0bVr1+Pe1x9//MGFF15IdHQ0UVFRJCYmhr4ACgsLaxzj9u3bAe2XRWJiYpnL66+/jsfjKXf/bdq0qdZjdOjQocx1u91Oamoqe/bsqfSYvXv30qFDh3K/RLp06RK6vbZkZ2fjcrno1KlTudu6dOlCMBhk//79ZbZX9TlISEjgjDPO4IMPPghtmz9/PgaDIdSlANr5VPb4pbeD9n7p1KkTBsPxe94XLlzIqaeeisViIS4uLtSdUdF75djXB6Bjx464XK7Q4PKTEQwG+de//kWHDh0wm80kJCSQmJjIb7/9VmE8FT23MTExnHfeebz33nuhbXPnzqVFixaMGjWqRnHV5nv/6GQHIDo6GoBWrVpVuP1EY+R27tx5wm7JuvqMWCyW0BigUrGxsVUa11eZXr160blz5zJ/EMyfP5+EhIQyr9/TTz/Npk2baNWqFQMHDmTGjBknlYBBxa9Zdd+Tx76+sbGxQPnXcffu3Vx11VVcfPHFvPDCCyccP9ccyBghAYDJZGLAgAEMGDCAjh07MmXKFBYsWFBu3EhlCgoKGDFiBFFRUTz66KO0a9cOi8XC+vXruf/++8v91Vodpcc+88wz9O7du8J9SsfRlDr2r6vmqDrPwWWXXcaUKVPYuHEjvXv35oMPPuCMM84gISGhTmL74YcfOP/88znttNN4+eWXSU1NxWg08uabb5ZJJKqjsl/oVWkxevLJJ/nHP/7BNddcw2OPPUZcXBw6nY4777yzwvduZc/tpEmTWLBgAatXr6ZHjx58/vnn3HLLLTX+i7s23/uVtQ5Wtl2twti62lLd166uxrJMnDiRJ554gpycHCIjI/n888+5/PLLyyT1l156KcOHD+eTTz5h8eLFPPPMMzz11FN8/PHHjB07tkaPW9FrVt33ZFVfx9TUVFJTU/nqq6/45ZdfpEYZkgiJCpR+MA4fPgxAu3btCAaDbN68udJfxsuXLyc3N5ePP/6Y0047LbR99+7d5fat7JdeZdtLB9dGRUUxevToKp9HdWzfvp3TTz89dL24uJjDhw9zzjnnVHpMeno6v/32G8FgsMwXXWk3YOnsr9r4iysxMRGbzcbWrVvL3fbnn3+i0+nK/WVfHePHj+fGG28M/TW8bds2pk+fXmaf9PT0Sh+/9HbQXq+ffvoJn89XprvxaB999BEWi4VFixZhNptD2998880K9y9tGTnatm3bsNlsoZaB2NhYCgoKyu1XlVaHDz/8kNNPP53//e9/ZbYXFBRUKxk8++yzSUxMZO7cuQwaNAiXy8Xf/va3Ex4Xzvd+TbVr145NmzYdd5+qfkZKWy+Off1OplW1Jp+7iRMnMnPmTD766COSk5NxOBxcdtll5fZLTU3llltu4ZZbbiErK4u+ffvyxBNP1DgRqkhtvSePZbFYWLhwIaNGjeLss89mxYoVdOvW7WTDbdSka6wZW7ZsWYV/9ZWOiyntBhk/fjw6nY5HH3203F8ipceX/jVy9P15vV5efvnlcvcfERFRYdNuab2TY38Z9uvXj3bt2vHss89SXFxc7rja6Bp57bXX8Pl8oeuvvPIKfr//uL/YzjnnHDIyMso0pfv9fl544QXsdjsjRowAwGazAeXPqzr0ej1nnXUWn332WZnuuszMTN577z2GDRtGVFRUje8/JiaGMWPG8MEHHzBv3jxMJhPjx48vs88555zD2rVrWbNmTWib0+nktddeo3Xr1qGu04svvpicnJwy0+5LHf1+URSlzF/8e/bsqXQpkTVr1pQZF7F//34+++wzzjrrrNB7r127dhQWFvLbb7+F9jt8+DCffPLJCc9fr9eX+ywsWLCAgwcPnvDYoxkMBi6//HI++OAD5syZQ48ePSqdqXS0cL73a+riiy/m119/rfD5LX0uq/oZSU9PR6/XlyuJUNHvj6qqyeeuS5cu9OjRg/nz5zN//nxSU1PL/GEXCATK/e5KSkoiLS0Nj8cT2paTk8Off/6Jy+Wqcfy19Z6sSHR0NIsWLQpN/T+6rEVzJC1Czdhtt92Gy+XiwgsvpHPnzni9XlavXs38+fNp3bo1U6ZMAaB9+/Y8+OCDPPbYYwwfPpyLLroIs9nMzz//TFpaGrNmzWLIkCHExsYyefJkbr/9dhRF4Z133qkw0erXrx/z589n2rRpDBgwALvdznnnnUe7du2IiYlh9uzZREZGEhERwaBBg2jTpg2vv/46Y8eOpVu3bkyZMoUWLVpw8OBBli1bRlRUFF988cVJPRder5czzjgjNG345ZdfZtiwYZx//vmVHnPDDTfw6quvcvXVV7Nu3Tpat27Nhx9+yKpVq3j++eeJjIwEtGbvrl27Mn/+fDp27EhcXBzdu3ev9rTvxx9/nG+//ZZhw4Zxyy23YDAYePXVV/F4PDz99NMndf6g/TV81VVX8fLLLzNmzJhy9aUeeOAB3n//fcaOHcvtt99OXFwcb731Frt37+ajjz4K/cU/adIk3n77baZNm8batWsZPnw4TqeT7777jltuuYULLriAcePG8c9//pOzzz6bK664gqysLF566SXat29fJpEp1b17d8aMGVNm+jxQpv7RZZddxv3338+FF17I7bffjsvl4pVXXqFjx44nrPFy7rnn8uijjzJlyhSGDBnC77//zty5c8sMTK+qSZMm8Z///Idly5bx1FNPVemYcL73a+ree+/lww8/ZMKECVxzzTX069ePvLw8Pv/8c2bPnk2vXr2q/BmJjo5mwoQJoTEr7dq1Y+HChSdV2LWmn7uJEyfy8MMPY7FYuPbaa8u0ZBUVFdGyZUsuueQSevXqhd1u57vvvuPnn3/mueeeC+334osvMnPmTJYtW1bj9Qlr8z1ZkYSEhNDvk9GjR7Ny5UpatGhRK/fd6IRrupoIv6+//lq95ppr1M6dO6t2u101mUxq+/bt1dtuu03NzMwst/8bb7yh9unTRzWbzWpsbKw6YsQI9dtvvw3dvmrVKvXUU09VrVarmpaWFpqOzzHTYouLi9UrrrhCjYmJUYEyU54/++wztWvXrqrBYCg3bXbDhg3qRRddpMbHx6tms1lNT09XL730UnXJkiWhfUqnEGdnZ1fpOSidar5ixQr1hhtuUGNjY1W73a5eeeWVam5ubpl9j50+r6qqmpmZqU6ZMkVNSEhQTSaT2qNHj3JToFVVVVevXq3269dPNZlMJ5zSW9n0eVVV1fXr16tjxoxR7Xa7arPZ1NNPP73MlOijz+nnn3+u0nNQyuFwqFartdx056Pt3LlTveSSS9SYmBjVYrGoAwcOVBcuXFhuP5fLpT744INqmzZtVKPRqKakpKiXXHJJman///vf/9QOHTqoZrNZ7dy5s/rmm29WONUdUKdOnaq+++67of379OlT4fTuxYsXq927d1dNJpPaqVMn9d13363y9Pm7775bTU1NVa1Wqzp06FB1zZo15V7z0qnKCxYsOO5z2a1bN1Wn06kHDhw47n5Hq8v3fnp6ujpu3Lhy20uf26Md7/13rNzcXPXWW29VW7RooZpMJrVly5bq5MmT1ZycnNA+Vf2MZGdnqxdffLFqs9nU2NhY9cYbb1Q3bdpU4fT5iIiIcsdX9DpX53NXavv27aGSDStXrixzm8fjUe+99161V69eamRkpBoREaH26tVLffnllyuM5UQlCI73fjrZ92RFpQeOnj5faseOHWpqaqrapUuXKv/ebGoUVa3HEXFCNDBz5sxhypQp/PzzzzJoUNSaPn36EBcXx5IlS8IdihDiBGSMkBBC1KJffvmFjRs3MmnSpHCHIoSoAhkjJIQQtWDTpk2sW7eO5557jtTU1DJF+IQQDZe0CAkhRC348MMPmTJlCj6fj/fffx+LxRLukIQQVSBjhIQQQgjRbEmLkBBCCCGaLUmEhBBCCNFsyWDpEwgGgxw6dIjIyEhZnE4IIYRoJFRVpaioiLS0tOOu9yeJ0AkcOnTopNZwEkIIIUT47N+/n5YtW1Z6uyRCJ1BaAn7//v0ntZaTEEIIIeqPw+GgVatWoe/xykgidAKl3WFRUVGSCAkhhBCNzImGtchgaSGEEEI0W5IICSGEEKLZkkRICCGEEM2WJEJCCCGEaLYkERJCCCFEsyWJkBBCCCGaLUmEhBBCCNFsSSIkhBBCiGZLEiEhhBBCNFuSCAkhhBCi2ZIlNsLA4wuQ6/TiCwQx6nXER5gwG/XhDksIIYRodiQRqme5xR52ZRfj9AQIBFX0OoUIs562iXbi7eZwhyeEEEI0K9I1Vo88vgC7sospdPvxB1VUwB9UKXT72ZVdjMcXCHeIQgghRLMiiVA9ynV6cXqOJDuxP62k86P3oStx4/Ro3WVCCCGEqD/SNVaPfIEggaAKgK7ETbcHbsWSlUH0hl/Y9M/X8CX2Dm+AQgghRDMjLUL1yKjXodcpAAQtVjY/8W+8cfFEbtvMwEvPIurD+WGOUAghhGheJBGqR/ERJiLMR2aH5Q0ZwY8fLyVv4FD0LhexN10L114LLlcYoxRCCCGaD0mE6pHZqM0Oi7YaMOgUFCCYnMLOuR/j+vtDoCjwxhswYADk54c7XCGEEKLJkzFC9SzebsZuNpSvI/TEYzB6FFxxBfTqBTEx4Q5VCCGEaPIkEQoDs1FPWoy1/A2nnw6//goWi9Y6BFBQAAYD2O31GqMQQgjRHEjXWEOTlARRUdrPqgpXXw39+8Nvv4U1LCGEEKIpkkSoITt4EH75BbZuhYED4bXXtORICCGEELVCEqGGrGVL2LgRxo4FjwduvFEbQ+RwhDsyIYQQokmQRKihS0iAhQvh6adBr4d586BfP9iwIdyRCSGEEI2eJEKNgU4H994LP/wArVrBjh0wcSIEZG0yIYQQ4mQ0mkRo1qxZDBgwgMjISJKSkhg/fjxbt2497jFz5sxBUZQyF4vFUk8R14HBg7Wusosugrfe0lqIhBBCCFFjjSYRWrFiBVOnTuXHH3/k22+/xefzcdZZZ+F0Oo97XFRUFIcPHw5d9u7dW08R15G4OPjoIy0pKvXBB/Dzz+GLSQghhGikGk0doW+++abM9Tlz5pCUlMS6des47bTTKj1OURRSUlLqOrzw+eMPbYq936+NI7rjjiM1iIQQQghxXI2mRehYhYWFAMTFxR13v+LiYtLT02nVqhUXXHABf/zxx3H393g8OByOMpcGrUULOOcc8Pngrrtg/HjIy6vWXXh8AQ4VuNmb6+RQgRuPT8YeCSGEaB4aZSIUDAa58847GTp0KN27d690v06dOvHGG2/w2Wef8e677xIMBhkyZAgHDhyo9JhZs2YRHR0durRq1aouTqH2xMTAggXw4otgMsHnn0OfPrBmTZUOzy328NuBArZlFLEjs5htGUX8dqCA3GJP3cYthBBCNACKqja+Cn0333wzX3/9NStXrqRly5ZVPs7n89GlSxcuv/xyHnvssQr38Xg8eDxHkgCHw0GrVq0oLCwkqrTic0O1fj1ceins3KkNpP6//4N77ql0d48vwG8HCih0+8vdFm010LNlDGajDMgWQgjR+DgcDqKjo0/4/d3oWoRuvfVWFi5cyLJly6qVBAEYjUb69OnDjh07Kt3HbDYTFRVV5tJo9O2rJUOlU+uLio67e67Ti9NTcTeY0xMg1+mtiyiFEEKIBqPRJEKqqnLrrbfyySefsHTpUtq0aVPt+wgEAvz++++kpqbWQYQNRFQUvP8+fPIJPPzwke3+8q0+vkCQQLDiBsFAUMUXCNZVlEIIIUSD0GgSoalTp/Luu+/y3nvvERkZSUZGBhkZGbjd7tA+kyZNYvr06aHrjz76KIsXL2bXrl2sX7+eq666ir1793LdddeF4xTqj6Jog6ZL6wyVlGjT7Z98EoJHkhujXodeV/EMM71OwahvNG8PIYQQokYazfT5V155BYCRI0eW2f7mm29y9dVXA7Bv3z50uiNf3vn5+Vx//fVkZGQQGxtLv379WL16NV27dq2vsBuGDz7QFm/95RdYvhzefReSkoiPMBFh1lc4RijCrCc+wlT/sQohhBD1qFEOlq5PVR1s1aCpKsyZA1OngtsNKSnw3ntw+unkFnvYlV2M0xMgEFTR6xQizHraJdmJizCHO3IhhBCiRprsYGlRA4oCU6Zo1ae7doWMDBg9GmbOJP6v2WEdUyJpn2ynY0okPVvGSBIkhBCiWZBEqDnp1g3WrtWSomAQZsyAe+7BbNSTFmMlPT6CtBirTJkXQgjRbEgi1NxERMAbb8Dbb0PLltqSHEIIIUQzJYlQc/W3v8GOHdC69ZFtixdXOM2+JmTZDiGEEI1Bo5k1JuqA+ahxQF9/ra1Zdtpp2kDqFi1qfLeVDcBum2gn3i5jj4QQQjQc0iIkNG43REbC999D795aYlQDHl+AXdnFFLr9+IMqKuAPqhS6/ezKLpaWISGEEA2KJEJCc9FFsG6dtmBrTo7WOnT//dqq9tUgy3YIIYRoTCQREkd06ACrV2v1hgCefhpGjIB9+6p8F7JshxBCiMZEEiFRlsUCL74IH34I0dGwZg2sWlXlw2XZDiGEEI2JDJYWFbv4Yq2bbMECuPzyKh8my3YIIYRoTOTPc1G5tm21cUKlsrLgggtg9+5KDzEbtdlh0VYDBp2CAhh0CtFWA+2S7FKsUQghRIMiLUKi6m6/HT7/HFas0IoyXnRRhbvF283YzQZynV58gSBGvY74CJMkQUIIIRocaRESVffUU3DqqVBYqHWd3XYbeDwV7noyy3ZIMUYhhBD1RRIhUXXp6VqdoXvv1a6/+CIMGaJVqK4lucUefjtQwLaMInZkFrMto4jfDhSQW1xxwiWEEEKcDEmERPUYjdq0+oULIT4e1q+Hvn2rNbOsMlKMUQghRH2TREjUzLhxsHEjDBsGycnQo8dJ36UUYxRCCFHfZLC0qLmWLWHZMjh0CKKitG2qCgcOQKtW1b47KcYohBCivkmLkDg5BgOccsqR6y++CF26wLvvVvuupBijEEKI+ibfLKL2qCp8+SU4nfC3v8G114LLVeXDS4sxVkSKMQohhKgLkgiJ2qMoWiL0yCPaz2+8AQMHwubNVTpcijEKIYSob4qqqhUPyhAAOBwOoqOjKSwsJKp0HIw4saVL4corISMDrFZ4+WW4+uoqHerxBaQYoxBCiJNS1e9vaRESdWPUKG1W2ejR4HbDddfBtm1VOvRkijEKIYQQ1SGzxkTdSU6Gb76B//s/MJuhY8dwRySEEEKUIYmQqFt6PTz4YNltf/wBq1drrURKxbPEqkq60YQQQpwMSYRE/XK74dJLtQHUS5fCq68eqUFUTbnFHnZlF+P0BAgEVfQ6hQizNuA63m6u5cCFEEI0RTJGSNQvsxkmT9ZaiubNg379YMOGat+NLMchhBCiNkgiJOqXTgf33act3tqqlbZg66mnarPKqjGBUZbjEEIIURskERLhMWSINqvsvPPA64WpU7UuM6ezSofLchxCCCFqgyRCInzi4uCzz+Cf/9RWtc/K0rrOqkCW4xBCCFEbZLC0CC9FgbvugqFDoUULbe0yAJ9P+7mSWWWly3EUuv3lbpPlOIQQQlSV/NksGoaBA7VEqNS0aXDhhZCXV+HushyHEEKI2iAtQqLh2bMHXntNGzvUpw/Mn68NqD5GvN2M3WyQOkJCCCFqTFqERMPTurVWcLFdO9i3D4YPh2efhWD5AdBVXY7D4wtwqMDN3lwnhwrcMr1eCCEE0IgSoVmzZjFgwAAiIyNJSkpi/PjxbN269YTHLViwgM6dO2OxWOjRowdfffVVPUQrTlq/frBunTaTzO+He++F88+HnJxq31VusYffDhSwLaOIHZnFbMso4rcDBeQWe0L7SKIkhBDNU6NJhFasWMHUqVP58ccf+fbbb/H5fJx11lk4jzPdevXq1Vx++eVce+21bNiwgfHjxzN+/Hg2bdpUj5GLGouO1oouvvKKNpvsyy/h9NMrbBmqTFUKL54oUaosSZLkSQghGj9FVatRxa4Byc7OJikpiRUrVnDaaadVuM/EiRNxOp0sXLgwtO3UU0+ld+/ezJ49u0qP43A4iI6OprCwkKgaLgUhasGvv8LEiTBrljaIuooOFbjZllGEv4KaQwadQseUSA7muyqcfZZgN5IaY2VfrqvMMh4xNgOxEWayHCWyvIcQQjRQVf3+bjQtQscqLCwEIC4urtJ91qxZw+jRo8tsGzNmDGvWrKnT2EQd6NULfvutbBL0449a7aHjOF7hRUWBPKen0grVvoDK7wcd5VqTSvxB1u3JI6fYK8t7CCFEI9coE6FgMMidd97J0KFD6d69e6X7ZWRkkJycXGZbcnIyGRkZlR7j8XhwOBxlLqKBMB1VG+jQIa0qde/esGJFpYccr/CiUa8jqFJhomTUKxR7/OQfs1SHUa/g9gbIcHgoKinfiiTLewghROPSKBOhqVOnsmnTJubNm1fr9z1r1iyio6NDl1atWtX6Y4haUFQEiYlw+DCMGgWPPgqB8i0xpYUXK2Ix6oixGitMlHSKoiVIavntvoBKiS+Av4KxSrK8hxBCNC6NLhG69dZbWbhwIcuWLaNly5bH3TclJYXMzMwy2zIzM0lJSan0mOnTp1NYWBi67N+/v1biFrWsUyf4+WeYMkUbPP3II3DWWXBMa9/xCi+2ToggOcpSYaIUVFXsZgMxNmO57Ua9gsWox6Ar//HR6xRMhkb3sRJCiGar0fzGVlWVW2+9lU8++YSlS5fSpk2bEx4zePBglixZUmbbt99+y+DBgys9xmw2ExUVVeYiGqiICHjjDXj7bbDZYOlSbSzRd9+V2S3ebqZnyxg6pkTSPtlOx5RIeraMIS7CXGmiZDPpaZ9sL5cI+QIqVpOelCgzkZay9UhNBh3xEUZ8/qDMJBNCiEai0VSWnjp1Ku+99x6fffYZkZGRoXE+0dHRWK1WACZNmkSLFi2YNWsWAHfccQcjRozgueeeY9y4ccybN49ffvmF1157LWznIerA3/4GAwZoNYd+/x3efx+OGSRfWnixIserUK2gsCu7uMzsMItBR//WcWQeNWvMatRjNirkurzkOn1lZpi1iLXhC6hS/VoIIRqgRjN9Xqlk8c0333yTq6++GoCRI0fSunVr5syZE7p9wYIFPPTQQ+zZs4cOHTrw9NNPc84551T5cWX6fCPidsNTT2nFFyMiau1uPb5AhUnSsdv35jrLzEAzGXQoQHZRCZEWIzpFkWn2QghRT6r6/d1oEqFwkUSoEQsG4eqr4Yor4Oyz6/ShKqpXZLfo2ZnlJKvIQ4sYK3ER2qw3k0FHrNVAtM2ECtJKJIQQdaCq39+NpmtMiGp74w145x3tcv/98NhjYDSe+LgaOLZeUek0+zynl2BQDc0wK20l2nSoCItRR5TFiNWoJ7eoBIvJgMmgw2zQYTcZcHj80p0mhBB1TBIh0XRddRVs2AAvv6x1mf3wg7ZkRx2URCitV1TaIqRTFDz+IN5AEJ1OCc0wMxkUdmY5ySgsITXGQrzdTEBV+e2ggxJfgLYJEZiNOjIKSzAZdFiNBulOE0KIOtRoZo0JUW0WC7z0EixYAFFR2or2vXvDF1/U+kMdW6+odJq9Sa/DYtARaTGUaSUqTY5MBoV9eS4OF5bgKPETCMLuHBe7clzsz3PjDQTxB1Vc3gD781wyC00IIWqZJEKi6bvkEq1lqH9/yMvTVrF/5plafYhjp+H7A1odohYxFlJiLBj1ulAxRm8giMWgIy7CGEqMAEx6BafPH7pe4g/iCwSxW/QEVRVHiY99eS725TrZm+sko9BNsdsnC78KIcRJkK4x0Ty0bQsrV8IDD2ir2Z9xRq0/xLHT8M0GHafE20KLtgZVFbNeq2YdZzdhNuhx+wJ4/6pEbTToUVU1dN1q0LrbdmY5KfEFsJn0bM0oRqdAWrSFaJuxTBea2aAjy2LglDgbcdKFJoQQVSKJkGg+zGb417/gzjshPf3I9r17y14/mYeooF5RjNVUZpq9yagLJUal3WcefxCDAnazIXQ9JsLEoYISCt0+0mIsHCxwU1wSwG4xEGszkp3lJavIQ6LdRPskM95AgKwiDyW+AN31OuzWuhkYLoQQTYkkQqL5OTrpWb8ehgyBG27QusvMtd+ScmxyZDboQkUabSY9CXYzBS4vsXYTdouBuAgT+S4vCipOjw+LUUcgoFLg9BFpNWAx6nD5ghwqcBNtNWI26th8WBtsbdQrJEWacfsCtIixYrcY8QeCqGgz1uJsMvtMCCGOJomQaN6WLQOPB154QRtMPX8+tGtXpw95dBdaMKiSEGkhy1GCxxfE51dpk2Aj1mUEBQ4WlGAy6NHpFCxmHXqdDoNOhz8QxO3zc0q8jf15bhwlPtKirURajOzMdmLKd5Hn9OL2BdChkBZjQUHBbNSTFGXGpNeholLiC2qDuH1BTHoFq8kgU/WFEM2KJEKiebv7bujcGSZNgnXroE8feP11bbmOOnRsK1HLGGuZsUV2k4H9BW6cngBGnQ67RU+e04vHH8QfDGLQG4i1mQkEVLKLPNhMBqJtRg7ku9mf56JPqxh25zhBVUiNsbBxfyE2kw67xcgfhwqIizDj8QUw6HUcLHATDAaxmgy0jLWSHGWRqfpCiGZDZo0JMW4cbNwIQ4dCURFMnAg33wwlJfUWQmlilB4fQUq0FbvVSNuECFrH27BbDKEuM4ASXxCbUUeLWCt+VUVVVWIjjAQCKllFHmKsRvR6hUyHhxibgYMFbnZmF2PQ6TiQ76bIE+CPQw5ynV52ZBezLbOYA/naWKQsh4eiEj97c50yA00I0SxIIiQEaEUWly+H6dO167Nna6vah9HRU/KDAUiPs9EixkqszYjVpCc9zkpypJm0GAsWg54gKoqqEh9pIqiCQQcqkFXkwWbU4w+qFHv8EIRCtwdfQOVgnhsAi1FHot2MyajHUeLD7QmQ7/KG9fyFEKI+SNeYEKUMBnjySRgxAt59F667LtwRlRtP1DbJHhr8bDbo6Jgchcmop8DlxWLQ0yLWikGnw6AoxFhN6BUFjy9AtNWIqkIwqBJEBVUhqKo4fT6SI7XusAMFWlLUIsaC2xug2OvH6fHTMtYmY4aEEE2WJEJCHGvMGO1SyuXSlui4/36w2eo9nIqm5B+tT6sYdmUXA+Bw+8gs8hBh1tMyzkowqB2vABaTDp1OwahXsBi1Ao8RRiNJURYOO0rweIO0SbBxuNCDqpbQKtaKLxAk3+XFatQTYTaEBlXbzAaZgSaEaBIkERLiRO64QxtA/fHH8MEH0KVLuCMqo7TVKN/lJdpqZF++G6NOQa8oOL0B0t1eCt0+zAY9cTYTkRYDbp8Jo16hXbIdNQjZjhK6pkaRVeTBGwiSHm+jyBugJN/NziwnkRYDjhIfNqOOxCgLsREmYiNMmHU6gqjoFIVAUMVm0mM1GYiVJEkI0UgoqqqqJ96t+XI4HERHR1NYWEhUVFS4wxHhsGQJXHklZGZqLUIvvwyTJ4c7qkp5fIFQV5pBr+Bw+9id6yQQUIkwGchzeVEUKPEGsFuM5Dk9bDlcSFqsjV3ZTton2ilw+7CZDGQ5SoiyGnCU+PF4g7RPjiCgQpzViArkFGsz1jyBIBFGPWajjlibibgIE+2S7CRGWsL9dAghmqmqfn9Li5AQJ3LGGdqssquu0pKiq6/W6g+99BJERIQ7unKO7UpLidZmo5UmRx10CsGgioqK1x/E7bVhNxsJolLiCWK3GMgs8mAzgqPER6zNxOF8N51SI9mf7ybGYsKkV9ieWUyE2UCh24dRr+NAnovcYg/RViMdku0Ulfjokx5Dgr3ybj0hhAg3mTUmRFWkpMCiRfDYY6DTwVtvaYu4/vlnuCOrktLkqGWcLfR/q7gI2iVF0iE5ktYJNuIjTKEZZ6WDqhVAp4Ber6BXdDhcPswmhXynl3y3F71OAQUOFbrZcrgIRafQLslOaoyVIk+AjfsK2Z1dLFPxhRANlrQICVFVej089BAMHw5XXAE5ORAZGe6oTprZqCc9PoJD+S7ynD5cHj8GnYJJrxBpMaDX6Yi1GdEpEFBVDIoOb8CHGtRmrrm8AQ7muYm2GeiaEo0KrN6ZS1ZhCTaTnt6nxNKrVTQdkiKJjZAijUKIhkUSISGqa8QIrats505o0eLIdq8XTKawhXUySgdc2yxGshwl+IJBzH8t/moyKKTF2LQlOgw6dDoV61/jgax/1Sdy+QJ0TYtG0alsOuDgYEEJydFmuqRGkxRp5nCBB48vSHp8BMlRFhlILYRoMCQREqImEhO1S6lPPoEHHtDWKuvdO2xhnQyzUU+7RDstY6wkRVnYl+ckKdpCgdOHokBQVQmoAVQUUqKtONx+FAXibCZSoixEWQ0UunxkFXlIjDTRPS0ao0Hhj0MO9ArERhgp8vjZnVNM8l8zz2KsMrtMCBFekggJcbKCQZgxA7Ztg1NPhX/9C266CRQl3JHVyNEJUb5LW/+sxBfEYtDRNS2aLIcbvV5HjM3E4QI3KdFmBrSJA1XF4w+AAunxERj1CvtzXZiMeoq9fvwqHCrMJS3SQokvQL7LB6qTGKsBk1FHUNVmvLl8QYx6HfERJmk9EkLUOZk+fwIyfV5USW6uNpts4ULt+oQJ8N//QnR0WMOqCx6ftvxGIKji8gZQ1SBBFQ4VuNmb5+LPQ0W0TrBR7PHj8Qc5XFiC1aRHrygk2M3E2oyU+AJkF3tIibJgtxgwKDrcPj+7clzkFJeQFGmma2oUbRPttE+KJE4WgBVCVJNMnxeiPsXHw+efa61B998PCxZoq9nPn6/NLmtCzEY9KdHlp8S3jLFhMxswKAqgLeHh9gVxuH1EWY0YdQoGvUJusZfNGQ5ibSaiLAEcJX4AtmcV4/T46dMqhpQoCwEVsoq8WExuzHodEVZjPZ+pEKI5kERIiNqiKDBtmraK/cSJsGuX9vOff0KbNuGOrs7ZrUY6JkdiMej+WqZDDY0jirEaQQWPL0hhiZcit5fW8RGU+AMEgyrFngAOt5eeLWMw6nVsznAQYzUSiAri8vrILfbQJt5GaoyseyaEqF2SCAlR2wYNgg0b4JprtFllzSAJKhUXYSbCZCApysue7GIMBh35Ls9f3WFBikt8+PwqoE3PJwhBFbz+IIl2C3qdwu5cJ7E2I4pO4ffDDu1ntDXSsoo9BINgMxtItJuIizBLYiSEOCmSCAlRF2JjtbXJ/P4j2w4dgv37tUSpCSst3mgx6oi2Gihy+1FQsVsMePxB7GYVi1FPEDDodOjUIHE2bcmOQpcPo07BqNexP8dFvN1EjM1EntPDgXxXaFxRaoyV3XodMTYj6XHSUiSEqDlJhISoK4oCxr/GtQQCWhHGVavg//5P60JrpLPKqqq0dchiNrIjswiryUCk2U2+y0ue04vXHyTKbiQQULCYDKiqit9RgkGnUFjiw2BQMBl15Lu8qCocKighwW7C41f5ZW8+8TYTXdMiySrysDfXhcmgJ8FuIi3aKuOJhBBVJomQEPWhpASSk7UWonvugeXLYc4cbZB1E3b0VPxCt5eWsRYOF7hJtFs4UOAmyqrHpNNhMugxGbTur3yXD53Hj82gJ6iqqEEo8vgw6MFk0LE7x0mC3URSlJmikgAb9mVg0utpHW+jyONjd66LpEgTyVFW4iKkTpEQ4vhk+vwJyPR5UWtUFV59Fe68EzweaNkS5s3TBlQ3I6XT7wvdXkp8AaxGA3odKIpCRr6bgw4PuUUluP0BXJ4A/qCK2xdAURT8/gCeQJCkSAstY63syHISCAZJjbHi9QUp9vpJizJjNumJtZlIjbbQLjGCOFn4VYhmp6rf35IInYAkQqLWbdwIl14K27dr65c9/jjcd5+2mGszl+f0cDjfxZ5cN1lFntByH3qdVreyyONHURSizXrsViM7s4qJsZlAhWKvn1axNnKKPBgMCm3iI0iLsWI16THoFCKtRmJtRmJtMsBaiOZAEqFaIomQqBNFRVr16ffegy5d4JdfwGYLd1QNgscX4LCjhMxCN3tyXeQVe1AU8AdVikp8BINgNOiwmw3szXVh1Cu4PAFOibeSXeTBF4QW0VaMBoXSUVgtY6zE2824fAGMeoXUaCsto63YZCyREE2WFFQUoiGLjIR334VRo7RZZJIEhZiNelrHR5AaZSE52sr+XCeFbh86RaHQ5SXf7cXlDWI3G4ixGnB5g0RZDaiqQrHHT7TNRK7LQ5TZiEGvw2bSU1jiY/3+AmJsRjqn2Dlc4GZvrotYm5EWsVaZhi9EMyaJkBDhoihw7bVltz33HLhc8Pe/a91mzdjRCVG+y4vb58ftCVBY4mdfrpMIkwF/IMjefDcxVgNWkzbo2h9QUQCLUUeB20ekRc8fhxzYzAZSo63syHKR5SihdaINt8/MYUcJcTYjp8TZSIqySkIkRDMjiZAQDcX27dryHIEArFihtRilpIQ7qrA7dkkPjy9AaoyVjAI38ZFmjHo9Xr+fSJuRGKsXly+orW2m0zrGnJ4AxR4/7RPtHCxwsz/PSaeUSIrcAXZn53FKfAR2k45Mh4dd2U50Oh1xEUbS4yKwS9eZEE1eoxqd+f3333PeeeeRlpaGoih8+umnx91/+fLlKIpS7pKRkVE/AQtRHR06wP/+p3WTLVkCvXtr/4sySluK+pwSS4ekCE5tF0fbJDtt4+20SbBjMeqwGnWYjXosRh16PcTZTPhVlf35LtJirLi8AfbkFNMixorVqOdQoYf3ftrHU99s5b2fdvPLnlx+O5BPpsMV7tMVQtSxRtUi5HQ66dWrF9dccw0XXXRRlY/bunVrmYFSSUlJdRGeECdv8mQYOFCbVbZpE5x5Jjz0EDzySLPvKjuW2ajHbLSSEGnllLgIikq82C0GjEaFQpePCJOeEq8Bq1FPTIRWj8huNGAx6dmV6SQt1orLF8Cg07Fufz4Hct1c0q8F7ZIisZv1RJgMOJw+cooKyCn2YjHoSI40kxRpkUHWQjQhjSoRGjt2LGPHjq32cUlJScTExNR+QELUhS5d4Kef4I474PXX4bHHtOvffNPkq1HX1NFJUYzVyM7sYgKqilGvw6BX0Ou0BV/j7EZMOj0RZj0Wk55Clw8FhWKXjwn9WxBlNXG40E2k2YDRoFBU4mdfrouWMVZaJ9jJc/vYnefCYtCRGmWRpEiIJqBGiZDP5yMjIwOXy0ViYiJxcXG1HVet6t27Nx6Ph+7duzNjxgyGHqeAncfjwePxhK47HI76CFGIsmw2+O9/4fTT4cYbYdw4SYKqKCXGRmyEmUK3l8JoH7nFXhLsZgpcPvKdHkAh3m7GrNejqH58AZUuLaIxGvTsyi7GbvkrsSmBHdnFpMdZibQa2ZPnZE+2k1ZxVlrFRXCgsIQdOU4sRj0tYqykxchAayEaoyonQkVFRbz77rvMmzePtWvX4vV6UVUVRVFo2bIlZ511FjfccAMDBgyoy3irJTU1ldmzZ9O/f388Hg+vv/46I0eO5KeffqJv374VHjNr1ixmzpxZz5EKUYkrroDhw7Uq1KX27YO0NDA0qgbdemU26kkyWkmKsnJKXICiEi+OEj+ZhUZQFawmHR5/kDw9JEeaUBQoKvHh8gaIsRrx+gIUuv1EmPUY9XoKXD52ZBfTKs6KxWRgw/4Cft6dS77TR4+WUfRPj6PY66N1rI1Imzncpy+EqIYqFVT85z//yRNPPEG7du0477zzGDhwIGlpaVitVvLy8ti0aRM//PADn376KYMGDeKFF16gQ4cOdRu4ovDJJ58wfvz4ah03YsQITjnlFN55550Kb6+oRahVq1ZSUFE0DA4H9OunrVv2/vvQqlW4I2pUPD4tKcos9OD0+dl6uJg4u4mgqnIw301esYeWsdoMNY8/iKIDk05HoduPXw2SGmXlUIGb3w4WkhptYXiHRFKjLdjNekCh0O3F7QsSZTGSEmMmxir1iYQIl1otqPjzzz/z/fff061btwpvHzhwINdccw2zZ8/mzTff5IcffqjzRKimBg4cyMqVKyu93Ww2YzbLX3SigfrjD8jKgh07tFllb7+tdZuJKjl6LJHL7SPKbCTP5UEFikt8ONxeVECnQKTVSCCo4g8EMRt1WBUdjhIfBW4fHZPtdEmJRq9AYYmXAjccKijBFwjSLiECFPjjUBFGXTGp0VZSo8xEWE3hPn0hRAWqlAi9//77Vbozs9nMTTfddFIB1bWNGzeSmpoa7jCEqJnBg2H9epg4Edatg3PPhbvvhiefBJN80VaHzWqkszUap9tHfokXNQgl3iB6vQ6zQYfNqMcbCFLk1lp4AqpKodtHlNVItNWIxx/A4w8Qp5jIKHTj9AZIj4vA6Q2w8UAOLWMstE2KQEUls8iDK8dJrtNLUIX4SDNt46VOkRANQY0HGezYsYOdO3dy2mmnYbVaQ+OF6lJxcTE7duwIXd+9ezcbN24kLi6OU045henTp3Pw4EHefvttAJ5//nnatGlDt27dKCkp4fXXX2fp0qUsXry4TuMUok61awerVmnFF//9b60a9cqV2kr2rVuHO7pGJ8JqJMJqRIeCQa8jt9iLyQA6FBIjzWTpdSgo+AJBAoEgdpPW1eX1BzAbdDhK/BR5/ViNBgx6hc2HHaTFWEiMMuNw+Sku8ZPl8LJqZzYb9xfQMdHOiE4JKGqAVjE2YiKtJ4hQCFGXqp0I5ebmMnHiRJYuXYqiKGzfvp22bdty7bXXEhsby3PPPVcXcQLwyy+/cPrpp4euT5s2DYDJkyczZ84cDh8+zL59+0K3e71e7r77bg4ePIjNZqNnz5589913Ze5DiEbJbIbnn4eRI2HKFG16/X33wQcfhDuyRist1kaMxUi2y4vb68egUzAZdERbjeQ6fZiNCgXFetz+IE6PH48vgE6noNOB2aDDolfIKfaiKBBjNZHt8BBhNuD0+Fm8OZPCEj/3ntWJNokRmA16AkGVAwUl/HrQIa1EQoRRtVefnzRpEllZWbz++ut06dKFX3/9lbZt27Jo0SKmTZvGH3/8UVexhoWsPi8avD174M474bXXQIqF1gmX24fD46O4xE+u00uey6vVIFJUIswmiku8GAw6VBVKfH6iLWbcPh9Wo5F9+U62HHJwYd+WRFuNKIAvqJLhKGHV9mx2ZBbRKSWSQW3j6JBkl1YiIWpJna0+v3jxYhYtWkTLo6fzAh06dGDv3r3Vj1QIcXJat4Zjl5t56imYMAHatg1HRE2OzWoMFU5MdHrZnedEryi4fH4iLXoUjHj82hpnBp2CiopO0f53eQMMahuHXlEoLvEDUODy8uPuPGIjTEwf143ECCN6vQ4FyCzy8utBBxajjpRoC0kRZqwy0FqIOlPtRMjpdGKz2cptz8vLk9lWQjQE778PDzygDaD+3//gkkvCHVGTEh1hIl1RMemhwBUAVJKizRS6/ESY9BSVKJT4ggRVFQWF9DgrURYznkAQAF8gSL7bS3q8jV4to9EpkOfygQL5Lh/781x0TbGTEmVFryjkuX2oLh8ef4AD+W48AZUoi4EOyZHE2+V3rhAnq9qLrg4fPjw0GBm0ej7BYJCnn35axt4I0RAMGwZDh2o1hyZMgKlToaQk3FE1KbE2M+0So2ifFEFatIXkSAttEyKwmfWkRluIs5uwGA1YTDpaxNiIMOtQUFFQCQZVzAY9LWOteP0qxSV+ikr8FLn97M4upn2SHavZQEGJl1yXlxynh925Lt5du58nvtrMu6t3syPTwcG8InKL3OF+KoRo9Ko9RmjTpk2cccYZ9O3bl6VLl3L++efzxx9/kJeXx6pVq2jXrl1dxRoWMkZINEo+n7ZQ66xZ2vXevbWB1A20vldT4fEFcHq9FDr95P3VihNh0uP1Byl0+wioUNkv3HyXF70CreMjKPEFsBj1of+//P0w27OcXD+sNe2S7Bh0Orz+ADnFHgrdAQx6hbQYK6fE2aSAoxB/qer3d7UTIYDCwkJefPFFfv31V4qLi+nbty9Tp05tkvV5JBESjdqiRXDVVZCTA3Y7zJkDF18c7qiaBZdbK77oDQYw6BQOFZTg8QXQ63XoFBW3J4g3GAztX+ILYjXqMZt0BAIqer1CIKBS7A2w/M9MxvdpSXyEEVAoKgmwJcPBym3ZqKrKkHbx9GgRRUq0lQS7iQirdJkJUaeJUHMiiZBo9A4d0tYsW7ECvvwSzjkn3BE1S/tzneS5vNoVVaXY48PpCaKitRKZDQp6RQFUVFVbY1dVIbPIg8cXoFNKFPq/BjNs3F/InhwnozonhVqI9AoEVXC4fWQ4SvAEVGJsRjomRRITIYOtRfNTq7PGfvvttyo/cM+ePau8rxCiHqSlwXffwdKlcNZZR7Z7PFo9IlEvWsVHEGM1UlTiJ0gQk16HDh++YJCACnaTHm8wiD8IgaCKXqcQCKokR1sIBFR8wSC+IBSVBAgEg5zdPYUYm3Z/Bp2CPxgk0+Fh2dZsftufT9eUaE7vkkROcQlKMEC0TMkXokJVahHS6XQoisKJdlUUhUAgUGvBNQTSIiSapD17tFXtH38cJk8OdzTNUoHLQ7bDjcsTxI82eMhmMuD2BSgu8YXGCCVFmnGU+AmqWjdadpEXk0FPrM0YaiGyGPTkuXys2JZNpMXAOd1TiLYYCUgLkWjGarVFaPfu3bUWmBCiAXjhBThwAK6+GpYtg5degoiIcEfVrMTYzFiNBopKvKHp9iadDotBh0mvEETFbNSj0ymYDXpc3iBBIM6uJTGlLUQGnUJhiY88p5f0OCt902NRFMhxesg4qoWo/ylxnNYpgXynB30wQKS0EAkByBihE5IWIdEkBQJanaEZMyAYhC5dtFll3buHOzIBFLu8FHv8BP769awokFvsxRcMYjfrKSoJ4AsEARWjXkeJP0i+00eUxUB8hAkUyHNqLUTJUWbO6pqC1agVbHR6A+zNdVHiV7Fb9HRMjiQx0hLW8xWiLtT5YOnNmzezb98+vF5vme3nn39+Te6uwZJESDRpK1ZoA6kPHQKLBV58Ea65RvvmFQ3KwTwXxR4vRp2eIo8ft0+bjm/QKaFlOww6BaNOocQfJLvIi88foEfLGAx/daHlFPv47s8sVmzNpP8pcZzZLZnW8RHEWw1ESQuRaGLqbImNXbt2ceGFF/L777+XGTdUuvJ8UxsjJESTNmIEbNwIf/ubNtX+uuvAYJBxQw1QizgbxS4DLq8fm0lPdrHWPaaqKnaTAX9QxRtQtdlmBNHrFFomRuD/awA2wOItmegUeHVSPyKMBvQKuH0q23KcFO53oNNBixgr6fERUo9INBvVrix9xx130KZNG7KysrDZbPzxxx98//339O/fn+XLl9dBiEKIOpWYCF99pRVf7N8fJk4Md0SiEnabiaQYGymxNuIizESY9NhMBvQ6HQa9DkVRCQaDGHUKiZEmFLQijx5fgAMFJSTZTVzaryXBgIrL62d/QQlLt2Yz+/tdfLRuPw6nBwjicLtwuz3hPl0h6kW1u8YSEhJYunQpPXv2JDo6mrVr19KpUyeWLl3K3XffzYYNG+oq1rCQrjHRrPj9WosQaOOIPv8cxo+XrrIGqtjlxeX1E/yr7lCRx4/PH0Sn04oQeQIqwb9+xe/JdXFKnBWzQU/p2KIfd+VxuMDNuT1TSY4yc/Sr7AtAocw4E41YVb+/q90iFAgEiIyMBLSk6NChQwCkp6ezdevWGoYrhGgQDEf1lj/2GFx0EVx2GRQWhi8mUamjW4iSY2wkRpqxmbVZZ3qdDlXVWoiCwSBt4q3oFPD4/PgDQQ47StDp4PzeqdgtBpzeAMV/XbKLvWzOKOLdtfv47NeDFBaXEGlW8Pg8FLlkfTPRtFR7jFD37t359ddfadOmDYMGDeLpp5/GZDLx2muv0bZt27qIUQgRDjExWmL0wQewbh3Mnw/9+oU7KnEc0TYzFqMBp9dLwK8t0+H2BlBRiY8wkVPsxY+KQa/g9QVplxCJXtHh8QfL3E+O08fBAheX9W9JSrSZvxqYAHCUBNmamUOhO4BOgbRYK61lTJFoxKrdNbZo0SKcTicXXXQRO3bs4Nxzz2Xbtm3Ex8czf/58Ro0aVVexhoV0jYlm7aeftDFDe/eCyQTPPgu33ipdZY2Ew+Uh3+XBH1CJMOnIcQbw+v3odQoqoIDWjXYUlzeAP6ASH2HEbjaCAkFVRYdCrsvLb/sdrNiexa/7C7EbDfRtHcNpHRMZ1iGReLtUKhcNR72uNZaXl0dsbGxo5lhTIomQaPby87Up9Z9+ql2/8EL43/8gNjasYYmq8fgCOL1egn6VAJBX7CWISqzNhMPtC611Vsrh9hFpMWDU6TDotd/p/r+m5q/akceH6w/w64FCrhzUkrM6J5MabcZuNhBpMmCymKRlSDQYdZYIFRYWEggEiIuLK7M9Ly8Pg8HQ5JIFSYSEQOsXefFFuOce0Ong55+l+GIjtT+3GK8/gM2ko7AkiO+Ykid6nUIwqL3MCtp6Zzod5Bb7+GrTYU6Js3Fh7xaYDX8tDAt4fVDgkoHVomGps8HSl112GfPmzSu3/YMPPuCyyy6r7t0JIRoDRYHbboPVq+GttyQJasRaxduJjzRh1GnrlVmMBswGfegSYzVh1Ou0FiGdgl6nYNBp44iGtovn3B4p+INB3D4VpydIlsPL5sPawOrFWzIwqCopdhPBoIditwysFg1ftVuE4uLiWLVqFV26dCmz/c8//2To0KHk5ubWaoDhJi1CQhzHjz/C00/Df/8L8fHhjkbUwOF8F75AINQ9ZjZAoTsYmnZ/dItQarSF0iFFep3WWnSo0ENGoZuBbWKJsupDU/A9Psh3SiuRCJ86qyzt8Xjw+/3ltvt8PtyS/QvRfAQCWgXqbdvgl19g3jwYMiTcUYlqSo21UeTyaMnQX9lQjE1PUYk/lPT4gyodk60UOAP4AL2i4PMHcZT4MekVBraJ1VqNvKCqKg6Pj4P5HpZuzeL77dk43X46pkQytH08Z3ZNoWOK/FEpGo5qd40NHDiQ1157rdz22bNn00+m1grRfOj12pT6Dh1g/3447TR46iltEVfRqETazMRF2oiP0i7J0TYiTAZMeh0mgw6LUYfZYMCg12HUKRj0WpdZEIixGVFV8AWCeANBPIEgWUU+vvkjg3d+3EffU2L418SezLqwO5f0SSU5Qo/b7T1hTELUl2p3ja1atYrRo0czYMAAzjjjDACWLFnCzz//zOLFixk+fHidBBou0jUmxAkUFcFNN8F772nXzz4b3n5bW7pDNGpFLg/+oDaY2qDC4WI/ATWI/q91JhVFCXWF6RSFoKricPvYkeMkp6iEi/u1wnZUT5gCFHmCbM90UuJXsZn0dEy2kxwtC76K2len0+c3btzIM888w8aNG7FarfTs2ZPp06fToUOHkwq6IZJESIgqUFV44w2txlBJCaSlaeOHWrUKd2SiFmUWunF6fIA2Rshq1BaBVVVVGzOkqmQXeVFVaBVrxXBUjSKdAvsL3CzalMXybRnszffQOtbGiM6JjOmaQp90Kccgale91hFqyiQREqIaNm2CCROgUyf45BMpvNgEHcpzESSITlEwGlQKXSpqMFimRSg20ozxmEKNigLvrtnP7B92AXDvmA6M6phAvN2IVafHYFCwWqVlSNSeOhssvX79eoxGIz169ADgs88+480336Rr167MmDEDk0lmBAjRbHXvrg2c9vmOJEFFReByQXJyeGMTtSItzobD5SagqigoRFqgxKcAKjpFR3SEjmI3eANHxorpFIV9+S6Wb8vguQk9OKNzAgblSCFHtw+yckrIdjoIBFXi7WY6JNqxW41hOUfRvFR7sPSNN97Itm3bANi1axcTJ07EZrOxYMEC7rvvvloPUAjRyEREaOuUgdZlduON0Ls3LF0azqhELYqyWYmNsBEToQ2sthp02sBqvQ6z3oDRoGAyKhj/uphMCm5/gBtPa8ewdgkEAlDi06bYHyjwsHZPAW+t2cNDn27ioY838dLS7Xywbj97c53hPlXRDFQ7Edq2bRu9e/cGYMGCBYwYMYL33nuPOXPm8NFHH9V2fEKIxqywEH7/HTIyYPRoeOQRbdq9aFKSYmxEmvVYTKBTVHwBFa9PxffXxetVsRr09E2PwxsIUnLUZXeOmy9/O8SC9Yc4WFBCRrGH7/7MZuFvh1m1PYdity/cpyeauGp3jamqSvCv6bHfffcd5557LgCtWrUiJyendqMTQjRuMTHawq133AGvvw6PPgrffw9z52oDqkWTYbdZQj+7/W48vrIJb88WEeQ5gxh0CqWl6LKKStib52T9/nwAbhvVlnO7pxAfaUAPGDCgxw9IF5moO9VOhPr378/jjz/O6NGjWbFiBa+88goAu3fvJlnGAAghjmWzaZWnTz9d6yZbvlzrKnvnHRgzJtzRiTqQFGUlu9DF0VNxLAYFkxEMem3qPYAfleISP3eMas+4nsmULtdaOsza6VPJyvHI2CFRp6qdCD3//PNceeWVfPrppzz44IO0b98egA8//JAhUlVWCFGZK66A/v3h0kvh11+1Fe137ACZKdQkJUbbKHa5Cf41JNqvqvj8KsGASvCvBMmAwtndk0mNsuD3g58jY+wzizzszHazYmsmq3flEfCrdG8ZxZD2CZzRJZn0+IjwnJhocmpt+nxJSQl6vR6jsWll6jJ9XohaVlICd98NF18Mo0aFOxpRjzIL3Xj9AXQ6Qq1Fx1ZYKL26cb+DL349xNd/ZHLvmA6M7ZpMvE2HDgPaPwNWaRkSx1Fnq89XxmKx1HkS9P3333PeeeeRlpaGoih8+umnJzxm+fLl9O3bF7PZTPv27ZkzZ06dxiiEOAGLBV56qWwS9NFH8OWX4YtJ1IvkaCtmvQ7LX5doo46AX8Xv0y5Bv0rAr7I/183ePCfn9kzh94dPZ8qgliRHGjHq9fgUldxiP2v35rFoUwardmSTXVQS7lMTjVi1u8bCyel00qtXL6655houuuiiE+6/e/duxo0bx0033cTcuXNZsmQJ1113HampqYyRsQlCNAw7d8LVV0NxMdx7LzzxBDSxlmVxRFKM1mUGKnpFwWTUU1pRSPdX85AflWHt42kRbcEfBJ+qtRw5SvxkFHr57s8slm3NJKvQQ9sEO8M6xnN2t1S6t4wJ23mJxqtRJUJjx45l7NixVd5/9uzZtGnThueeew6ALl26sHLlSv71r39JIiREQ9GypTZe6D//gWeegR9+0FayT08Pd2SijthtR8aFZRcXhbrJSotRG1CItZko8ZddwLfYE2Txlkz+t3JPaNvGg4XsyXOioJAcbSEx0oIQ1dGoEqHqWrNmDaNHjy6zbcyYMdx5552VHuPxePB4PKHrDoejrsITQgCYzfDvf2uzyqZM0dYo690b5syBCy4Id3SijplNhtDCrqCNEeqUbKWoREVVIKhqCVKey8ehIg8rtmYB2lT78T3TSIzQoaAHRcGglxWjRPU16UQoIyOj3JT+5ORkHA4Hbre7wnVtZs2axcyZM+srRCFEqfHjtQTossu02kPjx8Odd8I//ylrljVhydFWch2u0HVFAbNBwWvSEqCgqqJTFHQeP25vgGuHtubco6bal3J4gxzIdlPodqBTIC3WSuv4CMzGY/cUoqxqJUKbN2/mxRdfZM2aNWRkZACQkpLC4MGDufXWW+natWudBFmfpk+fzrRp00LXHQ4HrWQFbSHqR+vWWsHFv/8dnnsODAZJgpqB+CgbrqOm2gdVFa8vgMKRFqFgQKVPq2giLXoCKpS2ISkK7MsrYeM+B99vz2LD/kIijAb6tY5heIdETuuYSLzdHLZzEw1flROhr7/+mvHjx9O3b18uuOCCUEtLZmYm3377LX379uWzzz5rUGNvUlJSyMzMLLMtMzOTqKioSlc5NpvNmM3yoREibEwmePZZOPdcGDr0yPaSEm3GmWiSbLayv5OLfW4CwUCo5Sc+UlvQu8QX5Fgb9zn4cP0Bfj1QCEAOXvbmu3B5A0RbjQxtnyAtQ6JSVU6EHnjgAe6//34effTRcrfNmDGDGTNmcO+99zaoRGjw4MF89dVXZbZ9++23DB48OEwRCSGqbOTIIz/7fNp0+z59tJYiSYiavORoK3lFLhRFqzlkVKDIE+Do9kFFgZ05brZkFLK7ggVaNx100LtVEZ1To0iLkcKdomJVToS2bdvGlVdeWentl19+OU899VStBFWZ4uJiduzYEbq+e/duNm7cSFxcHKeccgrTp0/n4MGDvP322wDcdNNNvPjii9x3331cc801LF26lA8++IAvpV6JEI3Ld9/BmjXaZfVq+OAD6NAh3FGJOhYXqXWZgYqiKJiMhtDMMtASIW8wSK7Ti9OjdZb969IeDGsfi0XRvt50hlosmCeapConQq1bt+bLL7+kU6dOFd7+5Zdfkl7H011/+eUXTj/99ND10rE8kydPZs6cORw+fJh9+/aFbm/Tpg1ffvkld911F//+979p2bIlr7/+eoNqtRJCVMHYsfD11/C3v8HGjdC3L7z2Glx+ebgjE3Xs6C6zfFdRuRYhk05HfISJ968bSPcU21G3qqH/3EHIzC5ib66LEr+K3aKnY3KkTLUXQDWW2FiwYAFXXHEFY8eOZfTo0WXGCC1ZsoRvvvmG9957j4svvrhOA65vssSGEA3IwYPammXff69dv/56beq9rFfWLGQWauOGylEVIixailSaKJX+n+eSIozNVVW/v6vcIjRhwgRatGjBf/7zH5577rlys8aWL18uY2+EEHWrRQtYsgQefRQef1xb1d7phLlzwx2ZqAfJ0Vbyi11lJhIqCjg9QZyeiv+mlyKM4kSqNX1+yJAhssK8ECK8DAYtETrtNLj5ZnjkkXBHJOpRrP3IuKFSOlRQj1RaCK1g7yhbhPFoLk+AtbvzGNI+QRKhZq5JF1QUQjRho0fDli1aYlTq88/hjDMgIiJ8cYk6d+xUe0/QDZRPhHR6rQjj/nzt9scu6MKYrolYddoK9jpkILWoxURoy5YtjBs3jl27dtXWXQohxPEdnQQtW6ZVo+7SRZtV1q1b2MIS9cvj81fYIhQMqFhNeuZM7k+vFvajjlBDDUolKmTnFLM7x0mJXyXCrKdjsp2kKBl31lzUWiLk9XrZu3dvbd2dEEJUj8EAKSmweTMMGAAvvqitXSaVqZs8RWcoO4j6ryQnPtJEy3gLBgWOLsNY+o7Idwc47PDy7eYslm/L4FC+hzbxEQzvlMDZ3VLo2Sq2vk5BhFGVE6Gjl52oSHZ29kkHI4QQNTZ8uDa1/m9/g8WL4dprYelSeOUViIwMd3SiDiVHWyl0aeuVHTtrrMhTvhJ1KZcvyOLNmfz3h92hbb8fdrArtxgFSIm2SMtQM1DlROjf//43vXv3rnQKWnFxca0FJYQQNZKUpNUbevppeOghbTbZzz9rXWW9eoU7OlGHom3lB1EDGHRqudpDAFkOP4eLPKzYWnYZJgCPL8ianbkMaZ8giVAzUOVEqH379tx1111cddVVFd6+ceNG+vXrV2uBCSFEjeh08MADMGyYVnBx2zZtNXtJhJq8YwdRA/hc7rLT7f/6X68P4PIG2JvnLneMX4WD+SWhatWiaatyItS/f3/WrVtXaSKkKApVrM0ohBB1b9gwravs7be1wouiWfL4/BW2CAUCKjaTnvQ4K9uyXPzr0h6c3iEWvcEg06mbmSpXls7IyMDj8dT5MhoNjVSWFqIJyc+HCRO0rrO+fcMdjagHlVajBoo9AYw6hWT78VMfP9pYox2Z2swym0mbWZYcLd1mDVmtV5ZOSUmplcCEECJs/v53rTL14MHaKvZTp8qssiYuOdpKkctVtlXor/9jzCeuIlTshYOFbhZt0maW7c330DrWxojOiYzpmkKfdJlZ1thJC6AQovl48knIyIBPP4XbbtNqD/3vfxATE+7IRB2KrGQgdZEPAsHKZ5UBBIKwaFMWs384UiNvS2YRW7OKAEiLsUjLUCNX7aKasbGxxMXFlbvEx8fTokULRowYwZtvvlkXsQohxMmJjYWPP9YWajUatZ/79IG1a8MdmahjNpsVm81W5mLQBTAbVCxHXawGFdtf/7s9QXZlO1m+LaPc/QVV+H5rNtsyZcZ0Y1ftROjhhx9Gp9Mxbtw4Zs6cycyZMxk3bhw6nY6pU6fSsWNHbr75Zv773//WRbxCCHFyFAVuvx1Wr4a2bWHPHm1g9eefhzsyUc/MBsNxLyajgssfYHdu+ZllAAfyXbi8MrOssat219jKlSt5/PHHuemmm8psf/XVV1m8eDEfffQRPXv25D//+Q/Xy0wNIURD1b8/rF8P112ntQgNGxbuiEQ98/j9AOXGDylonWhen4rNoKdNvJU/M13ljm8Za8Nm0tdHqKIOVXnWWCm73c7GjRtp3759me07duygd+/eFBcXs3PnTnr27InT6azVYMNBZo0J0cSpKmRmastzlF7fvh06dgxvXKLOHW9GWalAEOb+uJ/ZP+zipct7cVrbaPQGQ5nkyWqVMUINUVW/v6vdNRYXF8cXX3xRbvsXX3xBXFwcAE6nk0gpaS+EaAwU5UgSBPDGG9C1qzbF/gQDaUXjlhxtJcasO+7FrNdx8+npbH74dE7vEIderwdVRT3q4nK5yC1ysTPTwYo/M1m0KYM1O3PILfaE+xRFFVS7a+wf//gHN998M8uWLWPgwIEA/Pzzz3z11VfMnj0bgG+//ZYRI0bUbqRCCFEf1qyBQADuvx+WL4e33oLExHBHJeqIrZIZZaXsphPfR0axj8OFXpb8mcXybVnkFXlpn2RnaId4xnRLpWtadO0GLWpVtbvGAFatWsWLL77I1q1bAejUqRO33XYbQ4YMqfUAw026xoRoZlRVm1J/221QUgJpafD++3DaaeGOTIRBVbrPMhxePtt4kLfW7CuzPTnSzBWDTuGqU9OJt5vrMkxRgVovqHi0oUOHMnTo0BoHJ4QQDZaiaAOoBw2CSy+FP/+E00+HRx+F6dO1tcxEs2HW+dHptPai0oHUR9uZ6+VgYQnfb8sud6zD7WPd3jwGt42XRKgBq1EiFAgE+PTTT9myZQsA3bp14/zzz9f6ToUQoino0UNbuX7qVG29sn/8A846CwYMCHdkoh6ZDSf4mtT5cHv9HCosKXdTiT9IVpEHp9dfR9GJ2lDtRGjHjh2cc845HDx4kE6dOgEwa9YsWrVqxZdffkm7du1qPUghhAgLu10bIzRqFBw6JElQM+Tx+9FReYsQQRWryUBatKVcvSGLQUdSpJkIkyzi0JBVu4339ttvp127duzfv5/169ezfv169u3bR5s2bbj99tvrIkYhhAivyZO1brFSO3bA449rg6pFk+YJGij2Kzj9CsV+haJjLknRZlrEWDmtY/kB9VFWI/3S42iXZA9D5KKqqj1YOiIigh9//JEePXqU2f7rr78ydOhQioubVrlxGSwthCjD74eBA2HDBhg5Et57D1JTwx2VqEMuV/liikc7etbYlQNakBRh5ugeNakzFB51NljabDZTVFRUbntxcTEmUxXmGQohRGNmMMDdd8ONN2rT63v1gnff1cYPiSbpRFPsU+xGUuxG+rRo89cWlaObGI5NpAo8QbZnOinxq9hMejom22Xh1jCqdtfYueeeyw033MBPP/0UKib1448/ctNNN3H++efXRYxCCNGwXHmltjxHr16QnQ1nnw0PPqi1FokmqaJFW4++VEUA2Jrt5u3V+5n19Wbu/GAjT365hTdX72HD3vy6PQFRqWp3jRUUFDB58mS++OILjEYjAH6/n/PPP585c+YQHd20CkdJ15gQolIlJXDXXfBXMVmGDYNPPoGEhPDGJepVVWoNlXp7tbZcx7FuGtGWKUNaS8tQLaqzrrGYmBg+++wztm/fzp9//glAly5dyq09JoQQTZ7FAq+8otUZuu468HpB/mBqdmw6/wn7V9wB2JLhYvm2jApvX/FnNkPbJUgiFAY1ntPXoUMHOnToUJuxCCFE43TppdCvnzZ+qHSspM+n/f9Xy7lougwnqjUEmPXg9gfKTbEvtSffhcsrsxDDoUqJ0LRp06p8h//85z9rHIwQQjRax9ZQ+8c/tMHU8+dDenpYQhL1w1+FsWGeAFgNetrEW/kzs/wstNaxNmwmKUocDlVKhDZs2FClO1OUcqWmhBCi+cnNhddeg/x86N0b5syBCy4Id1SijriChiqNEeqQHMHIjin8mVl+jNCIzol0TJZ6Q+FQo0VXmxMZLC2EqJHdu+Gyy2DtWu36HXfA008f6ToTTcqJag2BNmtsR7abRZuyWL4tg735HlrH2hjROZGxPVLo1TK27gNtRqr6/S2J0AlIIiSEqDGvV6tIXTpkYMAArausTZvjHycapePVGjqa1BGqH1X9/q5SHaGbbrqJAwcOVOmB58+fz9y5c6sWZQ289NJLtG7dGovFwqBBg1hb+tdWBebMmYOiKGUuFoulzmITQogyTCZ47jn4/HOIjdUWcR0+HDyecEcm6sCJag2VXtJi7YzonMyY7ikM75goSVCYVWmMUGJiIt26dWPo0KGcd9559O/fn7S0NCwWC/n5+WzevJmVK1cyb9480tLSeO211+ok2Pnz5zNt2jRmz57NoEGDeP755xkzZgxbt24lKSmpwmOioqLYunVr6LqMYxJC1LvzzoONG+Hyy+GWW8BsDndEQoi/VLlrLDMzk9dff5158+axefPmMrdFRkYyevRorrvuOs4+++w6CRRg0KBBDBgwgBdffBGAYDBIq1atuO2223jggQfK7T9nzhzuvPNOCgoKavyY0jUmhKg1gQDoj5oZ9OOPEB8PUoqkWalqF9qxpEutemq9oGJycjIPPvggDz74IPn5+ezbtw+3201CQgLt2rWr85YWr9fLunXrmH7UCtA6nY7Ro0ezZs2aSo8rLi4mPT2dYDBI3759efLJJ+nWrVul+3s8HjxHNVs7HI7aOQEhhDg6CcrKgosugqIi+O9/tYHVosmryqDqimytZJD1mK4p9EmXQdYno0YFFWNjY4mNrd8nPicnh0AgQHJycpntycnJoQrXx+rUqRNvvPEGPXv2pLCwkGeffZYhQ4bwxx9/0LJlywqPmTVrFjNnzqz1+IUQooxAQGsJ+v57rcts2TJ4/nmQlcqbLG0pjmCNjl20KavM0hxbMovYkqktgJ4WY5GWoZNQ7UVXG5PBgwczadIkevfuzYgRI/j4449JTEzk1VdfrfSY6dOnU1hYGLrs37+/HiMWQjQbqamwZAk89BAoilZ36NRT4agxjaJpsen8RBrUal+2ZzqPuzTHtsziej6TpqXRJEIJCQno9XoyMzPLbM/MzCQlJaVK92E0GunTpw87duyodB+z2UxUVFSZixBC1AmDAR57DBYtgqQk+O03bamOd98Nd2SiDhgMhhpdZGmOutVoEiGTyUS/fv1YsmRJaFswGGTJkiUMHjy4SvcRCAT4/fffSU1NraswhRCi+s48U5tVNmoUOJ2wcCFIibcmx+/31+hSujRHRWRpjpNX40VXw2HatGlMnjyZ/v37M3DgQJ5//nmcTidTpkwBYNKkSbRo0YJZs2YB8Oijj3LqqafSvn17CgoKeOaZZ9i7dy/XXXddOE9DCCHKS02FxYvhhRfgmmu07jLRpFR1KY5jydIcdavaidCoUaP4+OOPiYmJKbPd4XAwfvx4li5dWluxlTNx4kSys7N5+OGHycjIoHfv3nzzzTehAdT79u1DpzvSyJWfn8/1119PRkYGsbGx9OvXj9WrV9O1a9c6i1EIIWpMr4c77zxyXVVh0iQYOVKSoyYgOdpa41ljY7prtfIqWppDBkqfnGovsaHT6cjIyChXwDArK4sWLVrg8/lqNcBwkzpCQoiw+fhjuPhi7ecrr4TZs8Euf/03dlJHqH7Ueh2h3377LfTz5s2bycg4MoI9EAjwzTff0KJFixqGK4QQopzx4+HJJ+Ef/4C5c7UlOhYsgJ49wx2ZOAk2W82SF5sN0mIlEa5tVW4R0ul0oaKJFR1itVp54YUXuOaaa2o3wjCTFiEhRNitXKkVXDx4UFue4z//geuvl64yIY6j1luEdu/ejaqqtG3blrVr15KYmBi6zWQykZSUhF4vI9eFEKLWDRumzSqbPBm++gpuvBH+/PPIqvZCiBqrciKUnp4OaFPWhRBC1LOEBPjiCy35+cc/4JJLwh2REE1CtQdLA2zfvp1ly5aRlZVVLjF6+OGHay24hkC6xoQQDU5mJhy93NCWLdC5s3SVCXGUWu8aK/Xf//6Xm2++mYSEBFJSUsostqooSpNLhIQQosE5Ogn64w8YMADGjoX//Q+OKW0ihDi+aidCjz/+OE888QT3339/XcQjhBCiOjZuBL9fm2q/fj3Mnw8DB4Y7KiEajWonQvn5+UyYMKEuYhFCCFFdV14JHTvCxImwe7c2sPqpp7TCjNJV1qTUtP7QsaQeUVnVXmtswoQJLF68uC5iEUIIURMDBmitQRdfDD4fTJum1SDKywt3ZKKWaBWpTz4J2prt5u3V+5n19Wbu/GAjT365hTdX72HD3vyTD7KRqnaLUPv27fnHP/7Bjz/+SI8ePTAajWVuv/3222stOCGEEFUUE6MVW3zlFbjrLvj8c/jvf0GGMTR6mYVuArU0Y3vRpixm/3BkzbItmUVsySwCIC3G0ixbhqo9a6xNmzaV35misGtX+UXhGjOZNSaEaHQ2bNCKLv73v2BoVGtriwoUFRXVyv2sP+hi1teb+TOz/HpnXZIj+fu4LgzvmFjBkY1Tnc0a271790kFJoQQoo716QNvvnnkuscD06drl8Sm80XXXBhqKZl1+wPsznVXeNuefBcub6BWHqexqfYYISGEEI3MAw/Av/4FvXvDDz+EOxpRTX6/v1YuVoOeNvEVd321jrVhMzXP1SGqnWaeaC2xN954o8bBCCGEqANTpsDXX8PWrTByJDz6qNY6pJO/hRsDV9BAIHjyrTUdkiMY2TGFPzPLD2EZ0TmRjsnNc0HXGk2fP5rP52PTpk0UFBQwatSoWgtMCCFELenZE375BW65Bd55Bx56CFas0H4+ujijaJCSo61/zRo7eWO6JwGwfFsGe/M9tI61MaJzImN7pDTLgdJQwyU2jhUMBrn55ptp164d9913X23E1WDIYGkhRJMyZ46WELndkJICn3wCp54a7qhEFUgdoeqp6vd3rSRCAFu3bmXkyJEcPny4Nu6uwZBESAjR5GzeDJdeCgcOaDPMjjMbWIjGqs5mjVVm586d+P3+2ro7IYQQdaVrV1i7Vlun7OgkyOmEiIjwxSVEGFQ7EZo2bVqZ66qqcvjwYb788ksmT55ca4EJIYSoQzabVpG61DffwNVXa+OGzjwzbGEJUd+qnQht2LChzHWdTkdiYiLPPffcCWeUCSGEaKCeew4yM2HMGPj732HGDCnGKJqFWhsj1FTJGCEhRLPgdmtLc7z6qnZ9+HB47z1o2TK8cQlRQ1X9/q5xEYns7GxWrlzJypUryc7OrundCCGEaAisVpg9G+bNg8hIrfBi797w1VfhjkyIOlXtRMjpdHLNNdeQmprKaaedxmmnnUZaWhrXXnttrdU5EEIIESYTJ2or2fftC7m5MG6cNrNMiCaq2onQtGnTWLFiBV988QUFBQUUFBTw2WefsWLFCu6+++66iFEIIUR9at8eVq+GW2/VqlL36RPuiISoM9UeI5SQkMCHH37IyJEjy2xftmwZl156aZPrJpMxQkKIZi0YPLIUR3Y2/PQTnHtueGMSogrqbIyQy+UiuYKS7ElJSdI1JoQQTU1pEhQMwqRJcN552qBqrze8cQlRS6qdCA0ePJhHHnmEkpKS0Da3283MmTMZPHhwrQYnhBCigQgEoEsX7efnn4dhw2D37rCGJERtqHbX2KZNmxgzZgwej4devXoB8Ouvv2KxWFi0aBHdunWrk0DDRbrGhBDiKJ9/rhVezM+H6Gh44w246KJwRyVEOXW61pjL5WLu3Ln8+eefAHTp0oUrr7wSq7X5LtomhBDNxt69cPnlsGaNdv3WW+GZZ8BiCW9cQhyl3hddbaokERJCiAr4fPDQQ/D009CuHaxbp7UQCdFA1Nmiq7NmzSI5ObncchpvvPEG2dnZ3H///dWPVgghRONiNMJTT8HIkZCUJEmQaLSqPVj61VdfpXPnzuW2d+vWjdmzZ9dKUEIIIRqJsWOhX78j1196CW68UVuyQ4hGoNotQhkZGaSmppbbnpiYyOHDh2slKCGEEI1QZibccw+UlGjjhz74ACr4w1mIhqTaLUKtWrVi1apV5bavWrWKtLS0WgnqeF566SVat26NxWJh0KBBrF279rj7L1iwgM6dO2OxWOjRowdfybo5QghRN5KTtVllSUnw++/Qvz+88064oxLiuKrdInT99ddz55134vP5GDVqFABLlizhvvvuq/MlNubPn8+0adOYPXs2gwYN4vnnn2fMmDFs3bqVpKSkcvuvXr2ayy+/nFmzZnHuuefy3nvvMX78eNavX0/37t3rNFYhhGiWzjwTNm6EK6+EZcu0IozLlsELL0BERLijE4DL5QbCP0+qwBNke6aTEr+KzaSnY7Kd5Oj6n31e7VljqqrywAMP8J///AfvX5VFLRYL999/Pw8//HCdBFlq0KBBDBgwgBdffBGAYDBIq1atuO2223jggQfK7T9x4kScTicLFy4MbTv11FPp3bt3lcczyawxIYSogUAAHn8cZs4EVYWePeGXX7RB1iJsGsoKEFuz3SzalMXybRnszffQOtbGiM6JjOmaQp/02Fp5jDqbNaYoCk899RT/+Mc/2LJlC1arlQ4dOmA2m08q4BPxer2sW7eO6dOnh7bpdDpGjx7NmtJaFsdYs2YN06ZNK7NtzJgxfPrpp3UZqhBCCL0eHnkETjsNrrhCqzskSVBYZRa6CQSD4Q4DgEWbspj9w67Q9S2ZRWzJLAIgLcZSry1D1U6EStntdgYMGFCbsRxXTk4OgUCg3DpnycnJocKOx8rIyKhw/4yMjEofx+Px4PF4QtcdDsdJRC2EEM3c6afDpk0Qe9Rf+Xv2QEIC2O1hC6s5sun8NRgZXPvWH3SxfFvF38Mr/sxmaLuExpEINVWzZs1i5syZ4Q5DCCGajvj4Iz+7XNrq9X6/NqusZ8/wxdXMGAwN4yvf7Q+wO7fi8gp78l24vIF6jacB5IZVk5CQgF6vJzMzs8z2zMxMUlJSKjwmJSWlWvsDTJ8+ncLCwtBl//79Jx+8EEIIzZ49UFAAW7fCwIHw2mvaGCJR5/x+f4O4WA162sRX3OLTOtaGzaSv1+el0SRCJpOJfv36sWTJktC2YDDIkiVLKl31fvDgwWX2B/j2228r3R/AbDYTFRVV5iKEEKKWdO2qzSobOxY8Hq344hVXgAxDqHOuoIEivxL2S4fkCEZ2rLhBYkTnRDom12+XacNoJ6uiadOmMXnyZPr378/AgQN5/vnncTqdTJkyBYBJkybRokULZs2aBcAdd9zBiBEjeO655xg3bhzz5s3jl19+4bXXXgvnaQghRPOWkAALF8Jzz8H06TBvnjajbP586Ns33NE1WcnR1gYza2xMd63kzbGzxsb2SKn3KfSNKhGaOHEi2dnZPPzww2RkZNC7d2+++eab0IDoffv2odMdaeQaMmQI7733Hg899BB///vf6dChA59++qnUEBJCiHDT6eDee2HYMLjsMtixAx54ABYvDndkTZrNZmsQdYQ6JVpJHtKKwe3iGl8doeZG6ggJIUQdy8uDadPgiSegRYtwRyOaiKp+fzeaMUJCCCGaqLg4mDOnbBI0axb8/HPYQhLNhyRCQgghGpaFC+Hvf4ehQ+H552VWmahTkggJIYRoWIYNg4suAp8P7roLxo/Xus+EqAOSCAkhhGhYYmLgww/hxRfBZNJWtO/TBypZTkmIkyGJkBBCiIZHUWDqVC35adcO9u3T1i37a9FtIWqLJEJCCCEarr59Yf16mDhRW5YjNTXcEYkmRhIhIYQQDVtUFLz/PvzwA1x88ZHtxcXhi0k0GZIICSGEaPgURRtEXerQIejYEZ58EoLB8MUlGj1JhIQQQjQ+c+fC4cPw4IPaumVZWeGOSDRSkggJIYRofO65B954A6xWbVmOXr1g2bJwRyUaIUmEhBBCND6KAlOmaNWnu3aFjAwYPRpmzoRAINzRiUZEEiEhhBCNV7dusHatlhQFgzBjhraqvRBVJImQEEKIxi0iQusme/ttGDBAqz8kRBVJIiSEEKJp+Nvf4McftcQItBait97S6g8JUQlJhIQQQjQduqO+1p59Fq6+GkaNgoMHwxaSaNgkERJCCNE0padDZKRWiLF3b/j663BHJBogSYSEEEI0TRMnwrp12oKtOTlwzjlw//3aqvZC/EUSISGEEE1Xhw6wevWRAdRPPw0jR2qLuAqBJEJCCCGaOotFW7X+ww8hOlqrPSSVqMVfDOEOQAghhKgXF1+sdZP9/DP0739ku6pqBRpFsyQtQkIIIZqPtm21sUOlNmyAESNg9+7wxSTCShIhIYQQzZOqws03a7PK+vSBjz8Od0QiDCQREkII0TwpCsyfD6eeCoWFWtfZbbeBxxPuyEQ9kkRICCFE85WeDt9/D/feq11/8UUYMgR27AhvXKLeSCIkhBCieTMatWn1CxdCfDysXw99+8Iff4Q7MlEPZNaYEEIIATBuHGzcCJdfDjYbdOkS7ohEPZBESAghhCjVsiUsWwbFxUfWLXO5tLXKOnQIb2yiTkjXmBBCCHE0gwFiYo5cv+MObVbZ3LlhC0nUHUmEhBBCiMqUlGgDp51OuOoquO46rYVINBmSCAkhhBCVsVjgu+/gkUe06fb/+x8MHAibN4c7MlFLJBESQgghjkevhxkztIQoJUWbTTZgAMyZE+7IRC2QREgIIYSoilGjtFllo0dr3WP33AN5eeGOSpwkmTUmhBBCVFVyMixaBLNmQe/eEBcX7ojESZJESAghhKgOnQ4efLDsts8/h4wMuP56Wcm+kWk0XWN5eXlceeWVREVFERMTw7XXXktxcfFxjxk5ciSKopS53HTTTfUUsRBCiGbh8GGYPBluvBGuuAIcjnBHJKqh0SRCV155JX/88QfffvstCxcu5Pvvv+eGG2444XHXX389hw8fDl2efvrpeohWCCFEs5GcDNOna4Oq582Dfv1gw4ZwRyWqqFEkQlu2bOGbb77h9ddfZ9CgQQwbNowXXniBefPmcejQoeMea7PZSElJCV2ioqLqKWohhBDNgk4H992nLd7aqpVWd+jUU+Hll0FVwx2dOIFGkQitWbOGmJgY+vfvH9o2evRodDodP/3003GPnTt3LgkJCXTv3p3p06fjOkEhLI/Hg8PhKHMRQgghTmjIEK0l6LzzwOuFqVPh0kvB7w93ZOI4GkUilJGRQVJSUpltBoOBuLg4MjIyKj3uiiuu4N1332XZsmVMnz6dd955h6uuuuq4jzVr1iyio6NDl1atWtXKOQghhGgG4uPhs8/gn//UlupIStL+Fw1WWF+dBx54gKeeeuq4+2zZsqXG93/0GKIePXqQmprKGWecwc6dO2nXrl2Fx0yfPp1p06aFrjscDkmGhBBCVJ2iwF13wYgR0LXrke1FRWC3y6yyBiasidDdd9/N1Vdffdx92rZtS0pKCllZWWW2+/1+8vLySElJqfLjDRo0CIAdO3ZUmgiZzWbMZnOV71MIIYSoUN++R34OBOCCCyA6Gt54A2JjwxeXKCOsiVBiYiKJiYkn3G/w4MEUFBSwbt06+vXrB8DSpUsJBoOh5KYqNm7cCEBqamqN4hVCCCFqZN06WLVKGzu0YYM2u+zUU8MdlaCRjBHq0qULZ599Ntdffz1r165l1apV3HrrrVx22WWkpaUBcPDgQTp37szatWsB2LlzJ4899hjr1q1jz549fP7550yaNInTTjuNnj17hvN0hBBCNDcDB8Lq1dCuHezdC8OHw7PPQjAY7siavUaRCIE2+6tz586cccYZnHPOOQwbNozXXnstdLvP52Pr1q2hWWEmk4nvvvuOs846i86dO3P33Xdz8cUX88UXX4TrFIQQQjRn/frB+vVHZpLdey+cfz7k5IQ7smZNUVUpcnA8DoeD6OhoCgsLpQaREEKIk6eq8NprcMcd4PFoi7h++224o2pyqvr93WhahIQQQogmQVG05Th++gl69dKm2ouwkURICCGECIdevbSB0z16HNn28cdwzCxpUbckERJCCCHC5eiaQj/+CBMnQu/esHx5uCJqdiQREkIIIRqCyEjo0EFbzf6MM+DRR7X6Q6JOSSIkhBBCNATdusHPP8PVV2vT6h95BM46C46zlJQ4eZIICSGEEA1FRAS8+Sa89RbYbLB0qTaW6Lvvwh1ZkyWJkBBCCNHQTJqkVaPu0UMbPL1pU7gjarJkSVwhhBCiIercWZtiP2cO3HTTke2qKgu31iJpERJCCCEaKqsVbr75SOJTVKQtz/HNN+GNqwmRREgIIYRoLJ56Slu8dexYmD4dfL5wR9ToSSIkhBBCNBYPPQS33KL9/H//ByNHwv79YQ2psZNESAghhGgsLBZ46SVYsACiorQV7Xv3BllQvMYkERJCCCEam0su0Zbn6N8f8vK0Vexffz3cUTVKkggJIYQQjVHbtrBypbaKfUoKnHdeuCNqlCQREkIIIRorsxmefx7++AOSk49s37AhbCE1NpIICSGEEI1dXNyRn99/H/r2hdtvB48nfDE1EpIICSGEEE3J1q3a/y+8AEOHws6d4Y2ngZNESAghhGhKZsyAhQu1VqJ167TWoQULwh1VgyWJkBBCCNHUjBsHGzdqLUIOB1x6qVZ/qKQk3JE1OJIICSGEEE1Rq1awfLlWgRrglVe0qtSiDFl0VQghhGiqDAZ48kkYMQJ++QXOOCPcETU40iIkhBBCNHVjxsCDDx65vm+fVn/I5QpfTA2EJEJCCCFEc6KqcOWV8J//wKBBsGVLuCMKK0mEhBBCiOZEUbSZZcnJsGmTtkzHW2+FO6qwkURICCGEaG7OOEObVXbGGVr32NVXw+TJ4HSGO7J6J4mQEEII0RylpMCiRfDYY6DTwdtva61De/aEO7J6JYmQEEII0Vzp9fDQQ7B0KaSlaWuXpaSEO6p6JdPnhRBCiOZuxAitq6yoCCwWbVsgAG432O1hDa2uSYuQEEIIISAxEdq2PXL9iSe05Tk2bgxbSPVBEiEhhBBClOVywZtvwvbtcOqpWlVqVQ13VHVCEiEhhBBClGWzaZWozz0XPB5tnbKJE6GwMNyR1TpJhIQQQghRXnw8fP45PPectlTHggVaV9kvv4Q7sloliZAQQgghKqYoMG0arFwJ6emwa5dWe6igINyR1ZpGkwg98cQTDBkyBJvNRkxMTJWOUVWVhx9+mNTUVKxWK6NHj2b79u11G6gQQgjR1AwaBBs2wPjx8MwzUMXv4cag0SRCXq+XCRMmcPPNN1f5mKeffpr//Oc/zJ49m59++omIiAjGjBlDSUlJHUYqhBBCNEGxsfDxx3D99Ue2/fwz/PRT+GKqBY0mEZo5cyZ33XUXPXr0qNL+qqry/PPP89BDD3HBBRfQs2dP3n77bQ4dOsSnn35at8EKIYQQTZGiaBeA/HyYMAGGDdPGETXSWWWNJhGqrt27d5ORkcHo0aND26Kjoxk0aBBr1qyp9DiPx4PD4ShzEUIIIcQx9HoYOBD8frjnHjj/fMjNDXdU1dZkE6GMjAwAkpOTy2xPTk4O3VaRWbNmER0dHbq0atWqTuMUQgghGqWoKJg/X6sxZDbDwoXQuzesWhXuyKolrInQAw88gKIox738+eef9RrT9OnTKSwsDF32799fr48vhBBCNBqKAjfdBD/+CB06wIED2nId//d/EAyGO7oqCetaY3fffTdXX331cfdpe3S572pI+WvRuMzMTFJTU0PbMzMz6d27d6XHmc1mzGZzjR5TCCGEaJZ694Z167Sk6L33YM2aI2OJGriwJkKJiYkkJibWyX23adOGlJQUlixZEkp8HA4HP/30U7VmngkhhBCiCiIj4d134eyzYdy4I4mQqjbopKjRjBHat28fGzduZN++fQQCATZu3MjGjRspLi4O7dO5c2c++eQTABRF4c477+Txxx/n888/5/fff2fSpEmkpaUxfvz4MJ2FEEII0YQpCvztbxAXp11XVZgyBR57TFvNvgEKa4tQdTz88MO89dZboet9+vQBYNmyZYwcORKArVu3UnjUOij33XcfTqeTG264gYKCAoYNG8Y333yDxWKp19iFEEKIZumHH6D0u3vFCq3F6K+hKw2FoqqNdOJ/PXE4HERHR1NYWEhUVFS4wxFCCCEal7fe0hZtdbkgORnmztWW6ahjVf3+bjRdY0IIIYRohCZP1hZq7d4dMjPhzDPh4YcbTFeZJEJCCCGEqFtdumhLcVx3nTZu6LHH4Iorwh0VIImQEEIIIeqDzQb//a/WNRYVBTfcEO6IgEY0WFoIIYQQTcAVV8A55zSYFeylRUgIIYQQ9auBJEEgiZAQQgghmjFJhIQQQgjRbEkiJIQQQohmSxIhIYQQQjRbkggJIYQQotmSREgIIYQQzZYkQkIIIYRotiQREkIIIUSzJYmQEEIIIZotSYSEEEII0WxJIiSEEEKIZksSISGEEEI0W5IICSGEEKLZkkRICCGEEM2WJEJCCCGEaLYM4Q6goVNVFQCHwxHmSIQQQghRVaXf26Xf45WRROgEioqKAGjVqlWYIxFCCCFEdRUVFREdHV3p7Yp6olSpmQsGgxw6dIjIyEgURam1+3U4HLRq1Yr9+/cTFRVVa/fbUDWn821O5wrN63zlXJuu5nS+zeVcVVWlqKiItLQ0dLrKRwJJi9AJ6HQ6WrZsWWf3HxUV1aTfiMdqTufbnM4Vmtf5yrk2Xc3pfJvDuR6vJaiUDJYWQgghRLMliZAQQgghmi1JhMLEbDbzyCOPYDabwx1KvWhO59uczhWa1/nKuTZdzel8m9O5VoUMlhZCCCFEsyUtQkIIIYRotiQREkIIIUSzJYmQEEIIIZotSYSEEEII0WxJIlSHXnrpJVq3bo3FYmHQoEGsXbv2uPsvWLCAzp07Y7FY6NGjB1999VU9RVo7qnO+c+bMQVGUMheLxVKP0dbc999/z3nnnUdaWhqKovDpp5+e8Jjly5fTt29fzGYz7du3Z86cOXUeZ22o7rkuX7683OuqKAoZGRn1E/BJmDVrFgMGDCAyMpKkpCTGjx/P1q1bT3hcY/zc1uRcG/Nn9pVXXqFnz56hAoKDBw/m66+/Pu4xjfF1heqfa2N+XWuLJEJ1ZP78+UybNo1HHnmE9evX06tXL8aMGUNWVlaF+69evZrLL7+ca6+9lg0bNjB+/HjGjx/Ppk2b6jnymqnu+YJW1fTw4cOhy969e+sx4ppzOp306tWLl156qUr77969m3HjxnH66aezceNG7rzzTq677joWLVpUx5GevOqea6mtW7eWeW2TkpLqKMLas2LFCqZOncqPP/7It99+i8/n46yzzsLpdFZ6TGP93NbkXKHxfmZbtmzJ//3f/7Fu3Tp++eUXRo0axQUXXMAff/xR4f6N9XWF6p8rNN7Xtdaook4MHDhQnTp1auh6IBBQ09LS1FmzZlW4/6WXXqqOGzeuzLZBgwapN954Y53GWVuqe75vvvmmGh0dXU/R1R1A/eSTT467z3333ad269atzLaJEyeqY8aMqcPIal9VznXZsmUqoObn59dLTHUpKytLBdQVK1ZUuk9j/9yWqsq5NpXPbKnY2Fj19ddfr/C2pvK6ljreuTa117UmpEWoDni9XtatW8fo0aND23Q6HaNHj2bNmjUVHrNmzZoy+wOMGTOm0v0bkpqcL0BxcTHp6em0atXqhH+xNGaN+bWtqd69e5OamsqZZ57JqlWrwh1OjRQWFgIQFxdX6T5N5bWtyrlC0/jMBgIB5s2bh9PpZPDgwRXu01Re16qcKzSN1/VkSCJUB3JycggEAiQnJ5fZnpycXOlYiYyMjGrt35DU5Hw7derEG2+8wWeffca7775LMBhkyJAhHDhwoD5CrleVvbYOhwO32x2mqOpGamoqs2fP5qOPPuKjjz6iVatWjBw5kvXr14c7tGoJBoPceeedDB06lO7du1e6X2P+3Jaq6rk29s/s77//jt1ux2w2c9NNN/HJJ5/QtWvXCvdt7K9rdc61sb+utUFWnxdhMXjw4DJ/oQwZMoQuXbrw6quv8thjj4UxMnEyOnXqRKdOnULXhwwZws6dO/nXv/7FO++8E8bIqmfq1Kls2rSJlStXhjuUOlfVc23sn9lOnTqxceNGCgsL+fDDD5k8eTIrVqyoNEFozKpzro39da0NkgjVgYSEBPR6PZmZmWW2Z2ZmkpKSUuExKSkp1dq/IanJ+R7LaDTSp08fduzYURchhlVlr21UVBRWqzVMUdWfgf/f3r3HVFmHcQD/HoNzDneSi9JSBhwgJJgBsZgRuNTTSIZbxixHx5mk1BFxKpdZs/6wRirMGEJrBcRahBestETlSFxSQ+AgcVFu3pJwDkTocOfpj8Y7D3IICILDeT7b+8f5vb/b8/72wrP3/R3w99erhEKpVOLUqVMoKirC008/PW5dfb5vgcnFOpq+3bNisRgymQwA4Ovri7KyMhw+fBiff/75Y3X1fV0nE+to+rau04Ffjc0AsVgMX19fFBQUCGXDw8MoKCjQ+Z42ICBAqz4AnDt3btz3unPFVOIdbWhoCNXV1XBwcJipac4afV7b6aBWq/ViXYkISqUSeXl5UKlUcHJy+tc2+rq2U4l1NH2/Z4eHh9HX1zfmOX1dV13Gi3U0fV/XKZnt3drzVU5ODkkkEsrMzKTa2lp65513yNramv78808iIoqIiKD4+HihfmlpKRkZGdHBgweprq6O9u3bR8bGxlRdXT1bIUzKZOP96KOPKD8/n5qamqi8vJw2bNhAUqmUampqZiuECevq6qLKykqqrKwkAJSUlESVlZV08+ZNIiKKj4+niIgIoX5zczOZmprSnj17qK6ujlJTU+mJJ56gM2fOzFYIEzbZWJOTk+nkyZPU0NBA1dXVtGPHDlqwYAGdP39+tkKYsKioKLKysqLCwkJqbW0VDo1GI9SZL/ftVGLV53s2Pj6efvnlF2ppaaGrV69SfHw8iUQiOnv2LBHNn3Ulmnys+ryu04UToRmUkpJCS5cuJbFYTP7+/nTp0iXhXFBQECkUCq36ubm55ObmRmKxmDw9Pen06dP/84z/m8nEGxMTI9RdtGgRhYSEUEVFxSzMevJGviI++hiJT6FQUFBQ0GNtli9fTmKxmJydnSkjI+N/n/dUTDbWxMREcnFxIalUSgsXLqTg4GBSqVSzM/lJGitOAFprNV/u26nEqs/37ObNm8nR0ZHEYjHZ2dnRyy+/LCQGRPNnXYkmH6s+r+t0ERER/X/PnxhjjDHG5g7eI8QYY4wxg8WJEGOMMcYMFidCjDHGGDNYnAgxxhhjzGBxIsQYY4wxg8WJEGOMMcYMFidCjDHGGDNYnAgxxmbcpk2bsG7dukm3+/LLL7FmzZr/3M98JBKJcPLkyQnXj4+Px/bt22duQozpKU6EGGNzUm9vLz744APs27dvxsfZtGkTvLy8YGRkpDPRKiwshI+PDyQSCWQyGTIzM2d0XtNt9+7dyMrKQnNz82xPhbE5hRMhxtiY+vv7Z3X8Y8eOwdLSEitWrJjRcYaGhmBiYoLo6GisWrVqzDotLS149dVXsXLlSqjVasTExGDLli3Iz8+f0blNJ1tbW8jlcqSlpc32VBibUzgRYowBAIKDg6FUKhETEyP80gSApKQkeHl5wczMDEuWLMG7776L7u5uoV1mZiasra2Rn58PDw8PmJub45VXXkFra6vOscrKymBnZ4fExESddXJychAaGjrunPv6+hAdHQ17e3tIpVK8+OKLKCsr06rzww8/wNXVFVKpFCtXrkRWVhZEIhEePHgAADAzM0NaWhoiIyOxePHiMcdJT0+Hk5MTDh06BA8PDyiVSqxfvx7Jyck653bz5k2EhobiySefhJmZGTw9PfHTTz8J52tqarB27VpYWlrCwsICgYGBaGpqEq7P6tWrYWtrCysrKwQFBaGiomLca3H79m2Eh4fD2toaCxcuRFhYGG7cuKFVJzQ0FDk5OeP2w5ih4USIMSbIysqCWCxGaWkp0tPTAQALFizAZ599hpqaGmRlZUGlUiE2NlarnUajwcGDB5GdnY2ioiLcunULu3fvHnMMlUqF1atXY//+/YiLi9M5l5KSEvj5+Y0739jYWBw/fhxZWVmoqKiATCaDXC5He3s7gH+e5Kxfvx7r1q1DVVUVtm7dir17907mkgAALl68+NjTIrlcjosXL+ps895776Gvrw9FRUWorq5GYmIizM3NAQB//PEHXnrpJUgkEqhUKpSXl2Pz5s0YHBwEAHR1dUGhUKCkpASXLl2Cq6srQkJC0NXVNeZYAwMDkMvlsLCwQHFxMUpLS4WE9NEne/7+/rhz585jCRJjBm22/+srY2xuCAoKoueee+5f6x09epRsbGyEzxkZGQSAGhsbhbLU1FRatGiR8FmhUFBYWBidOHGCzM3NKScnZ9wxOjo6CAAVFRVplY/0Q0TU3d1NxsbG9M033wjn+/v76amnnqJPP/2UiIji4uLo2Wef1epj7969BIA6OjoeG/fR/h/l6upKH3/8sVbZ6dOnCQBpNJoxY/Dy8qIPP/xwzHMJCQnk5ORE/f39Y54fbWhoiCwsLOjHH38UygBQXl4eERFlZ2eTu7s7DQ8PC+f7+vrIxMSE8vPzhbLOzk4CQIWFhRMalzFDYDSbSRhjbG7x9fV9rOz8+fP45JNPUF9fj4cPH2JwcBC9vb3QaDQwNTUFAJiamsLFxUVo4+DggHv37mn1c/nyZZw6dQrHjh37129+9fT0AACkUqnOOk1NTRgYGNDaQ2RsbAx/f3/U1dUBAK5du4bnn39eq52/v/+4Y0+X6OhoREVF4ezZs1i1ahVee+01eHt7AwDUajUCAwNhbGw8Ztu2tja8//77KCwsxL179zA0NASNRoNbt26NWb+qqgqNjY2wsLDQKu/t7RVetwGAiYkJgH+e4DHG/sGvxhhjAjMzM63PN27cwNq1a+Ht7Y3jx4+jvLwcqampALQ3U4/+hS4SiUBEWmUuLi545pln8NVXX2FgYGDcedjY2EAkEqGjo+O/hDNtFi9ejLa2Nq2ytrY2WFpaCsnFaFu2bEFzczMiIiJQXV0NPz8/pKSkAIDONiMUCgXUajUOHz6MX3/9FWq1GjY2Njo3sHd3d8PX1xdqtVrruH79Ot58802h3sgrQzs7uwnHzth8x4kQY0yn8vJyDA8P49ChQ3jhhRfg5uaGu3fvTqkvW1tbqFQqNDY2Ijw8fNxkSCwWY9myZaitrdVZx8XFRdjPNGJgYABlZWVYtmwZAMDd3R1XrlzRajd6M/VEBAQEoKCgQKvs3LlzCAgIGLfdkiVLsG3bNpw4cQK7du3CF198AQDw9vZGcXGxzmtQWlqK6OhohISEwNPTExKJBPfv39c5jo+PDxoaGmBvbw+ZTKZ1WFlZCfV+//13GBsbw9PTc6KhMzbvcSLEGNNJJpNhYGAAKSkpaG5uRnZ2trCJeirs7e2hUqlQX1+PN954Q9gcPBa5XI6SkhKd583MzBAVFYU9e/bgzJkzqK2tRWRkJDQaDd5++20AwNatW1FfX4+4uDhcv34dubm5wt//EYlEQl+1tbVQq9Vob29HZ2en8ERlxLZt29Dc3IzY2FjU19fjyJEjyM3Nxc6dO3XOLyYmBvn5+WhpaUFFRQUuXLgADw8PAIBSqcTDhw+xYcMGXLlyBQ0NDcjOzsa1a9cAAK6ursjOzkZdXR0uX76MjRs3jvsUaePGjbC1tUVYWBiKi4vR0tKCwsJCREdH486dO0K94uJiBAYG/usTKcYMymxvUmKMzQ1BQUG0Y8eOx8qTkpLIwcGBTExMSC6X09dff6212TgjI4OsrKy02uTl5dGjP15Gb0K+e/cuubm5UXh4OA0ODo45n5qaGjIxMaEHDx7o7Kenp4e2b99Otra2JJFIaMWKFfTbb79p9fP999+TTCYjiURCwcHBlJaWRgCop6dHqOPo6EgAHjsedeHCBVq+fDmJxWJydnamjIyMMec9QqlUkouLC0kkErKzs6OIiAi6f/++cL6qqorWrFlDpqamZGFhQYGBgdTU1ERERBUVFeTn50dSqZRcXV3p6NGj5OjoSMnJyUJ7PLJZmoiotbWV3nrrLeFaODs7U2RkJHV2dgp13N3d6dtvvx133owZGhHRqBf5jDE2R7z++uvw8fFBQkLCtPW5f/9+pKen4/bt29PWpz74+eefsWvXLly9ehVGRvw9GcZG8KsxxticdeDAAeFv70zVkSNHUFZWJrzaO3DgABQKxTTNUH/89ddfyMjI4CSIsVH4iRBjbF7buXMnvvvuO7S3t2Pp0qWIiIhAQkICJwSMMQCcCDHGGGPMgPGrMcYYY4wZLE6EGGOMMWawOBFijDHGmMHiRIgxxhhjBosTIcYYY4wZLE6EGGOMMWawOBFijDHGmMHiRIgxxhhjBosTIcYYY4wZrL8B7p8TmgzlBmYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "plot_ranked_vocab_counts(fig, ax, corpus_norm, zipf=True)\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Accessing the corpus documents as SpaCy documents\n",
    "\n",
    "Sometimes it may be necessary to convert the documents of a corpus to SpaCy documents, i.e. [SpaCy Doc objects](https://spacy.io/api/doc). This may be the case when you need to use further functionality that SpaCy provides or if you interface with other code that requires SpaCy documents as input. For this purpose, you can use the [spacydocs](api.rst#tmtoolkit.corpus.spacydocs) function. The result will be a dictionary that maps document labels to SpaCy `Doc` objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:50.709079Z",
     "iopub.status.busy": "2023-05-03T13:22:50.708951Z",
     "iopub.status.idle": "2023-05-03T13:22:52.066511Z",
     "shell.execute_reply": "2023-05-03T13:22:52.066036Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spacy.tokens.doc.Doc"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import spacydocs\n",
    "\n",
    "spacy_norm = spacydocs(corpus_norm, collapse=' ')\n",
    "# check the type of a sample document\n",
    "type(spacy_norm['NewsArticles-2433'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.068382Z",
     "iopub.status.busy": "2023-05-03T13:22:52.068259Z",
     "iopub.status.idle": "2023-05-03T13:22:52.071506Z",
     "shell.execute_reply": "2023-05-03T13:22:52.071043Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "doj 1 russian spy indict yahoo hack washington cnnthe department"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# show the first 10 tokens of a sample document\n",
    "spacy_norm['NewsArticles-2433'][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the `spacydocs` functions will first generate the document texts from the current tokens via [doc_texts](api.rst#tmtoolkit.corpus.doc_texts). In case of `corpus_norm`, these are transformed (lower case) and filtered tokens. Because of that, we set `collapse=' '` for forcing a space between each token when recombining the document texts. Otherwise filtered tokens may be joint because of missing whitespace between them. After the document texts were generated, the corpus' NLP pipeline is applied to generate the SpaCy documents."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Keywords-in-context (KWIC) and general filtering methods\n",
    "\n",
    "*Keywords-in-context (KWIC)* allow you to quickly investigate certain keywords and their neighborhood of tokens, i.e. the tokens that appear right before and after this keyword.\n",
    "\n",
    "There are three corpus functions for this purpose:\n",
    "\n",
    "- [kwic](api.rst#tmtoolkit.corpus.kwic) is the base function accepting a search pattern and several options that control how the search pattern is matched (more on that below); use this function when you want to further process the output of a KWIC search;\n",
    "- [kwic_table](api.rst#tmtoolkit.corpus.kwic_table) is the more \"user friendly\" version of the above function; it produces a dataframe with the highlighted keyword by default;\n",
    "- [filter_tokens_with_kwic](api.rst#tmtoolkit.corpus.filter_tokens_with_kwic) works similar to the above functions but applies the result by filtering the documents; it is explained in the [section on filtering](#Filtering-tokens-and-documents);\n",
    "\n",
    "Let's see the KWIC functions in action. We will start with `kwic` and use the original, unprocessed data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.073231Z",
     "iopub.status.busy": "2023-05-03T13:22:52.073081Z",
     "iopub.status.idle": "2023-05-03T13:22:52.104309Z",
     "shell.execute_reply": "2023-05-03T13:22:52.103847Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'NewsArticles-2433': [],\n",
       " 'NewsArticles-2225': [],\n",
       " 'NewsArticles-2487': [['Dutch', 'lower', 'house', 'of', 'parliament']],\n",
       " 'NewsArticles-49': [['by', 'White', 'House', 'bullying', '.']],\n",
       " 'NewsArticles-469': [],\n",
       " 'NewsArticles-2766': [],\n",
       " 'NewsArticles-2712': [],\n",
       " 'NewsArticles-2301': [['on', 'the', 'House', 'of', 'Representatives'],\n",
       "  ['to', 'the', 'House', 'Intelligence', 'Committee'],\n",
       "  ['The', 'White', 'House', 'on', 'Monday']],\n",
       " 'NewsArticles-1377': [],\n",
       " 'NewsArticles-3428': [],\n",
       " 'NewsArticles-3208': [],\n",
       " 'NewsArticles-2156': [],\n",
       " 'NewsArticles-2143': [],\n",
       " 'NewsArticles-2730': [],\n",
       " 'NewsArticles-3159': [],\n",
       " 'NewsArticles-3201': [],\n",
       " 'NewsArticles-3353': [['purchased', 'a', 'house', 'outside', 'Mexico']],\n",
       " 'NewsArticles-355': [],\n",
       " 'NewsArticles-422': [],\n",
       " 'NewsArticles-2867': [['along', 'White', 'House', 'fence', '\\n\\n'],\n",
       "  ['the', 'White', 'House', ',', 'sources'],\n",
       "  ['the', 'White', 'House', 'fence', ','],\n",
       "  ['    ', 'White', 'House', 'press', 'secretary'],\n",
       "  ['the', 'White', 'House', 'fence', '.'],\n",
       "  ['a', 'White', 'House', 'fence', 'last']],\n",
       " ...}"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import kwic\n",
    "\n",
    "kwic(corpus_orig, 'house', ignore_case=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function returns a dictionary that maps document labels to the KWIC results. Each document contains a list of \"contexts\", i.e. a list of tokens that surround a keyword, here `\"house\"`. This keyword stands in the middle and is surrounded by its \"context tokens\", which by default means two tokens to the left and two tokens to the right (which may be less when the keyword is near the start or the end of a document). \n",
    "\n",
    "We can see that \"NewsArticles-2487\" and \"NewsArticles-49\" contain one context, \"NewsArticles-2301\" contains three contexts, etc., but most documents don't contain the search pattern and hence provide an empty list as result.\n",
    "\n",
    "With `kwic_table`, we get back a dataframe which provides a better formatting for quick investigation. See how the matched tokens are highlighted as `*house*` and empty results are removed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.106643Z",
     "iopub.status.busy": "2023-05-03T13:22:52.106464Z",
     "iopub.status.idle": "2023-05-03T13:22:52.141427Z",
     "shell.execute_reply": "2023-05-03T13:22:52.141028Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>context</th>\n",
       "      <th>token</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>0</td>\n",
       "      <td>new White *House* is being</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>1</td>\n",
       "      <td>his White *House* was in</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>2</td>\n",
       "      <td>his White *House* and a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1263</td>\n",
       "      <td>0</td>\n",
       "      <td>near our *house* . I</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1546</td>\n",
       "      <td>0</td>\n",
       "      <td>of White *House* counselor -</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1610</td>\n",
       "      <td>0</td>\n",
       "      <td>. White *House* spokesman Sean</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2132</td>\n",
       "      <td>0</td>\n",
       "      <td>the White *House* . \"</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2132</td>\n",
       "      <td>1</td>\n",
       "      <td>a White *House* gathering of</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2301</td>\n",
       "      <td>0</td>\n",
       "      <td>on the *House* of Representatives</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2301</td>\n",
       "      <td>1</td>\n",
       "      <td>to the *House* Intelligence Committee</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  context                                  token\n",
       "0  NewsArticles-1119        0             new White *House* is being\n",
       "1  NewsArticles-1119        1               his White *House* was in\n",
       "2  NewsArticles-1119        2                his White *House* and a\n",
       "0  NewsArticles-1263        0                   near our *house* . I\n",
       "0  NewsArticles-1546        0           of White *House* counselor -\n",
       "0  NewsArticles-1610        0         . White *House* spokesman Sean\n",
       "0  NewsArticles-2132        0                  the White *House* . \"\n",
       "1  NewsArticles-2132        1           a White *House* gathering of\n",
       "0  NewsArticles-2301        0      on the *House* of Representatives\n",
       "1  NewsArticles-2301        1  to the *House* Intelligence Committee"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import kwic_table\n",
    "\n",
    "# showing only the first 10 results\n",
    "kwic_table(corpus_orig, 'house', ignore_case=True).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An important parameter is `context_size`. It determines the number of tokens to display left and right to the found keyword. You can either pass a single integer for a symmetric context or a tuple with integers `(<left>, <right>)`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.143296Z",
     "iopub.status.busy": "2023-05-03T13:22:52.143182Z",
     "iopub.status.idle": "2023-05-03T13:22:52.175832Z",
     "shell.execute_reply": "2023-05-03T13:22:52.175547Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>context</th>\n",
       "      <th>token</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>0</td>\n",
       "      <td>way his new White *House* is being portrayed and</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>1</td>\n",
       "      <td>reports that his White *House* was in chaos ,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>2</td>\n",
       "      <td>coverage of his White *House* and a desire to</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1263</td>\n",
       "      <td>0</td>\n",
       "      <td>militants exploded near our *house* . I was fr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1546</td>\n",
       "      <td>0</td>\n",
       "      <td>bizarre image of White *House* counselor - Kel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1610</td>\n",
       "      <td>0</td>\n",
       "      <td>he said . White *House* spokesman Sean Spicer ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2132</td>\n",
       "      <td>0</td>\n",
       "      <td>Cabinet at the White *House* . \" Hopefully we</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2132</td>\n",
       "      <td>1</td>\n",
       "      <td>He told a White *House* gathering of Americans...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2301</td>\n",
       "      <td>0</td>\n",
       "      <td>from lawmakers on the *House* of Representativ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2301</td>\n",
       "      <td>1</td>\n",
       "      <td>by Monday to the *House* Intelligence Committe...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  context  \\\n",
       "0  NewsArticles-1119        0   \n",
       "1  NewsArticles-1119        1   \n",
       "2  NewsArticles-1119        2   \n",
       "0  NewsArticles-1263        0   \n",
       "0  NewsArticles-1546        0   \n",
       "0  NewsArticles-1610        0   \n",
       "0  NewsArticles-2132        0   \n",
       "1  NewsArticles-2132        1   \n",
       "0  NewsArticles-2301        0   \n",
       "1  NewsArticles-2301        1   \n",
       "\n",
       "                                               token  \n",
       "0   way his new White *House* is being portrayed and  \n",
       "1      reports that his White *House* was in chaos ,  \n",
       "2      coverage of his White *House* and a desire to  \n",
       "0  militants exploded near our *house* . I was fr...  \n",
       "0  bizarre image of White *House* counselor - Kel...  \n",
       "0  he said . White *House* spokesman Sean Spicer ...  \n",
       "0      Cabinet at the White *House* . \" Hopefully we  \n",
       "1  He told a White *House* gathering of Americans...  \n",
       "0  from lawmakers on the *House* of Representativ...  \n",
       "1  by Monday to the *House* Intelligence Committe...  "
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 4 tokens to each side of the keyword (only display first 10 rows)\n",
    "kwic_table(corpus_orig, 'house', ignore_case=True, context_size=4).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.177292Z",
     "iopub.status.busy": "2023-05-03T13:22:52.177162Z",
     "iopub.status.idle": "2023-05-03T13:22:52.213520Z",
     "shell.execute_reply": "2023-05-03T13:22:52.213130Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>context</th>\n",
       "      <th>token</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>0</td>\n",
       "      <td>White *House* is being portrayed and</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>1</td>\n",
       "      <td>White *House* was in chaos ,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>2</td>\n",
       "      <td>White *House* and a desire to</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1263</td>\n",
       "      <td>0</td>\n",
       "      <td>our *house* . I was frightened</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1546</td>\n",
       "      <td>0</td>\n",
       "      <td>White *House* counselor - Kellyanne Conway</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1610</td>\n",
       "      <td>0</td>\n",
       "      <td>White *House* spokesman Sean Spicer said</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2132</td>\n",
       "      <td>0</td>\n",
       "      <td>White *House* . \" Hopefully we</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2132</td>\n",
       "      <td>1</td>\n",
       "      <td>White *House* gathering of Americans who</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2301</td>\n",
       "      <td>0</td>\n",
       "      <td>the *House* of Representatives Intelligence Co...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2301</td>\n",
       "      <td>1</td>\n",
       "      <td>the *House* Intelligence Committee , which</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  context  \\\n",
       "0  NewsArticles-1119        0   \n",
       "1  NewsArticles-1119        1   \n",
       "2  NewsArticles-1119        2   \n",
       "0  NewsArticles-1263        0   \n",
       "0  NewsArticles-1546        0   \n",
       "0  NewsArticles-1610        0   \n",
       "0  NewsArticles-2132        0   \n",
       "1  NewsArticles-2132        1   \n",
       "0  NewsArticles-2301        0   \n",
       "1  NewsArticles-2301        1   \n",
       "\n",
       "                                               token  \n",
       "0               White *House* is being portrayed and  \n",
       "1                       White *House* was in chaos ,  \n",
       "2                      White *House* and a desire to  \n",
       "0                     our *house* . I was frightened  \n",
       "0         White *House* counselor - Kellyanne Conway  \n",
       "0           White *House* spokesman Sean Spicer said  \n",
       "0                     White *House* . \" Hopefully we  \n",
       "1           White *House* gathering of Americans who  \n",
       "0  the *House* of Representatives Intelligence Co...  \n",
       "1         the *House* Intelligence Committee , which  "
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1 token to the left, 4 tokens to the right of the keyword (only display first 10 rows)\n",
    "kwic_table(corpus_orig, 'house', ignore_case=True, context_size=(1, 4)).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The KWIC functions become really powerful when using the pattern matching options. So far, we were looking for *exact* (but case insensitive) matches between the corpus tokens and our keyword `\"house\"`. However, it is also possible to match patterns like `\"new*\"` (matches any token starting with \"new\") or `\"agenc(y|ies)\"` (a regular expression matching \"agency\" and \"agencies\"). The next section gives an introduction on the different options for pattern matching.\n",
    "\n",
    "#### Common parameters for pattern matching functions\n",
    "\n",
    "Several functions and methods in tmtoolkit support pattern matching, including the already mentioned function `find_documents` and the KWIC functions, but also functions for filtering tokens or documents as you will see later. They all share similar function signatures, i.e. similar parameters:\n",
    "\n",
    "- `search_token` or `search_tokens`: allows to specify one or more patterns as strings\n",
    "- `match_type`: sets the matching type and can be one of the following options:\n",
    "  - `'exact'` (default): exact string matching (optionally ignoring character case), i.e. no pattern matching\n",
    "  - `'regex'` uses [regular expression](https://docs.python.org/3/library/re.html) matching\n",
    "  - `'glob'` uses \"glob patterns\" like `\"politic*\"` which matches for example \"politic\", \"politics\" or \"politician\" (see [globre package](https://pypi.org/project/globre/))\n",
    "- `ignore_case`: ignore character case (applies to all three match types)\n",
    "- `glob_method`: if `match_type` is 'glob', use this glob method. Must be `'match'` or `'search'` (similar behavior as Python's [re.match](https://docs.python.org/3/library/re.html#re.match) or [re.search](https://docs.python.org/3/library/re.html#re.search))\n",
    "- `inverse`: inverse the match results, i.e. if matching for \"hello\", return all results that do *not* match \"hello\"\n",
    "\n",
    "Let's try out some of these options with `kwic_table`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.215474Z",
     "iopub.status.busy": "2023-05-03T13:22:52.215344Z",
     "iopub.status.idle": "2023-05-03T13:22:52.252991Z",
     "shell.execute_reply": "2023-05-03T13:22:52.252628Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>context</th>\n",
       "      <th>token</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>in various *agencies* who had</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1377</td>\n",
       "      <td>0</td>\n",
       "      <td>Anadolu news *agency* . Earlier</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1377</td>\n",
       "      <td>1</td>\n",
       "      <td>and news *agencies*</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1561</td>\n",
       "      <td>0</td>\n",
       "      <td>National Crime *Agency* every month</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1561</td>\n",
       "      <td>1</td>\n",
       "      <td>National Crime *Agency* for specialist</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1610</td>\n",
       "      <td>0</td>\n",
       "      <td>. Source:-News *agencies*</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1860</td>\n",
       "      <td>0</td>\n",
       "      <td>state news *agency* . The</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2156</td>\n",
       "      <td>0</td>\n",
       "      <td>ministries and *agencies* in all</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2301</td>\n",
       "      <td>0</td>\n",
       "      <td>. Source:-News *agencies*</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2433</td>\n",
       "      <td>0</td>\n",
       "      <td>said the *agency* has not</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  context                                   token\n",
       "0  NewsArticles-1100        0           in various *agencies* who had\n",
       "0  NewsArticles-1377        0         Anadolu news *agency* . Earlier\n",
       "1  NewsArticles-1377        1                     and news *agencies*\n",
       "0  NewsArticles-1561        0     National Crime *Agency* every month\n",
       "1  NewsArticles-1561        1  National Crime *Agency* for specialist\n",
       "0  NewsArticles-1610        0               . Source:-News *agencies*\n",
       "0  NewsArticles-1860        0               state news *agency* . The\n",
       "0  NewsArticles-2156        0        ministries and *agencies* in all\n",
       "0  NewsArticles-2301        0               . Source:-News *agencies*\n",
       "0  NewsArticles-2433        0               said the *agency* has not"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# using a regular expression, ignoring case (only display first 10 rows)\n",
    "kwic_table(corpus_orig, r'agenc(y|ies)', match_type='regex', ignore_case=True).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.254988Z",
     "iopub.status.busy": "2023-05-03T13:22:52.254827Z",
     "iopub.status.idle": "2023-05-03T13:22:52.304850Z",
     "shell.execute_reply": "2023-05-03T13:22:52.304344Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>context</th>\n",
       "      <th>token</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>0</td>\n",
       "      <td>modern American *political* history .</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>1</td>\n",
       "      <td>than the *political* media ,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>2</td>\n",
       "      <td>his own *poll* numbers ,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>3</td>\n",
       "      <td>Washington 's *political* establishment and</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>4</td>\n",
       "      <td>Trump among *political* elites in</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1185</td>\n",
       "      <td>0</td>\n",
       "      <td>over a *police* - officer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1185</td>\n",
       "      <td>1</td>\n",
       "      <td>the latest *police* violation to</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1185</td>\n",
       "      <td>2</td>\n",
       "      <td>- against *police* brutality.-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1185</td>\n",
       "      <td>3</td>\n",
       "      <td>when the *police* stopped him</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1185</td>\n",
       "      <td>4</td>\n",
       "      <td>by the *police* circulated on</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  context                                        token\n",
       "0  NewsArticles-1119        0        modern American *political* history .\n",
       "1  NewsArticles-1119        1                 than the *political* media ,\n",
       "2  NewsArticles-1119        2                     his own *poll* numbers ,\n",
       "3  NewsArticles-1119        3  Washington 's *political* establishment and\n",
       "4  NewsArticles-1119        4            Trump among *political* elites in\n",
       "0  NewsArticles-1185        0                    over a *police* - officer\n",
       "1  NewsArticles-1185        1             the latest *police* violation to\n",
       "2  NewsArticles-1185        2             - against *police* brutality.-  \n",
       "3  NewsArticles-1185        3                when the *police* stopped him\n",
       "4  NewsArticles-1185        4                by the *police* circulated on"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# using a glob, ignoring case (only display first 10 rows)\n",
    "kwic_table(corpus_orig, 'pol*', match_type='glob', ignore_case=True).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.306717Z",
     "iopub.status.busy": "2023-05-03T13:22:52.306601Z",
     "iopub.status.idle": "2023-05-03T13:22:52.350294Z",
     "shell.execute_reply": "2023-05-03T13:22:52.349757Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>context</th>\n",
       "      <th>token</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>0</td>\n",
       "      <td>leaks are *absolutely* real .</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1185</td>\n",
       "      <td>0</td>\n",
       "      <td>the biggest *unresolved* problem is</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1407</td>\n",
       "      <td>0</td>\n",
       "      <td>of Belfast *solicitor* - Pat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1587</td>\n",
       "      <td>0</td>\n",
       "      <td>North 's *isolationist* regime ,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1587</td>\n",
       "      <td>1</td>\n",
       "      <td>in the *isolated* country ,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1787</td>\n",
       "      <td>0</td>\n",
       "      <td>roundabout legal *solution* for an</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1860</td>\n",
       "      <td>0</td>\n",
       "      <td>embraced the *isolated* state ,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1860</td>\n",
       "      <td>1</td>\n",
       "      <td>have been *sold* to North</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-2152</td>\n",
       "      <td>0</td>\n",
       "      <td>\" rock *solid* \" support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-2152</td>\n",
       "      <td>1</td>\n",
       "      <td>a negotiated *solution* and deny</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  context                                token\n",
       "0  NewsArticles-1119        0        leaks are *absolutely* real .\n",
       "0  NewsArticles-1185        0  the biggest *unresolved* problem is\n",
       "0  NewsArticles-1407        0         of Belfast *solicitor* - Pat\n",
       "0  NewsArticles-1587        0     North 's *isolationist* regime ,\n",
       "1  NewsArticles-1587        1          in the *isolated* country ,\n",
       "0  NewsArticles-1787        0   roundabout legal *solution* for an\n",
       "0  NewsArticles-1860        0      embraced the *isolated* state ,\n",
       "1  NewsArticles-1860        1            have been *sold* to North\n",
       "0  NewsArticles-2152        0             \" rock *solid* \" support\n",
       "1  NewsArticles-2152        1     a negotiated *solution* and deny"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# using a glob, ignoring case (only display first 10 rows)\n",
    "kwic_table(corpus_orig, '*sol*', match_type='glob', ignore_case=True).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.352341Z",
     "iopub.status.busy": "2023-05-03T13:22:52.352216Z",
     "iopub.status.idle": "2023-05-03T13:22:52.473637Z",
     "shell.execute_reply": "2023-05-03T13:22:52.472991Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>context</th>\n",
       "      <th>token</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>investigate leaks *\\n\\n* President Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>his administration *.* The news</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>leaks are *\"* very serious</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>very serious *.* \" \"</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>serious . *\"* \" I</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>5</td>\n",
       "      <td>. \" *\"* I 've</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>6</td>\n",
       "      <td>the leaks *.* Those are</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>7</td>\n",
       "      <td>criminal leaks *,* \" Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>8</td>\n",
       "      <td>leaks , *\"* Trump said</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>9</td>\n",
       "      <td>this afternoon *.* \" We</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 doc  context                                     token\n",
       "0  NewsArticles-1100        0  investigate leaks *\\n\\n* President Trump\n",
       "1  NewsArticles-1100        1           his administration *.* The news\n",
       "2  NewsArticles-1100        2                leaks are *\"* very serious\n",
       "3  NewsArticles-1100        3                      very serious *.* \" \"\n",
       "4  NewsArticles-1100        4                         serious . *\"* \" I\n",
       "5  NewsArticles-1100        5                             . \" *\"* I 've\n",
       "6  NewsArticles-1100        6                   the leaks *.* Those are\n",
       "7  NewsArticles-1100        7                criminal leaks *,* \" Trump\n",
       "8  NewsArticles-1100        8                    leaks , *\"* Trump said\n",
       "9  NewsArticles-1100        9                   this afternoon *.* \" We"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# using a regex that matches all tokens with at least one vowel and\n",
    "# inverting these matches, i.e. all tokens *without* any vowels\n",
    "# (only display first 10 rows)\n",
    "kwic_table(corpus_orig, r'[AEIOUaeiou]', match_type='regex', inverse=True).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Filtering tokens and documents\n",
    "\n",
    "We can use the pattern matching parameters in numerous filtering methods. The heart of many of these methods is [token_match](api.rst#tmtoolkit.tokenseq.token_match). Given a search pattern, a list of tokens and optionally some pattern matching parameters, it returns a boolean NumPy array of the same length as the input tokens. Each occurrence of `True` in this boolean array signals a match."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.475361Z",
     "iopub.status.busy": "2023-05-03T13:22:52.475221Z",
     "iopub.status.idle": "2023-05-03T13:22:52.479855Z",
     "shell.execute_reply": "2023-05-03T13:22:52.479309Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('President', False),\n",
       " ('Trump', False),\n",
       " ('says', False),\n",
       " ('he', False),\n",
       " ('has', False),\n",
       " ('asked', False),\n",
       " ('the', False),\n",
       " ('Justice', False),\n",
       " ('Department', False),\n",
       " ('to', True),\n",
       " ('investigate', False),\n",
       " ('leaks', False),\n",
       " ('\\n\\n', False),\n",
       " ('President', False),\n",
       " ('Trump', False),\n",
       " ('said', False),\n",
       " ('today', True),\n",
       " ('he', False),\n",
       " ('has', False),\n",
       " ('directed', False)]"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.tokenseq import token_match\n",
    "\n",
    "# first 20 tokens of document \"NewsArticles-1100\"\n",
    "doc_snippet = corpus_orig['NewsArticles-1100']['token'][:20]\n",
    "# get all tokens that match \"to*\"\n",
    "matches = token_match('to*', doc_snippet, match_type='glob')\n",
    "\n",
    "# show pair-wise results\n",
    "list(zip(doc_snippet, matches))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `token_match` function is a rather low-level function that you may use for pattern matching against any list/array of strings, e.g. a list of tokens, file names, etc.\n",
    "\n",
    "The following functions cover common use-cases for filtering during text preprocessing. Many of these functions start either with `filter_...()` or `remove_...()` and these pairs of filter and remove functions are complements. A *filter function* will always *retain* the matched elements whereas a *remove function* will always *drop* the matched elements. Note that a remove function is actually a shortcut for a filter function with the parameter `inverse=True`.\n",
    "\n",
    "We can observe that behavior with the first pair of functions, [filter_tokens](api.rst#tmtoolkit.corpus.filter_tokens) and [remove_tokens](api.rst#tmtoolkit.corpus.remove_tokens). Since these functions *modify* a corpus, you can again choose to make these modifications to the existing corpus object (\"in-place\") or return a modified corpus using the `inplace` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.481489Z",
     "iopub.status.busy": "2023-05-03T13:22:52.481319Z",
     "iopub.status.idle": "2023-05-03T13:22:52.572552Z",
     "shell.execute_reply": "2023-05-03T13:22:52.572054Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (0 tokens): \n",
      "> NewsArticles-1407 (0 tokens): \n",
      "> NewsArticles-1100 (0 tokens): \n",
      "> NewsArticles-1119 (3 tokens): House House House\n",
      "> NewsArticles-1472 (0 tokens): \n",
      "> NewsArticles-1377 (0 tokens): \n",
      "> NewsArticles-1185 (0 tokens): \n",
      "> NewsArticles-1263 (1 tokens): house\n",
      "> NewsArticles-1353 (0 tokens): \n",
      "> NewsArticles-1387 (0 tokens): \n",
      "(and 90 more documents)\n",
      "total number of tokens: 53 / vocabulary size: 8\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import filter_tokens\n",
    "\n",
    "# retain only the tokens that match the pattern in each document\n",
    "corpus_filtered = filter_tokens(corpus_orig, '*house*', match_type='glob',\n",
    "                                ignore_case=True, inplace=False)\n",
    "print_summary(corpus_filtered)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.574511Z",
     "iopub.status.busy": "2023-05-03T13:22:52.574397Z",
     "iopub.status.idle": "2023-05-03T13:22:52.676466Z",
     "shell.execute_reply": "2023-05-03T13:22:52.675825Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Trump suggests Obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterates Govt support for Finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): President Trump says he has asked the Justice Depa...\n",
      "> NewsArticles-1119 (972 tokens): An amazing moment in history : Donald Trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland sees losses widening    Bai...\n",
      "> NewsArticles-1377 (774 tokens): Turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1185 (1271 tokens): For more than a week,-France - has been rocked by ...\n",
      "> NewsArticles-1263 (409 tokens): Russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-1353 (30 tokens): Islamic State battle : Fierce gunfight outside Mos...\n",
      "> NewsArticles-1387 (513 tokens): Protests after Anaheim policeman drags teen , fire...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 59545 / vocabulary size: 9215\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import remove_tokens\n",
    "\n",
    "# remove the tokens that match the pattern in each document\n",
    "corpus_filtered = remove_tokens(corpus_orig, '*house*', match_type='glob',\n",
    "                                ignore_case=True, inplace=False)\n",
    "print_summary(corpus_filtered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pair [filter_documents](api.rst#tmtoolkit.corpus.filter_documents) and [remove_documents](api.rst#tmtoolkit.corpus.remove_documents) works similarily, but filters or drops whole documents regarding the supplied match criteria. Both accept the standard pattern matching parameters, but also a parameter `matches_threshold` with default value `1`. When this number of matching tokens is hit, the document will be part of the result set (`filter_documents`) or removed from the result set (`remove_documents`). By this, we can for example retain only those documents that contain certain token patterns.\n",
    "\n",
    "Let's try out these functions in practice:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.678947Z",
     "iopub.status.busy": "2023-05-03T13:22:52.678724Z",
     "iopub.status.idle": "2023-05-03T13:22:52.765028Z",
     "shell.execute_reply": "2023-05-03T13:22:52.764335Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 21 documents in English\n",
      "> NewsArticles-1610 (380 tokens): Jewish community centres hit by wave of bomb threa...\n",
      "> NewsArticles-2867 (170 tokens): Person detained after hopping bike - rack barrier ...\n",
      "> NewsArticles-1119 (975 tokens): An amazing moment in history : Donald Trump 's pre...\n",
      "> NewsArticles-2487 (1015 tokens): Dutch election : High turnout in key national vote...\n",
      "> NewsArticles-1546 (277 tokens): Kellyanne Conway 's Ultra - Casual Oval Office Pho...\n",
      "> NewsArticles-2431 (1678 tokens): Will Europe ride the populist wave ? A visual guid...\n",
      "> NewsArticles-2132 (490 tokens): Trump on health care : ' It 's a big , fat , beaut...\n",
      "> NewsArticles-2641 (1225 tokens): Muslim Artist 's Dreamy Nude Self - Portraits Show...\n",
      "> NewsArticles-1263 (410 tokens): Russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-2301 (464 tokens): DOJ seeks more time on Trump wiretapping inquiry  ...\n",
      "(and 11 more documents)\n",
      "total number of tokens: 16655 / vocabulary size: 3813\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import filter_documents\n",
    "\n",
    "corpus_filtered = filter_documents(corpus_orig, '*house*', match_type='glob',\n",
    "                                   ignore_case=True, inplace=False)\n",
    "print_summary(corpus_filtered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that 21 out of 100 documents contained the pattern `'*house*'` and hence were retained.\n",
    "\n",
    "We can also adjust `matches_threshold` to set the minimum number of token matches for filtering:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.767189Z",
     "iopub.status.busy": "2023-05-03T13:22:52.767071Z",
     "iopub.status.idle": "2023-05-03T13:22:52.840142Z",
     "shell.execute_reply": "2023-05-03T13:22:52.839719Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 5 documents in English\n",
      "> NewsArticles-3156 (554 tokens): Republicans working on changes to healthcare overh...\n",
      "> NewsArticles-3739 (491 tokens): Trump ally : Ivanka Trump 's new gig is n't nepoti...\n",
      "> NewsArticles-2867 (170 tokens): Person detained after hopping bike - rack barrier ...\n",
      "> NewsArticles-72 (1054 tokens): Speaker John Bercow defends his comments on Donald...\n",
      "> NewsArticles-3094 (694 tokens): House Intel Chair : Trump Administration Documents...\n",
      "total number of tokens: 2963 / vocabulary size: 936\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import filter_documents\n",
    "\n",
    "corpus_filtered = filter_documents(corpus_orig, '*house*', match_type='glob',\n",
    "                                   matches_threshold=4,\n",
    "                                   ignore_case=True, inplace=False)\n",
    "print_summary(corpus_filtered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Five out of 100 documents contained the pattern `'*house*'` at least four times and hence were retained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.842433Z",
     "iopub.status.busy": "2023-05-03T13:22:52.842278Z",
     "iopub.status.idle": "2023-05-03T13:22:52.925325Z",
     "shell.execute_reply": "2023-05-03T13:22:52.924911Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 79 documents in English\n",
      "> NewsArticles-1561 (539 tokens): Do not jail all paedophiles , says police chief   ...\n",
      "> NewsArticles-1519 (246 tokens): SpaceX announces planned private trip around moon ...\n",
      "> NewsArticles-1515 (426 tokens): Trump suggests Obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterates Govt support for Finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): President Trump says he has asked the Justice Depa...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland sees losses widening    Bai...\n",
      "> NewsArticles-1185 (1271 tokens): For more than a week,-France - has been rocked by ...\n",
      "> NewsArticles-1377 (774 tokens): Turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1353 (30 tokens): Islamic State battle : Fierce gunfight outside Mos...\n",
      "> NewsArticles-1387 (513 tokens): Protests after Anaheim policeman drags teen , fire...\n",
      "(and 69 more documents)\n",
      "total number of tokens: 42943 / vocabulary size: 7573\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import remove_documents\n",
    "\n",
    "corpus_filtered = remove_documents(corpus_orig, '*house*', match_type='glob',\n",
    "                 ignore_case=True, inplace=False)\n",
    "print_summary(corpus_filtered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we use `remove_documents` we get only the documents that did *not* contain the specified pattern. Since we had 21 documents that contained the \"house\" pattern, we now have the complement set with the 79 documents that don't contain this pattern."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another useful pair of functions is [filter_documents_by_label](api.rst#tmtoolkit.corpus.filter_documents_by_label) and [remove_documents_by_label](api.rst#tmtoolkit.corpus.remove_documents_by_label). Both functions again accept the same pattern matching parameters but they only apply them to the document names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.927616Z",
     "iopub.status.busy": "2023-05-03T13:22:52.927500Z",
     "iopub.status.idle": "2023-05-03T13:22:52.988533Z",
     "shell.execute_reply": "2023-05-03T13:22:52.988097Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 73 documents in English\n",
      "> NewsArticles-1515 (426 tokens): Trump suggests Obama was ' behind ' town hall prot...\n",
      "> NewsArticles-1407 (202 tokens): Minister reiterates Govt support for Finucane inqu...\n",
      "> NewsArticles-1100 (224 tokens): President Trump says he has asked the Justice Depa...\n",
      "> NewsArticles-1119 (975 tokens): An amazing moment in history : Donald Trump 's pre...\n",
      "> NewsArticles-1472 (298 tokens): Royal Bank of Scotland sees losses widening    Bai...\n",
      "> NewsArticles-1377 (774 tokens): Turkey - backed rebels in ' near full control ' of...\n",
      "> NewsArticles-1185 (1271 tokens): For more than a week,-France - has been rocked by ...\n",
      "> NewsArticles-1263 (410 tokens): Russian doctors use mobile field hospital to provi...\n",
      "> NewsArticles-1353 (30 tokens): Islamic State battle : Fierce gunfight outside Mos...\n",
      "> NewsArticles-1387 (513 tokens): Protests after Anaheim policeman drags teen , fire...\n",
      "(and 63 more documents)\n",
      "total number of tokens: 43114 / vocabulary size: 7598\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import filter_documents_by_label\n",
    "\n",
    "corpus_filtered = filter_documents_by_label(corpus_orig, r'-\\d{4}$',\n",
    "                                            match_type='regex', inplace=False)\n",
    "print_summary(corpus_filtered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above example we wanted to retain only the documents whose document labels ended with exactly 4 digits, like \"...-1234\". Hence, we only get \"NewsArticles-1880\" and \"NewsArticles-3350\" but not \"NewsArticles-99\". Again, `remove_documents_by_label` will do the exact opposite."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may also use [Keywords-in-context (KWIC)](#Keywords-in-context-(KWIC)-and-general-filtering-methods) to filter your tokens in the neighborhood around certain keyword pattern(s). The method for that is called [filter_tokens_with_kwic](api.rst#tmtoolkit.corpus.filter_tokens_with_kwic) and works very similar to [kwic](api.rst#tmtoolkit.corpus.kwic), but filters the documents in the `Corpus` instance with which you can continue working as usual. Here, we filter the tokens in each document to get the tokens directly in front and after the glob pattern `'*house*'` (`context_size=1`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:52.990377Z",
     "iopub.status.busy": "2023-05-03T13:22:52.990259Z",
     "iopub.status.idle": "2023-05-03T13:22:53.065304Z",
     "shell.execute_reply": "2023-05-03T13:22:53.064888Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (0 tokens): \n",
      "> NewsArticles-1407 (0 tokens): \n",
      "> NewsArticles-1100 (0 tokens): \n",
      "> NewsArticles-1119 (9 tokens): White House is White House was White House and\n",
      "> NewsArticles-1472 (0 tokens): \n",
      "> NewsArticles-1377 (0 tokens): \n",
      "> NewsArticles-1185 (0 tokens): \n",
      "> NewsArticles-1263 (3 tokens): our house .\n",
      "> NewsArticles-1353 (0 tokens): \n",
      "> NewsArticles-1387 (0 tokens): \n",
      "(and 90 more documents)\n",
      "total number of tokens: 158 / vocabulary size: 50\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import filter_tokens_with_kwic\n",
    "\n",
    "corpus_filtered = filter_tokens_with_kwic(corpus_orig, '*house*',\n",
    "                                          context_size=1, match_type='glob',\n",
    "                                          ignore_case=True, inplace=False)\n",
    "print_summary(corpus_filtered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When your NLP pipeline annotated your documents' tokens with Part-of-Speech (POS) tags, you can also filter them using [filter_for_pos](api.rst#tmtoolkit.corpus.filter_for_pos):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:53.067475Z",
     "iopub.status.busy": "2023-05-03T13:22:53.067283Z",
     "iopub.status.idle": "2023-05-03T13:22:53.170186Z",
     "shell.execute_reply": "2023-05-03T13:22:53.169783Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>is_punct</th>\n",
       "      <th>is_stop</th>\n",
       "      <th>lemma</th>\n",
       "      <th>like_num</th>\n",
       "      <th>pos</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>President</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>President</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>Trump</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Trump</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>Justice</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Justice</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>Department</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Department</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>leaks</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>leak</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NNS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17594</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>90</td>\n",
       "      <td>Putin</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Putin</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17595</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>91</td>\n",
       "      <td>Russia</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Russia</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17596</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>92</td>\n",
       "      <td>capabilities</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>capability</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NNS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17597</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>93</td>\n",
       "      <td>priorities</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>priority</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NNS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17598</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>94</td>\n",
       "      <td>nation</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>nation</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>17599 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position         token  is_punct  is_stop  \\\n",
       "0      NewsArticles-1100         0     President     False    False   \n",
       "1      NewsArticles-1100         1         Trump     False    False   \n",
       "2      NewsArticles-1100         2       Justice     False    False   \n",
       "3      NewsArticles-1100         3    Department     False    False   \n",
       "4      NewsArticles-1100         4         leaks     False    False   \n",
       "...                  ...       ...           ...       ...      ...   \n",
       "17594   NewsArticles-960        90         Putin     False    False   \n",
       "17595   NewsArticles-960        91        Russia     False    False   \n",
       "17596   NewsArticles-960        92  capabilities     False    False   \n",
       "17597   NewsArticles-960        93    priorities     False    False   \n",
       "17598   NewsArticles-960        94        nation     False    False   \n",
       "\n",
       "            lemma  like_num    pos  tag  \n",
       "0       President     False  PROPN  NNP  \n",
       "1           Trump     False  PROPN  NNP  \n",
       "2         Justice     False  PROPN  NNP  \n",
       "3      Department     False  PROPN  NNP  \n",
       "4            leak     False   NOUN  NNS  \n",
       "...           ...       ...    ...  ...  \n",
       "17594       Putin     False  PROPN  NNP  \n",
       "17595      Russia     False  PROPN  NNP  \n",
       "17596  capability     False   NOUN  NNS  \n",
       "17597    priority     False   NOUN  NNS  \n",
       "17598      nation     False   NOUN   NN  \n",
       "\n",
       "[17599 rows x 9 columns]"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import filter_for_pos\n",
    "\n",
    "# \"N\" means filter for nouns\n",
    "corpus_filtered = filter_for_pos(corpus_orig, 'N', inplace=False)\n",
    "tokens_table(corpus_filtered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we filtered for tokens that were identified as nouns by passing the *simplified POS tag* `'N'` (for more on simplified tags, see the function documentation). We can also filter for more than one tag, e.g. nouns or verbs by passing a list of required POS tags."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `filter_for_pos` function has no `remove_...` counterpart, but you can set the `inverse` parameter to `True` to achieve the same effect.\n",
    "\n",
    "Finally there are functions for removing tokens based on their document frequency: [filter_tokens_by_doc_frequency](api.rst#tmtoolkit.corpus.filter_tokens_by_doc_frequency) along with the shortcut functions [remove_common_tokens](api.rst#tmtoolkit.corpus.remove_common_tokens) and [remove_uncommon_tokens](api.rst#tmtoolkit.corpus.remove_uncommon_tokens). The former removes all tokens that have a document frequency greater or equal a certain threshold defined by parameter `df_threshold`. The latter does the same for all tokens that have a document frequency lower or equal `df_threshold`. This parameter accepts a frequency proportion (default) or absolute count (via parameter `proportions`).\n",
    "\n",
    "Before applying the function, let's have a look at the total number of tokens again, to later see how many we  removed. We will also store the vocabulary in `orig_vocab` for later comparison:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:53.172244Z",
     "iopub.status.busy": "2023-05-03T13:22:53.172081Z",
     "iopub.status.idle": "2023-05-03T13:22:53.175035Z",
     "shell.execute_reply": "2023-05-03T13:22:53.174764Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "59598"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import doc_lengths\n",
    "\n",
    "orig_vocab = vocabulary(corpus_orig, sort=False)\n",
    "corpus_num_tokens(corpus_orig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:53.176804Z",
     "iopub.status.busy": "2023-05-03T13:22:53.176650Z",
     "iopub.status.idle": "2023-05-03T13:22:53.257255Z",
     "shell.execute_reply": "2023-05-03T13:22:53.256868Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "44102"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import remove_common_tokens\n",
    "\n",
    "corpus_filtered = remove_common_tokens(corpus_orig, df_threshold=0.9, inplace=False)\n",
    "corpus_num_tokens(corpus_filtered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By removing all tokens with a document frequency threshold of 0.9, we removed quite a number of tokens in each document. Let's investigate the vocabulary in order to see which tokens were removed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:53.259036Z",
     "iopub.status.busy": "2023-05-03T13:22:53.258883Z",
     "iopub.status.idle": "2023-05-03T13:22:53.262107Z",
     "shell.execute_reply": "2023-05-03T13:22:53.261734Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'\\n\\n',\n",
       " \"'s\",\n",
       " ',',\n",
       " '.',\n",
       " 'a',\n",
       " 'and',\n",
       " 'in',\n",
       " 'is',\n",
       " 'of',\n",
       " 'on',\n",
       " 'that',\n",
       " 'the',\n",
       " 'to'}"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# set difference gives removed vocabulary tokens\n",
    "orig_vocab - vocabulary(corpus_filtered, sort=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that this – as expected – removed very common token types.\n",
    "\n",
    "The `remove_uncommon_tokens` function works similarily. This time, let's use an absolute number as threshold:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:53.263994Z",
     "iopub.status.busy": "2023-05-03T13:22:53.263820Z",
     "iopub.status.idle": "2023-05-03T13:22:53.351589Z",
     "shell.execute_reply": "2023-05-03T13:22:53.351288Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'whatsoever',\n",
       " 'revisionist',\n",
       " 'erratic',\n",
       " 'Trust',\n",
       " 'pic.twitter.com/ctNZhFYjtv',\n",
       " 'thriving',\n",
       " 'capture',\n",
       " 'Neeshan',\n",
       " 'promoters',\n",
       " 'Bad',\n",
       " 'jokily',\n",
       " 'raping',\n",
       " 'headstones',\n",
       " 'testing',\n",
       " 'Coral',\n",
       " 'indecent',\n",
       " 'clinics',\n",
       " 'fundraising',\n",
       " 'divestiture',\n",
       " 'Fillon',\n",
       " ...}"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import remove_uncommon_tokens\n",
    "\n",
    "corpus_filtered = remove_uncommon_tokens(corpus_orig, df_threshold=1,\n",
    "                                         proportions=0, inplace=False)\n",
    "\n",
    "# set difference gives removed vocabulary tokens\n",
    "orig_vocab - vocabulary(corpus_filtered, sort=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above means that we removed all tokens that appear only in exactly one document. As expected, these are rather uncommon token types.\n",
    "\n",
    "There are more filtering functions available. See the [corpus functions API](api.rst#tmtoolkit-corpus) and search for `filter_` functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Token cooccurrence matrices\n",
    "\n",
    "We can use [token_cooccurrence](api.rst#tmtoolkit.corpus.token_cooccurrence) to calculate a token cooccurrence matrix, i.e. the number of times a token pair cooccures within a given context window:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:53.353151Z",
     "iopub.status.busy": "2023-05-03T13:22:53.353039Z",
     "iopub.status.idle": "2023-05-03T13:22:56.431341Z",
     "shell.execute_reply": "2023-05-03T13:22:56.430880Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<2898x2898 sparse matrix of type '<class 'numpy.int32'>'\n",
       "\twith 98901 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import token_cooccurrence\n",
    "from tmtoolkit.utils import pairwise_max_table\n",
    "\n",
    "# remove stopwords and other uninformative tokens\n",
    "to_lowercase(corpus_filtered)\n",
    "filter_clean_tokens(corpus_filtered, remove_numbers=True)\n",
    "\n",
    "# use `return_tokens` to obtain the labels of the matrix\n",
    "cooc, cooc_labels = token_cooccurrence(corpus_filtered,\n",
    "                                       context_size=3,\n",
    "                                       return_tokens=True)\n",
    "cooc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, the returned matrix is a sparse matrix as indicated above and it is computed for the whole vocabulary, i.e. for all unique tokens in the corpus. We obtain the vocabulary via `return_tokens=True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.433151Z",
     "iopub.status.busy": "2023-05-03T13:22:56.433015Z",
     "iopub.status.idle": "2023-05-03T13:22:56.436413Z",
     "shell.execute_reply": "2023-05-03T13:22:56.436032Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['$',\n",
       " '+',\n",
       " '/tass/.',\n",
       " '01/27/2017',\n",
       " '12/16/2016',\n",
       " '1970s',\n",
       " '1980s',\n",
       " '80s',\n",
       " 'a.m.',\n",
       " 'abc']"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cooc_labels[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's display the cooccurrences for the first 10 tokens in `cooc_labels`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.438045Z",
     "iopub.status.busy": "2023-05-03T13:22:56.437802Z",
     "iopub.status.idle": "2023-05-03T13:22:56.440809Z",
     "shell.execute_reply": "2023-05-03T13:22:56.440431Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[8, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=int32)"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cooc[:10, :10].todense()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tells us for example that within a context window of 3 tokens to the left and to the right of the token `\"$\"` there is another `\"$\"` counted eight times for the given corpus.\n",
    "\n",
    "The [pairwise_max_table](api.rst#tmtoolkit.utils.pairwise_max_table) helps to obtain the top cooccurrences in tabular form:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.442669Z",
     "iopub.status.busy": "2023-05-03T13:22:56.442533Z",
     "iopub.status.idle": "2023-05-03T13:22:56.485032Z",
     "shell.execute_reply": "2023-05-03T13:22:56.484476Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2675</th>\n",
       "      <td>trump</td>\n",
       "      <td>donald</td>\n",
       "      <td>43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>762</th>\n",
       "      <td>donald</td>\n",
       "      <td>trump</td>\n",
       "      <td>43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1951</th>\n",
       "      <td>president</td>\n",
       "      <td>trump</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2247</th>\n",
       "      <td>said</td>\n",
       "      <td>trump</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2897</th>\n",
       "      <td>�</td>\n",
       "      <td>�</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>853</th>\n",
       "      <td>enjoy</td>\n",
       "      <td>caring</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>854</th>\n",
       "      <td>enjoying</td>\n",
       "      <td>boost</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>856</th>\n",
       "      <td>ensuring</td>\n",
       "      <td>added</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1996</th>\n",
       "      <td>prompted</td>\n",
       "      <td>anti</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1449</th>\n",
       "      <td>lawyer</td>\n",
       "      <td>accused</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2898 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              x        y  value\n",
       "2675      trump   donald     43\n",
       "762      donald    trump     43\n",
       "1951  president    trump     40\n",
       "2247       said    trump     37\n",
       "2897          �        �     32\n",
       "...         ...      ...    ...\n",
       "853       enjoy   caring      1\n",
       "854    enjoying    boost      1\n",
       "856    ensuring    added      1\n",
       "1996   prompted     anti      1\n",
       "1449     lawyer  accused      1\n",
       "\n",
       "[2898 rows x 3 columns]"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pairwise_max_table(cooc, labels=cooc_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also generate a token cooccurrence matrix for a subset of the corpus vocabulary by passing it via the `tokens` argument. We additionally set the output as dataframe via `as_table=True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.487473Z",
     "iopub.status.busy": "2023-05-03T13:22:56.487275Z",
     "iopub.status.idle": "2023-05-03T13:22:56.503038Z",
     "shell.execute_reply": "2023-05-03T13:22:56.502687Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trump</th>\n",
       "      <th>obama</th>\n",
       "      <th>merkel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>trump</th>\n",
       "      <td>32</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>obama</th>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>merkel</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        trump  obama  merkel\n",
       "trump      32     19       0\n",
       "obama      19      0       0\n",
       "merkel      0      0       2"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cooc_specific_tokens = ['trump', 'obama', 'merkel']\n",
    "cooc_specific = token_cooccurrence(corpus_filtered,\n",
    "                                   context_size=5,\n",
    "                                   tokens=cooc_specific_tokens,\n",
    "                                   as_table=True)\n",
    "cooc_specific"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tells us for example that within a context window of five tokens left and right of the token \"trump\", the token \"obama\" appears 19 times (and vice versa since the context window is symmetric)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Working with document and token attributes\n",
    "\n",
    "Each document and each token in a corpus can have an arbitrary number of *attributes* attached to it. Think of these attributes as meta information or \"annotations\" at document or token level. An example for a document attribute is the document label, i.e. its name. An example for a token attribute is the POS tag.\n",
    "\n",
    "While the mentioned examples are attributes that tmtoolkit creates itself, you can also create your own attributes. For example, you may add a publication year as document attribute or a token attribute that indicates whether a token is in all caps. You can then use these attributes for example for filtering or in further analyses.\n",
    "\n",
    "#### Document attributes\n",
    "\n",
    "There are two functions for adding or updating document or token attributes, respectively: [set_document_attr](api.rst#tmtoolkit.corpus.set_document_attr) and [set_token_attr](api.rst#tmtoolkit.corpus.set_token_attr). We'll start with adding a new document attribute, `year`. At first we need to provide the attribute data as dict that maps document labels to document attribute values. For the purpose of this tutorial, we'll simply make up some data by drawing a random year for each document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.504960Z",
     "iopub.status.busy": "2023-05-03T13:22:56.504797Z",
     "iopub.status.idle": "2023-05-03T13:22:56.508440Z",
     "shell.execute_reply": "2023-05-03T13:22:56.507967Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'NewsArticles-2433': 2020,\n",
       " 'NewsArticles-2225': 2020,\n",
       " 'NewsArticles-2487': 2016,\n",
       " 'NewsArticles-49': 2017,\n",
       " 'NewsArticles-469': 2016,\n",
       " 'NewsArticles-2766': 2017,\n",
       " 'NewsArticles-2712': 2019,\n",
       " 'NewsArticles-2301': 2019,\n",
       " 'NewsArticles-1377': 2020,\n",
       " 'NewsArticles-3428': 2016,\n",
       " 'NewsArticles-3208': 2015,\n",
       " 'NewsArticles-2156': 2018,\n",
       " 'NewsArticles-2143': 2019,\n",
       " 'NewsArticles-2730': 2019,\n",
       " 'NewsArticles-3159': 2015,\n",
       " 'NewsArticles-3201': 2018,\n",
       " 'NewsArticles-3353': 2016,\n",
       " 'NewsArticles-355': 2019,\n",
       " 'NewsArticles-422': 2019,\n",
       " 'NewsArticles-2867': 2019,\n",
       " ...}"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc_years = {lbl: random.randint(2015, 2020) for lbl in corpus_orig}\n",
    "doc_years"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now use `set_document_attr` to create the new document attribute and pass the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.510013Z",
     "iopub.status.busy": "2023-05-03T13:22:56.509859Z",
     "iopub.status.idle": "2023-05-03T13:22:56.569299Z",
     "shell.execute_reply": "2023-05-03T13:22:56.568707Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('label', 'has_sents', 'year')"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import set_document_attr\n",
    "\n",
    "corpus_new = set_document_attr(corpus_orig, 'year', data=doc_years, inplace=False)\n",
    "# using the `doc_attrs` property to check that the new attribute is recorded:\n",
    "corpus_new.doc_attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we investigate the token table, we can see a new column `year` which is constant for each document:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.571672Z",
     "iopub.status.busy": "2023-05-03T13:22:56.571486Z",
     "iopub.status.idle": "2023-05-03T13:22:56.688993Z",
     "shell.execute_reply": "2023-05-03T13:22:56.688466Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>is_punct</th>\n",
       "      <th>is_stop</th>\n",
       "      <th>lemma</th>\n",
       "      <th>like_num</th>\n",
       "      <th>pos</th>\n",
       "      <th>tag</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>President</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>President</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>Trump</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Trump</td>\n",
       "      <td>False</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>says</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>say</td>\n",
       "      <td>False</td>\n",
       "      <td>VERB</td>\n",
       "      <td>VBZ</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>he</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>he</td>\n",
       "      <td>False</td>\n",
       "      <td>PRON</td>\n",
       "      <td>PRP</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>has</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>have</td>\n",
       "      <td>False</td>\n",
       "      <td>AUX</td>\n",
       "      <td>VBZ</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59593</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>282</td>\n",
       "      <td>priorities</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>priority</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NNS</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59594</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>283</td>\n",
       "      <td>for</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>for</td>\n",
       "      <td>False</td>\n",
       "      <td>ADP</td>\n",
       "      <td>IN</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59595</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>284</td>\n",
       "      <td>the</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>the</td>\n",
       "      <td>False</td>\n",
       "      <td>DET</td>\n",
       "      <td>DT</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59596</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>285</td>\n",
       "      <td>nation</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>nation</td>\n",
       "      <td>False</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>NN</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59597</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>286</td>\n",
       "      <td>.</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>.</td>\n",
       "      <td>False</td>\n",
       "      <td>PUNCT</td>\n",
       "      <td>.</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59598 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position       token  is_punct  is_stop      lemma  \\\n",
       "0      NewsArticles-1100         0   President     False    False  President   \n",
       "1      NewsArticles-1100         1       Trump     False    False      Trump   \n",
       "2      NewsArticles-1100         2        says     False    False        say   \n",
       "3      NewsArticles-1100         3          he     False     True         he   \n",
       "4      NewsArticles-1100         4         has     False     True       have   \n",
       "...                  ...       ...         ...       ...      ...        ...   \n",
       "59593   NewsArticles-960       282  priorities     False    False   priority   \n",
       "59594   NewsArticles-960       283         for     False     True        for   \n",
       "59595   NewsArticles-960       284         the     False     True        the   \n",
       "59596   NewsArticles-960       285      nation     False    False     nation   \n",
       "59597   NewsArticles-960       286           .      True    False          .   \n",
       "\n",
       "       like_num    pos  tag  year  \n",
       "0         False  PROPN  NNP  2018  \n",
       "1         False  PROPN  NNP  2018  \n",
       "2         False   VERB  VBZ  2018  \n",
       "3         False   PRON  PRP  2018  \n",
       "4         False    AUX  VBZ  2018  \n",
       "...         ...    ...  ...   ...  \n",
       "59593     False   NOUN  NNS  2016  \n",
       "59594     False    ADP   IN  2016  \n",
       "59595     False    DET   DT  2016  \n",
       "59596     False   NOUN   NN  2016  \n",
       "59597     False  PUNCT    .  2016  \n",
       "\n",
       "[59598 rows x 10 columns]"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens_table(corpus_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above example, we set a document attribute value for each document in the corpus. However, you can also just set values for a subset of the documents. All other documents will then also contain that document attribute, but with a default value which is determined with the `default` parameter. Let's find out all documents that contain the token \"president\" (ignoring case). This is only a subset of all documents. We create an attribute dictionary that assigns `True` to all these documents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.690793Z",
     "iopub.status.busy": "2023-05-03T13:22:56.690658Z",
     "iopub.status.idle": "2023-05-03T13:22:56.715330Z",
     "shell.execute_reply": "2023-05-03T13:22:56.714963Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'NewsArticles-2433': True,\n",
       " 'NewsArticles-2225': True,\n",
       " 'NewsArticles-2487': True,\n",
       " 'NewsArticles-49': True,\n",
       " 'NewsArticles-2766': True,\n",
       " 'NewsArticles-2301': True,\n",
       " 'NewsArticles-1377': True,\n",
       " 'NewsArticles-3208': True,\n",
       " 'NewsArticles-2143': True,\n",
       " 'NewsArticles-3201': True,\n",
       " 'NewsArticles-3353': True,\n",
       " 'NewsArticles-2502': True,\n",
       " 'NewsArticles-2431': True,\n",
       " 'NewsArticles-3309': True,\n",
       " 'NewsArticles-1185': True,\n",
       " 'NewsArticles-21': True,\n",
       " 'NewsArticles-2132': True,\n",
       " 'NewsArticles-1119': True,\n",
       " 'NewsArticles-549': True,\n",
       " 'NewsArticles-760': True,\n",
       " ...}"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "president_docs = find_documents(corpus_new, 'president', ignore_case=True)\n",
    "president_attrs = {lbl: True for lbl in president_docs}\n",
    "president_attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now use this dictionary to create a new document attribute `president`. All documents not contained in `president_attrs` will get the default attribute value `False`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.717572Z",
     "iopub.status.busy": "2023-05-03T13:22:56.717333Z",
     "iopub.status.idle": "2023-05-03T13:22:56.762136Z",
     "shell.execute_reply": "2023-05-03T13:22:56.761589Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>year</th>\n",
       "      <th>president</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2018</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>224</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>2017</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1199</th>\n",
       "      <td>NewsArticles-1185</td>\n",
       "      <td>2017</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2470</th>\n",
       "      <td>NewsArticles-1263</td>\n",
       "      <td>2016</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2880</th>\n",
       "      <td>NewsArticles-1353</td>\n",
       "      <td>2020</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57599</th>\n",
       "      <td>NewsArticles-770</td>\n",
       "      <td>2017</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57987</th>\n",
       "      <td>NewsArticles-780</td>\n",
       "      <td>2016</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58110</th>\n",
       "      <td>NewsArticles-836</td>\n",
       "      <td>2015</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58397</th>\n",
       "      <td>NewsArticles-901</td>\n",
       "      <td>2015</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59311</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>2016</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  year  president\n",
       "0      NewsArticles-1100  2018       True\n",
       "224    NewsArticles-1119  2017       True\n",
       "1199   NewsArticles-1185  2017       True\n",
       "2470   NewsArticles-1263  2016      False\n",
       "2880   NewsArticles-1353  2020      False\n",
       "...                  ...   ...        ...\n",
       "57599   NewsArticles-770  2017      False\n",
       "57987   NewsArticles-780  2016      False\n",
       "58110   NewsArticles-836  2015      False\n",
       "58397   NewsArticles-901  2015      False\n",
       "59311   NewsArticles-960  2016       True\n",
       "\n",
       "[100 rows x 3 columns]"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_document_attr(corpus_new, 'president', data=president_attrs, default=False)\n",
    "toktbl = tokens_table(corpus_new, with_attr=['year', 'president'])\n",
    "\n",
    "# only show rows with document name, year and president indicator\n",
    "toktbl[['doc', 'year', 'president']].drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Token attributes\n",
    "\n",
    "Similar to document attributes, we can use `set_token_attr` for creating or updating token attributes. However, this function has two modes of assigning attribute values to tokens. The default mode assigns each token occurrence, i.e. each token type, a certain attribute value. We will start with this mode.\n",
    "\n",
    "We set a new token attribute `obama` and simply assign a boolean value to each token which is `True` when the token is \"Obama\", else `False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.763954Z",
     "iopub.status.busy": "2023-05-03T13:22:56.763851Z",
     "iopub.status.idle": "2023-05-03T13:22:56.771731Z",
     "shell.execute_reply": "2023-05-03T13:22:56.771305Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('is_punct', 'is_stop', 'like_num', 'tag', 'pos', 'lemma', 'obama')"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import set_token_attr\n",
    "\n",
    "set_token_attr(corpus_new, 'obama', data={'Obama': True}, default=False)\n",
    "# check the token attributes property\n",
    "corpus_new.token_attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that it worked, but it isn't really useful:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.773315Z",
     "iopub.status.busy": "2023-05-03T13:22:56.773223Z",
     "iopub.status.idle": "2023-05-03T13:22:56.866343Z",
     "shell.execute_reply": "2023-05-03T13:22:56.865809Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>is_punct</th>\n",
       "      <th>is_stop</th>\n",
       "      <th>lemma</th>\n",
       "      <th>like_num</th>\n",
       "      <th>obama</th>\n",
       "      <th>pos</th>\n",
       "      <th>president</th>\n",
       "      <th>tag</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>137</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1072</th>\n",
       "      <td>NewsArticles-1119</td>\n",
       "      <td>848</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4699</th>\n",
       "      <td>NewsArticles-1515</td>\n",
       "      <td>2</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4715</th>\n",
       "      <td>NewsArticles-1515</td>\n",
       "      <td>18</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4769</th>\n",
       "      <td>NewsArticles-1515</td>\n",
       "      <td>72</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4815</th>\n",
       "      <td>NewsArticles-1515</td>\n",
       "      <td>118</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4927</th>\n",
       "      <td>NewsArticles-1515</td>\n",
       "      <td>230</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4950</th>\n",
       "      <td>NewsArticles-1515</td>\n",
       "      <td>253</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4992</th>\n",
       "      <td>NewsArticles-1515</td>\n",
       "      <td>295</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5087</th>\n",
       "      <td>NewsArticles-1515</td>\n",
       "      <td>390</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>Obama</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "      <td>NNP</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    doc  position  token  is_punct  is_stop  lemma  like_num  \\\n",
       "137   NewsArticles-1100       137  Obama     False    False  Obama     False   \n",
       "1072  NewsArticles-1119       848  Obama     False    False  Obama     False   \n",
       "4699  NewsArticles-1515         2  Obama     False    False  Obama     False   \n",
       "4715  NewsArticles-1515        18  Obama     False    False  Obama     False   \n",
       "4769  NewsArticles-1515        72  Obama     False    False  Obama     False   \n",
       "4815  NewsArticles-1515       118  Obama     False    False  Obama     False   \n",
       "4927  NewsArticles-1515       230  Obama     False    False  Obama     False   \n",
       "4950  NewsArticles-1515       253  Obama     False    False  Obama     False   \n",
       "4992  NewsArticles-1515       295  Obama     False    False  Obama     False   \n",
       "5087  NewsArticles-1515       390  Obama     False    False  Obama     False   \n",
       "\n",
       "      obama    pos  president  tag  year  \n",
       "137    True  PROPN       True  NNP  2018  \n",
       "1072   True  PROPN       True  NNP  2017  \n",
       "4699   True  PROPN       True  NNP  2016  \n",
       "4715   True  PROPN       True  NNP  2016  \n",
       "4769   True  PROPN       True  NNP  2016  \n",
       "4815   True  PROPN       True  NNP  2016  \n",
       "4927   True  PROPN       True  NNP  2016  \n",
       "4950   True  PROPN       True  NNP  2016  \n",
       "4992   True  PROPN       True  NNP  2016  \n",
       "5087   True  PROPN       True  NNP  2016  "
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "toktbl = tokens_table(corpus_new)\n",
    "toktbl[toktbl.obama].head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second mode to assign token attribute values is much more useful. In this mode, you provide a dictionary that maps a document label to a list or array of token attribute values. The list's/array's size must match the number of tokens in the respective document. With this, you can assign an attribute value to each token in each document. We will use this to add a token attribute that records the number of characters in each token. At first, we generate that data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.868392Z",
     "iopub.status.busy": "2023-05-03T13:22:56.868243Z",
     "iopub.status.idle": "2023-05-03T13:22:56.878849Z",
     "shell.execute_reply": "2023-05-03T13:22:56.878545Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9, 5, 4, 2, 3, 5, 3, 7, 10, 2, 11, 5, 2, 9, 5, 4, 5, 2, 3, 8, ...]"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc_toks = doc_tokens(corpus_new)\n",
    "\n",
    "tok_lengths = {lbl: list(map(len, tok)) for lbl, tok in doc_toks.items()}\n",
    "# show the number of characters for each token in a sample document\n",
    "tok_lengths['NewsArticles-1100']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we pass this data but set `per_token_occurrence=False` to indicate that the data contains attribute values per token in each document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.880637Z",
     "iopub.status.busy": "2023-05-03T13:22:56.880531Z",
     "iopub.status.idle": "2023-05-03T13:22:56.917617Z",
     "shell.execute_reply": "2023-05-03T13:22:56.917301Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>nchar</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>President</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>Trump</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>says</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>he</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>has</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59593</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>282</td>\n",
       "      <td>priorities</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59594</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>283</td>\n",
       "      <td>for</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59595</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>284</td>\n",
       "      <td>the</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59596</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>285</td>\n",
       "      <td>nation</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59597</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>286</td>\n",
       "      <td>.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59598 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position       token  nchar\n",
       "0      NewsArticles-1100         0   President      9\n",
       "1      NewsArticles-1100         1       Trump      5\n",
       "2      NewsArticles-1100         2        says      4\n",
       "3      NewsArticles-1100         3          he      2\n",
       "4      NewsArticles-1100         4         has      3\n",
       "...                  ...       ...         ...    ...\n",
       "59593   NewsArticles-960       282  priorities     10\n",
       "59594   NewsArticles-960       283         for      3\n",
       "59595   NewsArticles-960       284         the      3\n",
       "59596   NewsArticles-960       285      nation      6\n",
       "59597   NewsArticles-960       286           .      1\n",
       "\n",
       "[59598 rows x 4 columns]"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_token_attr(corpus_new, 'nchar', data=tok_lengths, per_token_occurrence=False)\n",
    "tokens_table(corpus_new, with_attr='nchar')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Removing attributes\n",
    "\n",
    "Document and token attributes can be removed with [remove_document_attr](api.rst#tmtoolkit.corpus.remove_document_attr) and [remove_token_attr](api.rst#tmtoolkit.corpus.remove_token_attr) respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.919688Z",
     "iopub.status.busy": "2023-05-03T13:22:56.919525Z",
     "iopub.status.idle": "2023-05-03T13:22:56.922137Z",
     "shell.execute_reply": "2023-05-03T13:22:56.921874Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('label', 'has_sents', 'president')"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import remove_document_attr\n",
    "\n",
    "remove_document_attr(corpus_new, 'year')\n",
    "corpus_new.doc_attrs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.923901Z",
     "iopub.status.busy": "2023-05-03T13:22:56.923802Z",
     "iopub.status.idle": "2023-05-03T13:22:56.926315Z",
     "shell.execute_reply": "2023-05-03T13:22:56.926063Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('is_punct', 'is_stop', 'like_num', 'tag', 'pos', 'lemma', 'nchar')"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import remove_token_attr\n",
    "\n",
    "remove_token_attr(corpus_new, 'obama')\n",
    "corpus_new.token_attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can tell [filter_tokens](api.rst#tmtoolkit.corpus.filter_tokens) and similar functions to use document or token attributes instead of the tokens for matching. The common parameter name for this option is `by_attr`. For example, we can use the `nchar` attribute, which we created before, to filter for tokens of a certain length:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.928223Z",
     "iopub.status.busy": "2023-05-03T13:22:56.928086Z",
     "iopub.status.idle": "2023-05-03T13:22:56.990722Z",
     "shell.execute_reply": "2023-05-03T13:22:56.990329Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>nchar</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>has</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>the</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>has</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>the</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>the</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9277</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>39</td>\n",
       "      <td>its</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9278</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>40</td>\n",
       "      <td>has</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9279</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>41</td>\n",
       "      <td>the</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9280</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>42</td>\n",
       "      <td>for</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9281</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>43</td>\n",
       "      <td>the</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9282 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                    doc  position token  nchar\n",
       "0     NewsArticles-1100         0   has      3\n",
       "1     NewsArticles-1100         1   the      3\n",
       "2     NewsArticles-1100         2   has      3\n",
       "3     NewsArticles-1100         3   the      3\n",
       "4     NewsArticles-1100         4   the      3\n",
       "...                 ...       ...   ...    ...\n",
       "9277   NewsArticles-960        39   its      3\n",
       "9278   NewsArticles-960        40   has      3\n",
       "9279   NewsArticles-960        41   the      3\n",
       "9280   NewsArticles-960        42   for      3\n",
       "9281   NewsArticles-960        43   the      3\n",
       "\n",
       "[9282 rows x 4 columns]"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corpus_3chars = filter_tokens(corpus_new, 3, by_attr='nchar', inplace=False)\n",
    "tokens_table(corpus_3chars, with_attr='nchar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.992703Z",
     "iopub.status.busy": "2023-05-03T13:22:56.992582Z",
     "iopub.status.idle": "2023-05-03T13:22:56.995387Z",
     "shell.execute_reply": "2023-05-03T13:22:56.994957Z"
    }
   },
   "outputs": [],
   "source": [
    "del corpus_3chars"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that all matching options then apply to the token attribute column, in this case to the `nchar` column which contains integers. Since `filter_tokens` by default employs exact matching, we get all tokens where `nchar` equals the first argument, `3`. If we used regular expression or glob matching instead, this method would fail because you can only use that for string data.\n",
    "\n",
    "If you want to use more complex filter queries, you should create a \"filter mask\" and pass it to [filter_tokens_by_mask](api.rst#tmtoolkit.corpus.filter_tokens_by_mask). A filter mask is a dictionary that maps a document label to a sequence of boolean values. For all occurrences of `True`, the respective token in the document will be retained, all others will be removed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try that out with a small example: We now generate the filter mask, which means for each document we create a boolean list or array that for each token in that document indicates whether that token should be kept or removed.\n",
    "\n",
    "We will iterate through the document tokens with attributes supplied from `doc_tokens`. We set `as_arrays=True` to obtain the `nchar` and `pos` token attributes for each document as NumPy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:56.997237Z",
     "iopub.status.busy": "2023-05-03T13:22:56.997070Z",
     "iopub.status.idle": "2023-05-03T13:22:57.028374Z",
     "shell.execute_reply": "2023-05-03T13:22:57.027794Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([3, 1, 1, 7, 5, 8, 2, 5, 4, 2]),\n",
       " array(['NOUN', 'PUNCT', 'NUM', 'ADJ', 'NOUN', 'VERB', 'ADP', 'PROPN',\n",
       "        'NOUN', 'SPACE'], dtype='<U5'))"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc_tokattrs = doc_tokens(corpus_new, with_attr=['nchar', 'pos'], as_arrays=True)\n",
    "\n",
    "# show number of characters and POS tag of \n",
    "# first 10 tokens for a sample document\n",
    "(doc_tokattrs['NewsArticles-2433']['nchar'][:10],\n",
    " doc_tokattrs['NewsArticles-2433']['pos'][:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can create the filter mask. Since we generated the token attribute data as NumPy arrays before, we can directly and efficiently use NumPy functions such as `np.isin`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.030432Z",
     "iopub.status.busy": "2023-05-03T13:22:57.030319Z",
     "iopub.status.idle": "2023-05-03T13:22:57.087476Z",
     "shell.execute_reply": "2023-05-03T13:22:57.087004Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>nchar</th>\n",
       "      <th>pos</th>\n",
       "      <th>small_nouns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>President</td>\n",
       "      <td>9</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>Trump</td>\n",
       "      <td>5</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>says</td>\n",
       "      <td>4</td>\n",
       "      <td>VERB</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>he</td>\n",
       "      <td>2</td>\n",
       "      <td>PRON</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>has</td>\n",
       "      <td>3</td>\n",
       "      <td>AUX</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59593</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>282</td>\n",
       "      <td>priorities</td>\n",
       "      <td>10</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59594</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>283</td>\n",
       "      <td>for</td>\n",
       "      <td>3</td>\n",
       "      <td>ADP</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59595</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>284</td>\n",
       "      <td>the</td>\n",
       "      <td>3</td>\n",
       "      <td>DET</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59596</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>285</td>\n",
       "      <td>nation</td>\n",
       "      <td>6</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59597</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>286</td>\n",
       "      <td>.</td>\n",
       "      <td>1</td>\n",
       "      <td>PUNCT</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59598 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position       token  nchar    pos  small_nouns\n",
       "0      NewsArticles-1100         0   President      9  PROPN        False\n",
       "1      NewsArticles-1100         1       Trump      5  PROPN         True\n",
       "2      NewsArticles-1100         2        says      4   VERB        False\n",
       "3      NewsArticles-1100         3          he      2   PRON        False\n",
       "4      NewsArticles-1100         4         has      3    AUX        False\n",
       "...                  ...       ...         ...    ...    ...          ...\n",
       "59593   NewsArticles-960       282  priorities     10   NOUN        False\n",
       "59594   NewsArticles-960       283         for      3    ADP        False\n",
       "59595   NewsArticles-960       284         the      3    DET        False\n",
       "59596   NewsArticles-960       285      nation      6   NOUN        False\n",
       "59597   NewsArticles-960       286           .      1  PUNCT        False\n",
       "\n",
       "[59598 rows x 6 columns]"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "filter_mask = {}\n",
    "for doc_label, doc_data in doc_tokattrs.items():\n",
    "    tok_lengths = doc_data['nchar']\n",
    "    tok_pos = doc_data['pos']\n",
    "    # create a boolean array for nouns with token length less or equal 5\n",
    "    filter_mask[doc_label] = (tok_lengths <= 5) & np.isin(tok_pos, ['NOUN', 'PROPN'])\n",
    "\n",
    "# it's not necessary to add the filter mask as token attribute\n",
    "# but it's a good way to check the mask\n",
    "set_token_attr(corpus_new, 'small_nouns', data=filter_mask, per_token_occurrence=False)\n",
    "tokens_table(corpus_new, with_attr=['nchar', 'pos', 'small_nouns'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can pass the mask dict to `filter_tokens_by_mask`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.089557Z",
     "iopub.status.busy": "2023-05-03T13:22:57.089458Z",
     "iopub.status.idle": "2023-05-03T13:22:57.113138Z",
     "shell.execute_reply": "2023-05-03T13:22:57.112529Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>nchar</th>\n",
       "      <th>pos</th>\n",
       "      <th>small_nouns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>Trump</td>\n",
       "      <td>5</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>leaks</td>\n",
       "      <td>5</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>Trump</td>\n",
       "      <td>5</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>today</td>\n",
       "      <td>5</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>leaks</td>\n",
       "      <td>5</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6913</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>29</td>\n",
       "      <td>Trump</td>\n",
       "      <td>5</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6914</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>30</td>\n",
       "      <td>U.S.</td>\n",
       "      <td>4</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6915</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>31</td>\n",
       "      <td>rest</td>\n",
       "      <td>4</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6916</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>32</td>\n",
       "      <td>world</td>\n",
       "      <td>5</td>\n",
       "      <td>NOUN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6917</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>33</td>\n",
       "      <td>Putin</td>\n",
       "      <td>5</td>\n",
       "      <td>PROPN</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>6918 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                    doc  position  token  nchar    pos  small_nouns\n",
       "0     NewsArticles-1100         0  Trump      5  PROPN         True\n",
       "1     NewsArticles-1100         1  leaks      5   NOUN         True\n",
       "2     NewsArticles-1100         2  Trump      5  PROPN         True\n",
       "3     NewsArticles-1100         3  today      5   NOUN         True\n",
       "4     NewsArticles-1100         4  leaks      5   NOUN         True\n",
       "...                 ...       ...    ...    ...    ...          ...\n",
       "6913   NewsArticles-960        29  Trump      5  PROPN         True\n",
       "6914   NewsArticles-960        30   U.S.      4  PROPN         True\n",
       "6915   NewsArticles-960        31   rest      4   NOUN         True\n",
       "6916   NewsArticles-960        32  world      5   NOUN         True\n",
       "6917   NewsArticles-960        33  Putin      5  PROPN         True\n",
       "\n",
       "[6918 rows x 6 columns]"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import filter_tokens_by_mask\n",
    "\n",
    "filter_tokens_by_mask(corpus_new, mask=filter_mask)\n",
    "tokens_table(corpus_new, with_attr=['nchar', 'pos', 'small_nouns'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generating n-grams\n",
    "\n",
    "So far, we worked with *unigrams*, i.e. each document consisted of a sequence of discrete tokens. We can also generate *n-grams* from our corpus where each document consists of a sequence of *n* subsequent tokens. An example would be:\n",
    "\n",
    "Document: \"This is a simple example.\"\n",
    "\n",
    "**n=1 (unigrams):**\n",
    "\n",
    "    ['This', 'is', 'a', 'simple', 'example', '.']\n",
    "\n",
    "**n=2 (bigrams):**\n",
    "\n",
    "    ['This is', 'is a', 'a simple', 'simple example', 'example .']\n",
    "\n",
    "**n=3 (trigrams):**\n",
    "\n",
    "    ['This is a', 'is a simple', 'a simple example', 'simple example .']\n",
    "\n",
    "The function [ngrams](api.rst#tmtoolkit.corpus.ngrams) allows to generate n-grams. Here, we create bigrams and display a sample document, \"NewsArticles-2433\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.115159Z",
     "iopub.status.busy": "2023-05-03T13:22:57.114937Z",
     "iopub.status.idle": "2023-05-03T13:22:57.151669Z",
     "shell.execute_reply": "2023-05-03T13:22:57.151169Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['DOJ :',\n",
       " ': 2',\n",
       " '2 Russian',\n",
       " 'Russian spies',\n",
       " 'spies indicted',\n",
       " 'indicted in',\n",
       " 'in Yahoo',\n",
       " 'Yahoo hack',\n",
       " 'hack \\n\\n',\n",
       " '\\n\\n Washington',\n",
       " 'Washington (',\n",
       " '( CNN)The',\n",
       " 'CNN)The Department',\n",
       " 'Department of',\n",
       " 'of Justice',\n",
       " 'Justice announced',\n",
       " 'announced Wednesday',\n",
       " 'Wednesday that',\n",
       " 'that four',\n",
       " 'four people',\n",
       " ...]"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import ngrams\n",
    "\n",
    "ngrams(corpus_orig, n=2)['NewsArticles-2433']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, the ngrams are joined with a space (can be adjusted via `join_str`), but you can disable that using `join=False` so that each n-gram is a list of size *n*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.153840Z",
     "iopub.status.busy": "2023-05-03T13:22:57.153713Z",
     "iopub.status.idle": "2023-05-03T13:22:57.259342Z",
     "shell.execute_reply": "2023-05-03T13:22:57.258879Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['DOJ', ':', '2'],\n",
       " [':', '2', 'Russian'],\n",
       " ['2', 'Russian', 'spies'],\n",
       " ['Russian', 'spies', 'indicted'],\n",
       " ['spies', 'indicted', 'in'],\n",
       " ['indicted', 'in', 'Yahoo'],\n",
       " ['in', 'Yahoo', 'hack'],\n",
       " ['Yahoo', 'hack', '\\n\\n'],\n",
       " ['hack', '\\n\\n', 'Washington'],\n",
       " ['\\n\\n', 'Washington', '('],\n",
       " ['Washington', '(', 'CNN)The'],\n",
       " ['(', 'CNN)The', 'Department'],\n",
       " ['CNN)The', 'Department', 'of'],\n",
       " ['Department', 'of', 'Justice'],\n",
       " ['of', 'Justice', 'announced'],\n",
       " ['Justice', 'announced', 'Wednesday'],\n",
       " ['announced', 'Wednesday', 'that'],\n",
       " ['Wednesday', 'that', 'four'],\n",
       " ['that', 'four', 'people'],\n",
       " ['four', 'people', '--'],\n",
       " ...]"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ngrams(corpus_orig, n=3, join=False)['NewsArticles-2433']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the `ngrams` functions just retrieves the tokens as n-grams, but doesn't alter the corpus contents in any way. In contrast to that, when you use the [corpus_ngramify](api.rst#tmtoolkit.corpus.corpus_ngramify) function the corpus object internally stores that you want to operate on n-grams in all further operations.\n",
    "\n",
    "In the following example, we set the corpus to handle its tokens as bigrams. Note how `doc_tokens` then returns bigrams:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.261198Z",
     "iopub.status.busy": "2023-05-03T13:22:57.261025Z",
     "iopub.status.idle": "2023-05-03T13:22:57.341073Z",
     "shell.execute_reply": "2023-05-03T13:22:57.340599Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['DOJ :',\n",
       " ': 2',\n",
       " '2 Russian',\n",
       " 'Russian spies',\n",
       " 'spies indicted',\n",
       " 'indicted in',\n",
       " 'in Yahoo',\n",
       " 'Yahoo hack',\n",
       " 'hack \\n\\n',\n",
       " '\\n\\n Washington',\n",
       " 'Washington (',\n",
       " '( CNN)The',\n",
       " 'CNN)The Department',\n",
       " 'Department of',\n",
       " 'of Justice',\n",
       " 'Justice announced',\n",
       " 'announced Wednesday',\n",
       " 'Wednesday that',\n",
       " 'that four',\n",
       " 'four people',\n",
       " ...]"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import corpus_ngramify\n",
    "\n",
    "corpus_bigrams = corpus_ngramify(corpus_orig, n=2, inplace=False)\n",
    "doc_tokens(corpus_bigrams)['NewsArticles-2433']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same goes for `tokens_table`, too. Notice how each token attribute becomes a bigram, too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.343327Z",
     "iopub.status.busy": "2023-05-03T13:22:57.343180Z",
     "iopub.status.idle": "2023-05-03T13:22:57.668180Z",
     "shell.execute_reply": "2023-05-03T13:22:57.667404Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc</th>\n",
       "      <th>position</th>\n",
       "      <th>token</th>\n",
       "      <th>is_punct</th>\n",
       "      <th>is_stop</th>\n",
       "      <th>lemma</th>\n",
       "      <th>like_num</th>\n",
       "      <th>pos</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>0</td>\n",
       "      <td>President Trump</td>\n",
       "      <td>False False</td>\n",
       "      <td>False False</td>\n",
       "      <td>President Trump</td>\n",
       "      <td>False False</td>\n",
       "      <td>PROPN PROPN</td>\n",
       "      <td>NNP NNP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>1</td>\n",
       "      <td>Trump says</td>\n",
       "      <td>False False</td>\n",
       "      <td>False False</td>\n",
       "      <td>Trump say</td>\n",
       "      <td>False False</td>\n",
       "      <td>PROPN VERB</td>\n",
       "      <td>NNP VBZ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>2</td>\n",
       "      <td>says he</td>\n",
       "      <td>False False</td>\n",
       "      <td>False True</td>\n",
       "      <td>say he</td>\n",
       "      <td>False False</td>\n",
       "      <td>VERB PRON</td>\n",
       "      <td>VBZ PRP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>3</td>\n",
       "      <td>he has</td>\n",
       "      <td>False False</td>\n",
       "      <td>True True</td>\n",
       "      <td>he have</td>\n",
       "      <td>False False</td>\n",
       "      <td>PRON AUX</td>\n",
       "      <td>PRP VBZ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NewsArticles-1100</td>\n",
       "      <td>4</td>\n",
       "      <td>has asked</td>\n",
       "      <td>False False</td>\n",
       "      <td>True False</td>\n",
       "      <td>have ask</td>\n",
       "      <td>False False</td>\n",
       "      <td>AUX VERB</td>\n",
       "      <td>VBZ VBN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59493</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>281</td>\n",
       "      <td>the priorities</td>\n",
       "      <td>False False</td>\n",
       "      <td>True False</td>\n",
       "      <td>the priority</td>\n",
       "      <td>False False</td>\n",
       "      <td>DET NOUN</td>\n",
       "      <td>DT NNS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59494</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>282</td>\n",
       "      <td>priorities for</td>\n",
       "      <td>False False</td>\n",
       "      <td>False True</td>\n",
       "      <td>priority for</td>\n",
       "      <td>False False</td>\n",
       "      <td>NOUN ADP</td>\n",
       "      <td>NNS IN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59495</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>283</td>\n",
       "      <td>for the</td>\n",
       "      <td>False False</td>\n",
       "      <td>True True</td>\n",
       "      <td>for the</td>\n",
       "      <td>False False</td>\n",
       "      <td>ADP DET</td>\n",
       "      <td>IN DT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59496</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>284</td>\n",
       "      <td>the nation</td>\n",
       "      <td>False False</td>\n",
       "      <td>True False</td>\n",
       "      <td>the nation</td>\n",
       "      <td>False False</td>\n",
       "      <td>DET NOUN</td>\n",
       "      <td>DT NN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59497</th>\n",
       "      <td>NewsArticles-960</td>\n",
       "      <td>285</td>\n",
       "      <td>nation .</td>\n",
       "      <td>False True</td>\n",
       "      <td>False False</td>\n",
       "      <td>nation .</td>\n",
       "      <td>False False</td>\n",
       "      <td>NOUN PUNCT</td>\n",
       "      <td>NN .</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59498 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     doc  position            token     is_punct      is_stop  \\\n",
       "0      NewsArticles-1100         0  President Trump  False False  False False   \n",
       "1      NewsArticles-1100         1       Trump says  False False  False False   \n",
       "2      NewsArticles-1100         2          says he  False False   False True   \n",
       "3      NewsArticles-1100         3           he has  False False    True True   \n",
       "4      NewsArticles-1100         4        has asked  False False   True False   \n",
       "...                  ...       ...              ...          ...          ...   \n",
       "59493   NewsArticles-960       281   the priorities  False False   True False   \n",
       "59494   NewsArticles-960       282   priorities for  False False   False True   \n",
       "59495   NewsArticles-960       283          for the  False False    True True   \n",
       "59496   NewsArticles-960       284       the nation  False False   True False   \n",
       "59497   NewsArticles-960       285         nation .   False True  False False   \n",
       "\n",
       "                 lemma     like_num          pos      tag  \n",
       "0      President Trump  False False  PROPN PROPN  NNP NNP  \n",
       "1            Trump say  False False   PROPN VERB  NNP VBZ  \n",
       "2               say he  False False    VERB PRON  VBZ PRP  \n",
       "3              he have  False False     PRON AUX  PRP VBZ  \n",
       "4             have ask  False False     AUX VERB  VBZ VBN  \n",
       "...                ...          ...          ...      ...  \n",
       "59493     the priority  False False     DET NOUN   DT NNS  \n",
       "59494     priority for  False False     NOUN ADP   NNS IN  \n",
       "59495          for the  False False      ADP DET    IN DT  \n",
       "59496       the nation  False False     DET NOUN    DT NN  \n",
       "59497         nation .  False False   NOUN PUNCT     NN .  \n",
       "\n",
       "[59498 rows x 9 columns]"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens_table(corpus_bigrams)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.670122Z",
     "iopub.status.busy": "2023-05-03T13:22:57.670009Z",
     "iopub.status.idle": "2023-05-03T13:22:57.672758Z",
     "shell.execute_reply": "2023-05-03T13:22:57.672382Z"
    }
   },
   "outputs": [],
   "source": [
    "del corpus_bigrams"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generating a sparse document-term matrix (DTM)\n",
    "\n",
    "If you're working with a bag-of-words representation of your data, you usually convert the preprocessed documents to a document-term matrix (DTM), which represents of the number of occurrences of each term (i.e. vocabulary token) in each document. This is a *N* rows by *M* columns matrix, where *N* is the number of documents and *M* is the vocabulary size (i.e. the number of unique tokens in the corpus).\n",
    "\n",
    "Not all tokens from the vocabulary occur in all documents. In fact, many tokens will occur only in a small subset of the documents if you're dealing with a \"real world\" dataset. This means that most entries in such a DTM will be zero. Almost all functions in tmtoolkit therefore generate and work with *sparse matrices*, where only non-zero values are stored in computer memory.\n",
    "\n",
    "For this example, we'll generate a DTM from the `corpus_norm` instance. First, let's check the number of documents and the vocabulary size:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.674203Z",
     "iopub.status.busy": "2023-05-03T13:22:57.674116Z",
     "iopub.status.idle": "2023-05-03T13:22:57.676938Z",
     "shell.execute_reply": "2023-05-03T13:22:57.676613Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 6279)"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(corpus_norm), vocabulary_size(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the [dtm](api.rst#tmtoolkit.corpus.dtm) function to generate a sparse DTM from the current instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.678521Z",
     "iopub.status.busy": "2023-05-03T13:22:57.678352Z",
     "iopub.status.idle": "2023-05-03T13:22:57.755580Z",
     "shell.execute_reply": "2023-05-03T13:22:57.755159Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<100x6279 sparse matrix of type '<class 'numpy.int32'>'\n",
       "\twith 17207 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tmtoolkit.corpus import dtm\n",
    "\n",
    "dtm_norm = dtm(corpus_norm)\n",
    "dtm_norm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that a sparse matrix with 100 rows (which corresponds with the number of documents) and 6271 columns was generated (which corresponds to the vocabulary size). 17204 elements in this matrix are non-zero.\n",
    "\n",
    "We can convert this matrix to a non-sparse, i.e. *dense*, representation and see parts of its elements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.757283Z",
     "iopub.status.busy": "2023-05-03T13:22:57.757182Z",
     "iopub.status.idle": "2023-05-03T13:22:57.760351Z",
     "shell.execute_reply": "2023-05-03T13:22:57.759962Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        ...,\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0],\n",
       "        [0, 0, 0, ..., 0, 0, 0]], dtype=int32)"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dtm_norm.todense()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, note that you should only convert a sparse matrix to a dense representation when you're either dealing with a small amount of data (which is what we're doing in this example), or use only a part of the full matrix. Converting a sparse matrix to a dense representation can otherwise easily exceed the available computer memory.\n",
    "\n",
    "There exist different \"formats\" for sparse matrices, which have different advantages and disadvantages (see for example the [SciPy \"sparse\" module documentation](https://docs.scipy.org/doc/scipy/reference/sparse.html#usage-information)). **Not all formats support all operations that you can usually apply to an ordinary, dense matrix.** By default, the generated DTM is in *Compressed Sparse Row (CSR)* format. This format allows indexing and is especially optimized for fast row access. You may convert it to any other sparse matrix format; see the mentioned SciPy documentation for this.\n",
    "\n",
    "The rows of the DTM are aligned to the *sorted* list of document labels and its columns are aligned to the *sorted* vocabulary. For example, let's find the frequency of the term \"white_house\" in the document \"NewsArticles-2867\" (remember that we transformed all tokens to lower case and joint the collocation \"white house\" in the normalized corpus `corpus_norm`). To do this, we find out the indices into the matrix. By default, the corpus functions `doc_labels` and `vocabulary` return sorted lists, so these will be aligned with our matrix rows and columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.762299Z",
     "iopub.status.busy": "2023-05-03T13:22:57.762163Z",
     "iopub.status.idle": "2023-05-03T13:22:57.764419Z",
     "shell.execute_reply": "2023-05-03T13:22:57.764166Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc_labels(corpus_norm).index('NewsArticles-2867')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.766166Z",
     "iopub.status.busy": "2023-05-03T13:22:57.766069Z",
     "iopub.status.idle": "2023-05-03T13:22:57.769797Z",
     "shell.execute_reply": "2023-05-03T13:22:57.769540Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6146"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab_norm = vocabulary(corpus_norm)   # will later reuse that\n",
    "vocab_norm.index('white_house')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This means the frequency of the term \"white_house\" in the document \"NewsArticles-2867\" is located in row 49 and column 6139 of the DTM:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.771531Z",
     "iopub.status.busy": "2023-05-03T13:22:57.771413Z",
     "iopub.status.idle": "2023-05-03T13:22:57.773608Z",
     "shell.execute_reply": "2023-05-03T13:22:57.773355Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dtm_norm[49, 6139]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check that by using the `count` method on the tokens of this document:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.775414Z",
     "iopub.status.busy": "2023-05-03T13:22:57.775298Z",
     "iopub.status.idle": "2023-05-03T13:22:57.777456Z",
     "shell.execute_reply": "2023-05-03T13:22:57.777239Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corpus_norm['NewsArticles-2867']['token'].count('white_house')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See also the following example of finding out the index for \"administration\" and then getting an array that represents the number of occurrences of this token type across all 100 documents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.779049Z",
     "iopub.status.busy": "2023-05-03T13:22:57.778886Z",
     "iopub.status.idle": "2023-05-03T13:22:57.781879Z",
     "shell.execute_reply": "2023-05-03T13:22:57.781620Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[2, 6, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2, 1, 1, 0, 0, 0, 0, 0,\n",
       "         1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 1, 0, 0, 0,\n",
       "         0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]], dtype=int32)"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab_admin_ix = vocab_norm.index('administration')\n",
    "dtm_norm[:, vocab_admin_ix].todense().flatten()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `dtm` function also allows to directly return the document labels and vocabulary that are aligned to the matrix, which is quicker and less error-prone. This can be achieved with the `return_doc_labels` and `return_vocab` parameters, respectively.\n",
    "\n",
    "Another interesting option is the `as_table` parameter, which returns the result as dataframe. Note that you should only use this option for small datasets, as it returns a dense dataframe that consumes a lot of memory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.783749Z",
     "iopub.status.busy": "2023-05-03T13:22:57.783527Z",
     "iopub.status.idle": "2023-05-03T13:22:57.866191Z",
     "shell.execute_reply": "2023-05-03T13:22:57.865874Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>01062017</th>\n",
       "      <th>01132017</th>\n",
       "      <th>01202017</th>\n",
       "      <th>01272017</th>\n",
       "      <th>0830</th>\n",
       "      <th>1</th>\n",
       "      <th>10</th>\n",
       "      <th>100</th>\n",
       "      <th>1000</th>\n",
       "      <th>10000</th>\n",
       "      <th>...</th>\n",
       "      <th>zapad</th>\n",
       "      <th>zeleny</th>\n",
       "      <th>zeping</th>\n",
       "      <th>zhang</th>\n",
       "      <th>zionist</th>\n",
       "      <th>zone</th>\n",
       "      <th>zor</th>\n",
       "      <th>zuma</th>\n",
       "      <th>�</th>\n",
       "      <th>�_�</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>NewsArticles-1100</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-1119</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-1185</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-1263</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-1353</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-770</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-780</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-836</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-901</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewsArticles-960</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 6279 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   01062017  01132017  01202017  01272017  0830  1  10  100  \\\n",
       "NewsArticles-1100         0         0         0         0     0  0   0    0   \n",
       "NewsArticles-1119         0         0         0         0     0  0   0    1   \n",
       "NewsArticles-1185         0         0         0         0     0  0   4    1   \n",
       "NewsArticles-1263         0         0         0         0     0  0   1    2   \n",
       "NewsArticles-1353         0         0         0         0     0  0   0    0   \n",
       "...                     ...       ...       ...       ...   ... ..  ..  ...   \n",
       "NewsArticles-770          0         0         0         0     0  2   1    0   \n",
       "NewsArticles-780          0         0         0         0     0  1   0    0   \n",
       "NewsArticles-836          0         0         0         0     0  2   0    0   \n",
       "NewsArticles-901          0         0         0         0     0  2   4    0   \n",
       "NewsArticles-960          0         0         0         0     0  0   0    0   \n",
       "\n",
       "                   1000  10000  ...  zapad  zeleny  zeping  zhang  zionist  \\\n",
       "NewsArticles-1100     0      0  ...      0       0       0      0        0   \n",
       "NewsArticles-1119     0      0  ...      0       0       0      0        0   \n",
       "NewsArticles-1185     4      0  ...      0       0       0      0        0   \n",
       "NewsArticles-1263     0      0  ...      0       0       0      0        0   \n",
       "NewsArticles-1353     0      0  ...      0       0       0      0        0   \n",
       "...                 ...    ...  ...    ...     ...     ...    ...      ...   \n",
       "NewsArticles-770      1      0  ...      0       0       0      0        0   \n",
       "NewsArticles-780      0      0  ...      0       0       0      0        0   \n",
       "NewsArticles-836      2      0  ...      0       0       0      0        0   \n",
       "NewsArticles-901      7      1  ...      0       0       0      0        0   \n",
       "NewsArticles-960      3      0  ...      0       0       0      0        0   \n",
       "\n",
       "                   zone  zor  zuma  �  �_�  \n",
       "NewsArticles-1100     0    0     0  0    0  \n",
       "NewsArticles-1119     0    0     0  0    0  \n",
       "NewsArticles-1185     1    0     0  0    0  \n",
       "NewsArticles-1263     0    0     0  0    0  \n",
       "NewsArticles-1353     0    0     0  0    0  \n",
       "...                 ...  ...   ... ..  ...  \n",
       "NewsArticles-770      2    0     0  0    0  \n",
       "NewsArticles-780      0    0     0  0    0  \n",
       "NewsArticles-836      0    0     0  0    0  \n",
       "NewsArticles-901      0    0     0  0    0  \n",
       "NewsArticles-960      0    0     0  0    0  \n",
       "\n",
       "[100 rows x 6279 columns]"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dtm(corpus_norm, as_table=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Serialization: Saving and loading `Corpus` objects\n",
    "\n",
    "The current state of a `Corpus` object can also be stored to a file on disk so that you (or someone else who has tmtoolkit installed) can later restore it using that file. The functions for that are [save_corpus_to_picklefile](api.rst#tmtoolkit.corpus.save_corpus_to_picklefile) and [load_corpus_from_picklefile](api.rst#tmtoolkit.corpus.load_corpus_from_picklefile).\n",
    "\n",
    "Let's store the current state of the `corpus_norm` instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.868341Z",
     "iopub.status.busy": "2023-05-03T13:22:57.868209Z",
     "iopub.status.idle": "2023-05-03T13:22:57.883864Z",
     "shell.execute_reply": "2023-05-03T13:22:57.883259Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (181 tokens): trump suggest obama town hall protest president_do...\n",
      "> NewsArticles-1407 (105 tokens): minister reiterate govt support finucane inquiry m...\n",
      "> NewsArticles-1100 (96 tokens): president trump say ask justice department investi...\n",
      "> NewsArticles-1119 (398 tokens): amazing moment history donald_trump press conferen...\n",
      "> NewsArticles-1472 (163 tokens): royal bank scotland see loss widen bail royal bank...\n",
      "> NewsArticles-1377 (397 tokens): turkey back rebel near control al bab turkey defen...\n",
      "> NewsArticles-1185 (573 tokens): weekfrance rock tension flare police officer alleg...\n",
      "> NewsArticles-1263 (202 tokens): russian doctor use mobile field hospital provide m...\n",
      "> NewsArticles-1353 (21 tokens): islamic state battle fierce gunfight outside mosul...\n",
      "> NewsArticles-1387 (250 tokens): protest anaheim policeman drag teen fire gun lapd ...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 27787 / vocabulary size: 6279\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import save_corpus_to_picklefile\n",
    "\n",
    "print_summary(corpus_norm)\n",
    "\n",
    "save_corpus_to_picklefile(corpus_norm, 'data/corpus_norm.pickle')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's change the object by retaining only documents that contain the token \"house\" (see the reduced number of documents):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.886147Z",
     "iopub.status.busy": "2023-05-03T13:22:57.885977Z",
     "iopub.status.idle": "2023-05-03T13:22:57.918769Z",
     "shell.execute_reply": "2023-05-03T13:22:57.918210Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 21 documents in English\n",
      "> NewsArticles-1610 (192 tokens): jewish community centre hit wave bomb threat anti ...\n",
      "> NewsArticles-2867 (85 tokens): person detain hop bike rack barrier white_house fe...\n",
      "> NewsArticles-1119 (398 tokens): amazing moment history donald_trump press conferen...\n",
      "> NewsArticles-2487 (462 tokens): dutch election high turnout key national vote cruc...\n",
      "> NewsArticles-1546 (118 tokens): kellyanne conway ultra casual oval office photo we...\n",
      "> NewsArticles-2431 (777 tokens): europe ride populist wave visual guide cnneurope p...\n",
      "> NewsArticles-2132 (201 tokens): trump health care big fat beautiful negotiation wa...\n",
      "> NewsArticles-2641 (579 tokens): muslim artist dreamy nude self portrait power self...\n",
      "> NewsArticles-1263 (202 tokens): russian doctor use mobile field hospital provide m...\n",
      "> NewsArticles-2301 (214 tokens): doj seek time trump wiretappe inquiry justice depa...\n",
      "(and 11 more documents)\n",
      "total number of tokens: 7578 / vocabulary size: 2677\n"
     ]
    }
   ],
   "source": [
    "filter_documents(corpus_norm, '*house*', match_type='glob')\n",
    "print_summary(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can restore the saved data using `load_corpus_from_picklefile`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-03T13:22:57.920943Z",
     "iopub.status.busy": "2023-05-03T13:22:57.920739Z",
     "iopub.status.idle": "2023-05-03T13:22:58.154964Z",
     "shell.execute_reply": "2023-05-03T13:22:58.154545Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus with 100 documents in English\n",
      "> NewsArticles-1515 (181 tokens): trump suggest obama town hall protest president_do...\n",
      "> NewsArticles-1407 (105 tokens): minister reiterate govt support finucane inquiry m...\n",
      "> NewsArticles-1100 (96 tokens): president trump say ask justice department investi...\n",
      "> NewsArticles-1119 (398 tokens): amazing moment history donald_trump press conferen...\n",
      "> NewsArticles-1472 (163 tokens): royal bank scotland see loss widen bail royal bank...\n",
      "> NewsArticles-1377 (397 tokens): turkey back rebel near control al bab turkey defen...\n",
      "> NewsArticles-1185 (573 tokens): weekfrance rock tension flare police officer alleg...\n",
      "> NewsArticles-1263 (202 tokens): russian doctor use mobile field hospital provide m...\n",
      "> NewsArticles-1353 (21 tokens): islamic state battle fierce gunfight outside mosul...\n",
      "> NewsArticles-1387 (250 tokens): protest anaheim policeman drag teen fire gun lapd ...\n",
      "(and 90 more documents)\n",
      "total number of tokens: 27787 / vocabulary size: 6279\n"
     ]
    }
   ],
   "source": [
    "from tmtoolkit.corpus import load_corpus_from_picklefile\n",
    "\n",
    "corpus_norm = load_corpus_from_picklefile('data/corpus_norm.pickle')\n",
    "print_summary(corpus_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that the full dataset with 100 documents was restored.\n",
    "\n",
    "This is very useful especially when you have a large amount of data and time intensive operations, e.g. token transformations or filtering. When you're finished running these operations, you can easily store the current state to disk and later retrieve it without the need to re-run these operations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final result after applying preprocessing steps and hence transforming the text data is often a document-term matrix (DTM). The [bow module](api.rst#tmtoolkit-bow) contains several functions to work with DTMs, e.g. apply transformations such as *tf-idf* or compute some important summary statistics. The [next chapter](bow.ipynb) will introduce some of these functions."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}