chapter_recurrent-neural-networks/language-models-and-dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Studio Lab](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github/d2l-ai/d2l-pytorch-sagemaker-studio-lab/blob/main/chapter_recurrent-neural-networks/language-models-and-dataset.ipynb)\n",
    "\n",
    "# 8.3 Language Models and the Dataset\n",
    "\n",
    "\n",
    "\n",
    "In [Section 8.2](../chapter_recurrent-neural-networks/text-preprocessing.ipynb), we see how to map text data into tokens, where these tokens can be viewed as a sequence of discrete observations, such as words or characters.\n",
    "Assume that the tokens in a text sequence of length $T$ are in turn $x_1, x_2, \\ldots, x_T$. \n",
    "Then, in the text sequence,\n",
    "$x_t$($1 \\leq t \\leq T$) can be considered as the observation or label at time step $t$. Given such a text sequence,\n",
    "the goal of a *language model* is to estimate the joint probability of the sequence\n",
    "\n",
    "$$P(x_1, x_2, \\ldots, x_T).$$<i style=\"float: right\">(8.3.1)</i><br/><br/>\n",
    "\n",
    "Language models are incredibly useful. For instance, an ideal language model would be able to generate natural text just on its own, simply by drawing one token at a time $x_t \\sim P(x_t \\mid x_{t-1}, \\ldots, x_1)$.\n",
    "Quite unlike the monkey using a typewriter, all text emerging from such a model would pass as natural language, e.g., English text. Furthermore, it would be sufficient for generating a meaningful dialog, simply by conditioning the text on previous dialog fragments.\n",
    "Clearly we are still very far from designing such a system, since it would need to *understand* the text rather than just generate grammatically sensible content.\n",
    "\n",
    "Nonetheless, language models are of great service even in their limited form.\n",
    "For instance, the phrases \"to recognize speech\" and \"to wreck a nice beach\" sound very similar.\n",
    "This can cause ambiguity in speech recognition,\n",
    "which is easily resolved through a language model that rejects the second translation as outlandish.\n",
    "Likewise, in a document summarization algorithm\n",
    "it is worthwhile knowing that \"dog bites man\" is much more frequent than \"man bites dog\", or that \"I want to eat grandma\" is a rather disturbing statement, whereas \"I want to eat, grandma\" is much more benign.\n",
    "\n",
    "\n",
    "## 8.3.1 Learning a Language Model\n",
    "\n",
    "The obvious question is how we should model a document, or even a sequence of tokens. \n",
    "Suppose that we tokenize text data at the word level.\n",
    "We can take recourse to the analysis we applied to sequence models in [Section 8.1](../chapter_recurrent-neural-networks/sequence.ipynb).\n",
    "Let us start by applying basic probability rules:\n",
    "\n",
    "$$P(x_1, x_2, \\ldots, x_T) = \\prod_{t=1}^T P(x_t  \\mid  x_1, \\ldots, x_{t-1}).$$<i style=\"float: right\">(8.3.2)</i><br/><br/>\n",
    "\n",
    "For example, \n",
    "the probability of a text sequence containing four words would be given as:\n",
    "\n",
    "$$P(\\text{deep}, \\text{learning}, \\text{is}, \\text{fun}) =  P(\\text{deep}) P(\\text{learning}  \\mid  \\text{deep}) P(\\text{is}  \\mid  \\text{deep}, \\text{learning}) P(\\text{fun}  \\mid  \\text{deep}, \\text{learning}, \\text{is}).$$<i style=\"float: right\">(8.3.3)</i><br/><br/>\n",
    "\n",
    "In order to compute the language model, we need to calculate the\n",
    "probability of words and the conditional probability of a word given\n",
    "the previous few words.\n",
    "Such probabilities are essentially\n",
    "language model parameters.\n",
    "\n",
    "Here, we\n",
    "assume that the training dataset is a large text corpus, such as all\n",
    "Wikipedia entries, [Project Gutenberg](https://en.wikipedia.org/wiki/Project_Gutenberg),\n",
    "and all text posted on the\n",
    "Web.\n",
    "The probability of words can be calculated from the relative word\n",
    "frequency of a given word in the training dataset.\n",
    "For example, the estimate $\\hat{P}(\\text{deep})$ can be calculated as the\n",
    "probability of any sentence starting with the word \"deep\". A\n",
    "slightly less accurate approach would be to count all occurrences of\n",
    "the word \"deep\" and divide it by the total number of words in\n",
    "the corpus.\n",
    "This works fairly well, particularly for frequent\n",
    "words. Moving on, we could attempt to estimate\n",
    "\n",
    "$$\\hat{P}(\\text{learning} \\mid \\text{deep}) = \\frac{n(\\text{deep, learning})}{n(\\text{deep})},$$<i style=\"float: right\">(8.3.4)</i><br/><br/>\n",
    "\n",
    "where $n(x)$ and $n(x, x')$ are the number of occurrences of singletons\n",
    "and consecutive word pairs, respectively.\n",
    "Unfortunately, estimating the\n",
    "probability of a word pair is somewhat more difficult, since the\n",
    "occurrences of \"deep learning\" are a lot less frequent. In\n",
    "particular, for some unusual word combinations it may be tricky to\n",
    "find enough occurrences to get accurate estimates.\n",
    "Things take a turn for the worse for three-word combinations and beyond.\n",
    "There will be many plausible three-word combinations that we likely will not see in our dataset.\n",
    "Unless we provide some solution to assign such word combinations nonzero count, we will not be able to use them in a language model. If the dataset is small or if the words are very rare, we might not find even a single one of them.\n",
    "\n",
    "A common strategy is to perform some form of *Laplace smoothing*.\n",
    "The solution is to\n",
    "add a small constant to all counts. \n",
    "Denote by $n$ the total number of words in\n",
    "the training set\n",
    "and $m$ the number of unique words.\n",
    "This solution helps with singletons, e.g., via\n",
    "\n",
    "$$\\begin{aligned}\n",
    "\t\\hat{P}(x) & = \\frac{n(x) + \\epsilon_1/m}{n + \\epsilon_1}, \\\\\n",
    "\t\\hat{P}(x' \\mid x) & = \\frac{n(x, x') + \\epsilon_2 \\hat{P}(x')}{n(x) + \\epsilon_2}, \\\\\n",
    "\t\\hat{P}(x'' \\mid x,x') & = \\frac{n(x, x',x'') + \\epsilon_3 \\hat{P}(x'')}{n(x, x') + \\epsilon_3}.\n",
    "\\end{aligned}$$\n",
    "\n",
    "Here $\\epsilon_1,\\epsilon_2$, and $\\epsilon_3$ are hyperparameters.\n",
    "Take $\\epsilon_1$ as an example:\n",
    "when $\\epsilon_1 = 0$, no smoothing is applied;\n",
    "when $\\epsilon_1$ approaches positive infinity,\n",
    "$\\hat{P}(x)$ approaches the uniform probability $1/m$. \n",
    "The above is a rather primitive variant of what\n",
    "other techniques can accomplish [Wood et al., 2011](../chapter_references/zreferences.ipynb#Wood.Gasthaus.Archambeau.ea.2011).\n",
    "\n",
    "\n",
    "Unfortunately, models like this get unwieldy rather quickly\n",
    "for the following reasons. First, we need to store all counts.\n",
    "Second, this entirely ignores the meaning of the words. For\n",
    "instance, \"cat\" and \"feline\" should occur in related contexts.\n",
    "It is quite difficult to adjust such models to additional contexts,\n",
    "whereas, deep learning based language models are well suited to\n",
    "take this into account.\n",
    "Last, long word\n",
    "sequences are almost certain to be novel, hence a model that simply\n",
    "counts the frequency of previously seen word sequences is bound to perform poorly there.\n",
    "\n",
    "## 8.3.2 Markov Models and $n$-grams\n",
    "\n",
    "Before we discuss solutions involving deep learning, we need some more terminology and concepts. Recall our discussion of Markov Models in [Section 8.1](../chapter_recurrent-neural-networks/sequence.ipynb).\n",
    "Let us apply this to language modeling. A distribution over sequences satisfies the Markov property of first order if $P(x_{t+1} \\mid x_t, \\ldots, x_1) = P(x_{t+1} \\mid x_t)$. Higher orders correspond to longer dependencies. This leads to a number of approximations that we could apply to model a sequence:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "P(x_1, x_2, x_3, x_4) &=  P(x_1) P(x_2) P(x_3) P(x_4),\\\\\n",
    "P(x_1, x_2, x_3, x_4) &=  P(x_1) P(x_2  \\mid  x_1) P(x_3  \\mid  x_2) P(x_4  \\mid  x_3),\\\\\n",
    "P(x_1, x_2, x_3, x_4) &=  P(x_1) P(x_2  \\mid  x_1) P(x_3  \\mid  x_1, x_2) P(x_4  \\mid  x_2, x_3).\n",
    "\\end{aligned}\n",
    "$$\n",
    "<i style=\"float: right\">(8.3.5)</i><br/><br/>\n",
    "\n",
    "The probability formulae that involve one, two, and three variables are typically referred to as *unigram*, *bigram*, and *trigram* models, respectively. In the following, we will learn how to design better models.\n",
    "\n",
    "## 8.3.3 Natural Language Statistics\n",
    "\n",
    "Let us see how this works on real data.\n",
    "We construct a vocabulary based on the time machine dataset as introduced in [Section 8.2](../chapter_recurrent-neural-networks/text-preprocessing.ipynb) \n",
    "and print the top 10 most frequent words.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "origin_pos": 2,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "import random\n",
    "import torch\n",
    "from d2l import torch as d2l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "origin_pos": 4,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('the', 2261),\n",
       " ('i', 1267),\n",
       " ('and', 1245),\n",
       " ('of', 1155),\n",
       " ('a', 816),\n",
       " ('to', 695),\n",
       " ('was', 552),\n",
       " ('in', 541),\n",
       " ('that', 443),\n",
       " ('my', 440)]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens = d2l.tokenize(d2l.read_time_machine())\n",
    "# Since each text line is not necessarily a sentence or a paragraph, we\n",
    "# concatenate all text lines\n",
    "corpus = [token for line in tokens for token in line]\n",
    "vocab = d2l.Vocab(corpus)\n",
    "vocab.token_freqs[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the most popular words are actually quite boring to look at.\n",
    "They are often referred to as *stop words* and thus filtered out.\n",
    "Nonetheless, they still carry meaning and we will still use them.\n",
    "Besides, it is quite clear that the word frequency decays rather rapidly. The $10^{\\mathrm{th}}$ most frequent word is less than $1/5$ as common as the most popular one. To get a better idea, we plot the figure of the word frequency.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "origin_pos": 6,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"180.65625pt\" version=\"1.1\" viewBox=\"0 0 247.978125 180.65625\" width=\"247.978125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2021-07-24T08:44:32.989151</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.3.3, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 180.65625 \n",
       "L 247.978125 180.65625 \n",
       "L 247.978125 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 45.478125 143.1 \n",
       "L 240.778125 143.1 \n",
       "L 240.778125 7.2 \n",
       "L 45.478125 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path clip-path=\"url(#pcef42d838c)\" d=\"M 54.355398 143.1 \n",
       "L 54.355398 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"mc87c6f86ed\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.355398\" xlink:href=\"#mc87c6f86ed\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- $\\mathdefault{10^{0}}$ -->\n",
       "      <g transform=\"translate(45.555398 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-49\"/>\n",
       "        <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "z\n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-48\"/>\n",
       "       </defs>\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path clip-path=\"url(#pcef42d838c)\" d=\"M 102.85613 143.1 \n",
       "L 102.85613 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"102.85613\" xlink:href=\"#mc87c6f86ed\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- $\\mathdefault{10^{1}}$ -->\n",
       "      <g transform=\"translate(94.05613 157.698438)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.684375)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.684375)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path clip-path=\"url(#pcef42d838c)\" d=\"M 151.356861 143.1 \n",
       "L 151.356861 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"151.356861\" xlink:href=\"#mc87c6f86ed\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- $\\mathdefault{10^{2}}$ -->\n",
       "      <g transform=\"translate(142.556861 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-50\"/>\n",
       "       </defs>\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-50\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path clip-path=\"url(#pcef42d838c)\" d=\"M 199.857593 143.1 \n",
       "L 199.857593 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"199.857593\" xlink:href=\"#mc87c6f86ed\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- $\\mathdefault{10^{3}}$ -->\n",
       "      <g transform=\"translate(191.057593 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "z\n",
       "\" id=\"DejaVuSans-51\"/>\n",
       "       </defs>\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-51\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 2 \n",
       "\" id=\"m510fac4e9a\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"46.842539\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"49.655191\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_7\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"52.136126\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_8\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"68.955573\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_9\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"77.496128\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_10\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"83.555748\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_11\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"88.255954\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_12\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"92.096303\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_13\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"95.343271\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_14\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"98.155923\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_15\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"100.636858\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_16\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"117.456305\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_17\">\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"125.99686\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_18\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"132.05648\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_19\">\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"136.756686\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_20\">\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"140.597035\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_21\">\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"143.844003\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_22\">\n",
       "     <g id=\"line2d_26\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"146.656655\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_23\">\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"149.13759\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_24\">\n",
       "     <g id=\"line2d_28\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"165.957036\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_25\">\n",
       "     <g id=\"line2d_29\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"174.497591\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_26\">\n",
       "     <g id=\"line2d_30\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"180.557211\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_27\">\n",
       "     <g id=\"line2d_31\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"185.257418\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_28\">\n",
       "     <g id=\"line2d_32\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"189.097766\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_29\">\n",
       "     <g id=\"line2d_33\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"192.344735\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_30\">\n",
       "     <g id=\"line2d_34\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"195.157387\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_31\">\n",
       "     <g id=\"line2d_35\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"197.638321\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_32\">\n",
       "     <g id=\"line2d_36\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"214.457768\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_33\">\n",
       "     <g id=\"line2d_37\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"222.998323\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_34\">\n",
       "     <g id=\"line2d_38\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"229.057943\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_35\">\n",
       "     <g id=\"line2d_39\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"233.75815\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_36\">\n",
       "     <g id=\"line2d_40\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"237.598498\" xlink:href=\"#m510fac4e9a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_5\">\n",
       "     <!-- token: x -->\n",
       "     <g transform=\"translate(122.916406 171.376563)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 18.3125 70.21875 \n",
       "L 18.3125 54.6875 \n",
       "L 36.8125 54.6875 \n",
       "L 36.8125 47.703125 \n",
       "L 18.3125 47.703125 \n",
       "L 18.3125 18.015625 \n",
       "Q 18.3125 11.328125 20.140625 9.421875 \n",
       "Q 21.96875 7.515625 27.59375 7.515625 \n",
       "L 36.8125 7.515625 \n",
       "L 36.8125 0 \n",
       "L 27.59375 0 \n",
       "Q 17.1875 0 13.234375 3.875 \n",
       "Q 9.28125 7.765625 9.28125 18.015625 \n",
       "L 9.28125 47.703125 \n",
       "L 2.6875 47.703125 \n",
       "L 2.6875 54.6875 \n",
       "L 9.28125 54.6875 \n",
       "L 9.28125 70.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-116\"/>\n",
       "       <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "z\n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-111\"/>\n",
       "       <path d=\"M 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 31.109375 \n",
       "L 44.921875 54.6875 \n",
       "L 56.390625 54.6875 \n",
       "L 27.390625 29.109375 \n",
       "L 57.625 0 \n",
       "L 45.90625 0 \n",
       "L 18.109375 26.703125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-107\"/>\n",
       "       <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "z\n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"DejaVuSans-101\"/>\n",
       "       <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-110\"/>\n",
       "       <path d=\"M 11.71875 12.40625 \n",
       "L 22.015625 12.40625 \n",
       "L 22.015625 0 \n",
       "L 11.71875 0 \n",
       "z\n",
       "M 11.71875 51.703125 \n",
       "L 22.015625 51.703125 \n",
       "L 22.015625 39.3125 \n",
       "L 11.71875 39.3125 \n",
       "z\n",
       "\" id=\"DejaVuSans-58\"/>\n",
       "       <path id=\"DejaVuSans-32\"/>\n",
       "       <path d=\"M 54.890625 54.6875 \n",
       "L 35.109375 28.078125 \n",
       "L 55.90625 0 \n",
       "L 45.3125 0 \n",
       "L 29.390625 21.484375 \n",
       "L 13.484375 0 \n",
       "L 2.875 0 \n",
       "L 24.125 28.609375 \n",
       "L 4.6875 54.6875 \n",
       "L 15.28125 54.6875 \n",
       "L 29.78125 35.203125 \n",
       "L 44.28125 54.6875 \n",
       "z\n",
       "\" id=\"DejaVuSans-120\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-116\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "      <use x=\"100.390625\" xlink:href=\"#DejaVuSans-107\"/>\n",
       "      <use x=\"154.675781\" xlink:href=\"#DejaVuSans-101\"/>\n",
       "      <use x=\"216.199219\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"279.578125\" xlink:href=\"#DejaVuSans-58\"/>\n",
       "      <use x=\"313.269531\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      <use x=\"345.056641\" xlink:href=\"#DejaVuSans-120\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_41\">\n",
       "      <path clip-path=\"url(#pcef42d838c)\" d=\"M 45.478125 136.922727 \n",
       "L 240.778125 136.922727 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_42\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"m8c8f43802a\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#m8c8f43802a\" y=\"136.922727\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- $\\mathdefault{10^{0}}$ -->\n",
       "      <g transform=\"translate(20.878125 140.721946)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_43\">\n",
       "      <path clip-path=\"url(#pcef42d838c)\" d=\"M 45.478125 100.09077 \n",
       "L 240.778125 100.09077 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_44\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#m8c8f43802a\" y=\"100.09077\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- $\\mathdefault{10^{1}}$ -->\n",
       "      <g transform=\"translate(20.878125 103.889989)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.684375)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.684375)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_45\">\n",
       "      <path clip-path=\"url(#pcef42d838c)\" d=\"M 45.478125 63.258813 \n",
       "L 240.778125 63.258813 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_46\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#m8c8f43802a\" y=\"63.258813\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- $\\mathdefault{10^{2}}$ -->\n",
       "      <g transform=\"translate(20.878125 67.058032)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-50\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_47\">\n",
       "      <path clip-path=\"url(#pcef42d838c)\" d=\"M 45.478125 26.426856 \n",
       "L 240.778125 26.426856 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_48\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#m8c8f43802a\" y=\"26.426856\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- $\\mathdefault{10^{3}}$ -->\n",
       "      <g transform=\"translate(20.878125 30.226075)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-51\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_49\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -2 0 \n",
       "\" id=\"m736532f44a\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"142.62807\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_50\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"140.492113\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_51\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"138.608065\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_8\">\n",
       "     <g id=\"line2d_52\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"125.835203\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_9\">\n",
       "     <g id=\"line2d_53\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"119.349418\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_10\">\n",
       "     <g id=\"line2d_54\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"114.747679\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_11\">\n",
       "     <g id=\"line2d_55\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"111.178294\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_12\">\n",
       "     <g id=\"line2d_56\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"108.261894\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_13\">\n",
       "     <g id=\"line2d_57\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"105.796112\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_14\">\n",
       "     <g id=\"line2d_58\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"103.660156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_15\">\n",
       "     <g id=\"line2d_59\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"101.776108\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_16\">\n",
       "     <g id=\"line2d_60\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"89.003246\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_17\">\n",
       "     <g id=\"line2d_61\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"82.517461\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_18\">\n",
       "     <g id=\"line2d_62\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"77.915722\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_19\">\n",
       "     <g id=\"line2d_63\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"74.346337\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_20\">\n",
       "     <g id=\"line2d_64\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"71.429937\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_21\">\n",
       "     <g id=\"line2d_65\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"68.964155\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_22\">\n",
       "     <g id=\"line2d_66\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"66.828198\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_23\">\n",
       "     <g id=\"line2d_67\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"64.944151\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_24\">\n",
       "     <g id=\"line2d_68\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"52.171289\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_25\">\n",
       "     <g id=\"line2d_69\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"45.685503\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_26\">\n",
       "     <g id=\"line2d_70\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"41.083765\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_27\">\n",
       "     <g id=\"line2d_71\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"37.51438\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_28\">\n",
       "     <g id=\"line2d_72\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"34.597979\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_29\">\n",
       "     <g id=\"line2d_73\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"32.132198\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_30\">\n",
       "     <g id=\"line2d_74\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"29.996241\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_31\">\n",
       "     <g id=\"line2d_75\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"28.112194\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_32\">\n",
       "     <g id=\"line2d_76\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"15.339332\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_33\">\n",
       "     <g id=\"line2d_77\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m736532f44a\" y=\"8.853546\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- frequency: n(x) -->\n",
       "     <g transform=\"translate(14.798438 113.167187)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 37.109375 75.984375 \n",
       "L 37.109375 68.5 \n",
       "L 28.515625 68.5 \n",
       "Q 23.6875 68.5 21.796875 66.546875 \n",
       "Q 19.921875 64.59375 19.921875 59.515625 \n",
       "L 19.921875 54.6875 \n",
       "L 34.71875 54.6875 \n",
       "L 34.71875 47.703125 \n",
       "L 19.921875 47.703125 \n",
       "L 19.921875 0 \n",
       "L 10.890625 0 \n",
       "L 10.890625 47.703125 \n",
       "L 2.296875 47.703125 \n",
       "L 2.296875 54.6875 \n",
       "L 10.890625 54.6875 \n",
       "L 10.890625 58.5 \n",
       "Q 10.890625 67.625 15.140625 71.796875 \n",
       "Q 19.390625 75.984375 28.609375 75.984375 \n",
       "z\n",
       "\" id=\"DejaVuSans-102\"/>\n",
       "       <path d=\"M 41.109375 46.296875 \n",
       "Q 39.59375 47.171875 37.8125 47.578125 \n",
       "Q 36.03125 48 33.890625 48 \n",
       "Q 26.265625 48 22.1875 43.046875 \n",
       "Q 18.109375 38.09375 18.109375 28.8125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 20.953125 51.171875 25.484375 53.578125 \n",
       "Q 30.03125 56 36.53125 56 \n",
       "Q 37.453125 56 38.578125 55.875 \n",
       "Q 39.703125 55.765625 41.0625 55.515625 \n",
       "z\n",
       "\" id=\"DejaVuSans-114\"/>\n",
       "       <path d=\"M 14.796875 27.296875 \n",
       "Q 14.796875 17.390625 18.875 11.75 \n",
       "Q 22.953125 6.109375 30.078125 6.109375 \n",
       "Q 37.203125 6.109375 41.296875 11.75 \n",
       "Q 45.40625 17.390625 45.40625 27.296875 \n",
       "Q 45.40625 37.203125 41.296875 42.84375 \n",
       "Q 37.203125 48.484375 30.078125 48.484375 \n",
       "Q 22.953125 48.484375 18.875 42.84375 \n",
       "Q 14.796875 37.203125 14.796875 27.296875 \n",
       "z\n",
       "M 45.40625 8.203125 \n",
       "Q 42.578125 3.328125 38.25 0.953125 \n",
       "Q 33.9375 -1.421875 27.875 -1.421875 \n",
       "Q 17.96875 -1.421875 11.734375 6.484375 \n",
       "Q 5.515625 14.40625 5.515625 27.296875 \n",
       "Q 5.515625 40.1875 11.734375 48.09375 \n",
       "Q 17.96875 56 27.875 56 \n",
       "Q 33.9375 56 38.25 53.625 \n",
       "Q 42.578125 51.265625 45.40625 46.390625 \n",
       "L 45.40625 54.6875 \n",
       "L 54.390625 54.6875 \n",
       "L 54.390625 -20.796875 \n",
       "L 45.40625 -20.796875 \n",
       "z\n",
       "\" id=\"DejaVuSans-113\"/>\n",
       "       <path d=\"M 8.5 21.578125 \n",
       "L 8.5 54.6875 \n",
       "L 17.484375 54.6875 \n",
       "L 17.484375 21.921875 \n",
       "Q 17.484375 14.15625 20.5 10.265625 \n",
       "Q 23.53125 6.390625 29.59375 6.390625 \n",
       "Q 36.859375 6.390625 41.078125 11.03125 \n",
       "Q 45.3125 15.671875 45.3125 23.6875 \n",
       "L 45.3125 54.6875 \n",
       "L 54.296875 54.6875 \n",
       "L 54.296875 0 \n",
       "L 45.3125 0 \n",
       "L 45.3125 8.40625 \n",
       "Q 42.046875 3.421875 37.71875 1 \n",
       "Q 33.40625 -1.421875 27.6875 -1.421875 \n",
       "Q 18.265625 -1.421875 13.375 4.4375 \n",
       "Q 8.5 10.296875 8.5 21.578125 \n",
       "z\n",
       "M 31.109375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-117\"/>\n",
       "       <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-99\"/>\n",
       "       <path d=\"M 32.171875 -5.078125 \n",
       "Q 28.375 -14.84375 24.75 -17.8125 \n",
       "Q 21.140625 -20.796875 15.09375 -20.796875 \n",
       "L 7.90625 -20.796875 \n",
       "L 7.90625 -13.28125 \n",
       "L 13.1875 -13.28125 \n",
       "Q 16.890625 -13.28125 18.9375 -11.515625 \n",
       "Q 21 -9.765625 23.484375 -3.21875 \n",
       "L 25.09375 0.875 \n",
       "L 2.984375 54.6875 \n",
       "L 12.5 54.6875 \n",
       "L 29.59375 11.921875 \n",
       "L 46.6875 54.6875 \n",
       "L 56.203125 54.6875 \n",
       "z\n",
       "\" id=\"DejaVuSans-121\"/>\n",
       "       <path d=\"M 31 75.875 \n",
       "Q 24.46875 64.65625 21.28125 53.65625 \n",
       "Q 18.109375 42.671875 18.109375 31.390625 \n",
       "Q 18.109375 20.125 21.3125 9.0625 \n",
       "Q 24.515625 -2 31 -13.1875 \n",
       "L 23.1875 -13.1875 \n",
       "Q 15.875 -1.703125 12.234375 9.375 \n",
       "Q 8.59375 20.453125 8.59375 31.390625 \n",
       "Q 8.59375 42.28125 12.203125 53.3125 \n",
       "Q 15.828125 64.359375 23.1875 75.875 \n",
       "z\n",
       "\" id=\"DejaVuSans-40\"/>\n",
       "       <path d=\"M 8.015625 75.875 \n",
       "L 15.828125 75.875 \n",
       "Q 23.140625 64.359375 26.78125 53.3125 \n",
       "Q 30.421875 42.28125 30.421875 31.390625 \n",
       "Q 30.421875 20.453125 26.78125 9.375 \n",
       "Q 23.140625 -1.703125 15.828125 -13.1875 \n",
       "L 8.015625 -13.1875 \n",
       "Q 14.5 -2 17.703125 9.0625 \n",
       "Q 20.90625 20.125 20.90625 31.390625 \n",
       "Q 20.90625 42.671875 17.703125 53.65625 \n",
       "Q 14.5 64.65625 8.015625 75.875 \n",
       "z\n",
       "\" id=\"DejaVuSans-41\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-102\"/>\n",
       "      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"74.068359\" xlink:href=\"#DejaVuSans-101\"/>\n",
       "      <use x=\"135.591797\" xlink:href=\"#DejaVuSans-113\"/>\n",
       "      <use x=\"199.068359\" xlink:href=\"#DejaVuSans-117\"/>\n",
       "      <use x=\"262.447266\" xlink:href=\"#DejaVuSans-101\"/>\n",
       "      <use x=\"323.970703\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"387.349609\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"442.330078\" xlink:href=\"#DejaVuSans-121\"/>\n",
       "      <use x=\"494.259766\" xlink:href=\"#DejaVuSans-58\"/>\n",
       "      <use x=\"527.951172\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      <use x=\"559.738281\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"623.117188\" xlink:href=\"#DejaVuSans-40\"/>\n",
       "      <use x=\"662.130859\" xlink:href=\"#DejaVuSans-120\"/>\n",
       "      <use x=\"721.310547\" xlink:href=\"#DejaVuSans-41\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_78\">\n",
       "    <path clip-path=\"url(#pcef42d838c)\" d=\"M -1 22.630819 \n",
       "L 54.355398 22.641392 \n",
       "L 68.955573 22.921582 \n",
       "L 77.496128 24.121839 \n",
       "L 83.555748 29.67948 \n",
       "L 88.255954 32.246865 \n",
       "L 92.096303 35.931745 \n",
       "L 95.343271 36.253723 \n",
       "L 98.155923 39.450499 \n",
       "L 102.85613 39.668628 \n",
       "L 104.863705 43.037948 \n",
       "L 106.696478 46.732081 \n",
       "L 108.382465 47.370841 \n",
       "L 111.396684 50.574172 \n",
       "L 112.756098 50.940227 \n",
       "L 114.033071 51.854528 \n",
       "L 115.237033 52.171289 \n",
       "L 116.375883 55.941884 \n",
       "L 117.456305 56.561158 \n",
       "L 118.484001 58.223127 \n",
       "L 119.46388 58.223127 \n",
       "L 120.400194 58.577295 \n",
       "L 121.296653 59.185578 \n",
       "L 122.156511 59.435519 \n",
       "L 122.982641 59.947432 \n",
       "L 123.777588 60.078011 \n",
       "L 125.282771 60.078011 \n",
       "L 126.687531 60.611258 \n",
       "L 127.356273 61.162896 \n",
       "L 128.004435 61.162896 \n",
       "L 128.633246 61.30383 \n",
       "L 129.243828 61.30383 \n",
       "L 129.837208 61.446017 \n",
       "L 130.414329 62.326748 \n",
       "L 130.976058 62.47837 \n",
       "L 131.523195 62.785993 \n",
       "L 132.05648 64.248567 \n",
       "L 133.084176 64.592578 \n",
       "L 133.579813 64.592578 \n",
       "L 134.064055 64.767399 \n",
       "L 134.537414 65.122878 \n",
       "L 136.331144 65.858452 \n",
       "L 137.173801 65.858452 \n",
       "L 137.582816 66.047755 \n",
       "L 137.984039 66.047755 \n",
       "L 138.377763 66.433217 \n",
       "L 139.143796 67.439582 \n",
       "L 139.516613 67.860551 \n",
       "L 139.882946 68.075265 \n",
       "L 140.597035 68.075265 \n",
       "L 141.956448 68.964155 \n",
       "L 142.60461 68.964155 \n",
       "L 142.921362 69.194316 \n",
       "L 143.233421 69.905363 \n",
       "L 143.540924 70.905433 \n",
       "L 144.437383 71.698782 \n",
       "L 144.72792 71.698782 \n",
       "L 145.014504 72.25042 \n",
       "L 145.576233 72.25042 \n",
       "L 145.851578 72.821763 \n",
       "L 146.123371 73.115275 \n",
       "L 146.3917 73.115275 \n",
       "L 147.684351 74.669498 \n",
       "L 148.90224 74.669498 \n",
       "L 149.13759 74.999322 \n",
       "L 149.370339 74.999322 \n",
       "L 149.600544 75.336091 \n",
       "L 149.828261 75.336091 \n",
       "L 150.053543 75.680102 \n",
       "L 150.715281 75.680102 \n",
       "L 150.93132 76.031675 \n",
       "L 151.145165 76.031675 \n",
       "L 151.566451 76.758887 \n",
       "L 151.979476 76.758887 \n",
       "L 152.182991 77.135279 \n",
       "L 152.384558 77.135279 \n",
       "L 152.781996 77.915722 \n",
       "L 153.364437 77.915722 \n",
       "L 153.743971 78.736206 \n",
       "L 153.931205 78.736206 \n",
       "L 154.116788 79.162789 \n",
       "L 154.843191 79.162789 \n",
       "L 155.020944 79.60106 \n",
       "L 155.887881 79.60106 \n",
       "L 156.057068 80.051679 \n",
       "L 156.720543 80.051679 \n",
       "L 156.883197 80.515361 \n",
       "L 157.678144 80.515361 \n",
       "L 157.833596 80.992887 \n",
       "L 158.141099 80.992887 \n",
       "L 158.293184 81.485108 \n",
       "L 158.890773 81.485108 \n",
       "L 159.037558 81.992957 \n",
       "L 159.328095 81.992957 \n",
       "L 159.471874 82.517461 \n",
       "L 159.756522 82.517461 \n",
       "L 159.897416 83.059747 \n",
       "L 160.037374 83.059747 \n",
       "L 160.176408 83.621065 \n",
       "L 160.858138 83.621065 \n",
       "L 160.991875 84.202798 \n",
       "L 161.518493 84.202798 \n",
       "L 161.648115 84.80649 \n",
       "L 162.409533 84.80649 \n",
       "L 162.533803 85.433861 \n",
       "L 162.902271 85.433861 \n",
       "L 163.023676 86.086846 \n",
       "L 163.502415 86.086846 \n",
       "L 163.620418 86.767626 \n",
       "L 164.200719 86.767626 \n",
       "L 164.314886 87.478673 \n",
       "L 165.423752 87.478673 \n",
       "L 165.638689 88.222803 \n",
       "L 166.374151 88.222803 \n",
       "L 166.579651 89.003246 \n",
       "L 167.675405 89.003246 \n",
       "L 167.86865 89.823729 \n",
       "L 168.716963 89.823729 \n",
       "L 168.900926 90.688584 \n",
       "L 169.173892 90.688584 \n",
       "L 169.353924 91.602885 \n",
       "L 169.972187 91.602885 \n",
       "L 170.145551 92.572632 \n",
       "L 171.483372 92.572632 \n",
       "L 171.64478 93.604984 \n",
       "L 172.433771 93.604984 \n",
       "L 172.588084 94.708589 \n",
       "L 173.490948 94.708589 \n",
       "L 173.637733 95.894013 \n",
       "L 175.256101 95.894013 \n",
       "L 175.391124 97.17437 \n",
       "L 176.37712 97.17437 \n",
       "L 176.505167 98.566197 \n",
       "L 178.33794 98.566197 \n",
       "L 178.454636 100.09077 \n",
       "L 180.451629 100.09077 \n",
       "L 180.609805 101.776108 \n",
       "L 182.42068 101.776108 \n",
       "L 182.564787 103.660156 \n",
       "L 185.257418 103.660156 \n",
       "L 185.383422 105.796112 \n",
       "L 188.528445 105.796112 \n",
       "L 188.672225 108.261894 \n",
       "L 191.67211 108.261894 \n",
       "L 191.796014 111.178294 \n",
       "L 195.754423 111.178294 \n",
       "L 195.882004 114.747679 \n",
       "L 200.724191 114.747679 \n",
       "L 200.84513 119.349418 \n",
       "L 207.214001 119.349418 \n",
       "L 207.332503 125.835203 \n",
       "L 216.282641 125.835203 \n",
       "L 216.398216 136.922727 \n",
       "L 231.900852 136.922727 \n",
       "L 231.900852 136.922727 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 45.478125 143.1 \n",
       "L 45.478125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 240.778125 143.1 \n",
       "L 240.778125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 45.478125 143.1 \n",
       "L 240.778125 143.1 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 45.478125 7.2 \n",
       "L 240.778125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pcef42d838c\">\n",
       "   <rect height=\"135.9\" width=\"195.3\" x=\"45.478125\" y=\"7.2\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 252x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "freqs = [freq for token, freq in vocab.token_freqs]\n",
    "d2l.plot(freqs, xlabel='token: x', ylabel='frequency: n(x)', xscale='log',\n",
    "         yscale='log')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are on to something quite fundamental here: the word frequency decays rapidly in a well-defined way.\n",
    "After dealing with the first few words as exceptions, all the remaining words roughly follow a straight line on a log-log plot. This means that words satisfy *Zipf's law*,\n",
    "which states that the frequency $n_i$ of the $i^\\mathrm{th}$ most frequent word\n",
    "is:\n",
    "\n",
    "$$n_i \\propto \\frac{1}{i^\\alpha},$$\n",
    "<i id=\"eq8.3.6\" style=\"float: right\">(8.3.6)</i><br/><br/>\n",
    "\n",
    "which is equivalent to\n",
    "\n",
    "$$\\log n_i = -\\alpha \\log i + c,$$<i style=\"float: right\">(8.3.7)</i><br/><br/>\n",
    "\n",
    "where $\\alpha$ is the exponent that characterizes the distribution and $c$ is a constant.\n",
    "This should already give us pause if we want to model words by counting statistics and smoothing.\n",
    "After all, we will significantly overestimate the frequency of the tail, also known as the infrequent words. But what about the other word combinations, such as bigrams, trigrams, and beyond?\n",
    "Let us see whether the bigram frequency behaves in the same manner as the unigram frequency.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "origin_pos": 8,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(('of', 'the'), 309),\n",
       " (('in', 'the'), 169),\n",
       " (('i', 'had'), 130),\n",
       " (('i', 'was'), 112),\n",
       " (('and', 'the'), 109),\n",
       " (('the', 'time'), 102),\n",
       " (('it', 'was'), 99),\n",
       " (('to', 'the'), 85),\n",
       " (('as', 'i'), 78),\n",
       " (('of', 'a'), 73)]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bigram_tokens = [pair for pair in zip(corpus[:-1], corpus[1:])]\n",
    "bigram_vocab = d2l.Vocab(bigram_tokens)\n",
    "bigram_vocab.token_freqs[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One thing is notable here. Out of the ten most frequent word pairs, nine are composed of both stop words and only one is relevant to the actual book---\"the time\". Furthermore, let us see whether the trigram frequency behaves in the same manner.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "origin_pos": 10,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(('the', 'time', 'traveller'), 59),\n",
       " (('the', 'time', 'machine'), 30),\n",
       " (('the', 'medical', 'man'), 24),\n",
       " (('it', 'seemed', 'to'), 16),\n",
       " (('it', 'was', 'a'), 15),\n",
       " (('here', 'and', 'there'), 15),\n",
       " (('seemed', 'to', 'me'), 14),\n",
       " (('i', 'did', 'not'), 14),\n",
       " (('i', 'saw', 'the'), 13),\n",
       " (('i', 'began', 'to'), 13)]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trigram_tokens = [\n",
    "    triple for triple in zip(corpus[:-2], corpus[1:-1], corpus[2:])]\n",
    "trigram_vocab = d2l.Vocab(trigram_tokens)\n",
    "trigram_vocab.token_freqs[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Last, let us visualize the token frequency among these three models: unigrams, bigrams, and trigrams.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "origin_pos": 12,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"180.65625pt\" version=\"1.1\" viewBox=\"0 0 247.978125 180.65625\" width=\"247.978125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2021-07-24T08:44:33.926173</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.3.3, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 180.65625 \n",
       "L 247.978125 180.65625 \n",
       "L 247.978125 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 45.478125 143.1 \n",
       "L 240.778125 143.1 \n",
       "L 240.778125 7.2 \n",
       "L 45.478125 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 54.355398 143.1 \n",
       "L 54.355398 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"mc44648297a\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.355398\" xlink:href=\"#mc44648297a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- $\\mathdefault{10^{0}}$ -->\n",
       "      <g transform=\"translate(45.555398 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-49\"/>\n",
       "        <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "z\n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-48\"/>\n",
       "       </defs>\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 94.026857 143.1 \n",
       "L 94.026857 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"94.026857\" xlink:href=\"#mc44648297a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- $\\mathdefault{10^{1}}$ -->\n",
       "      <g transform=\"translate(85.226857 157.698438)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.684375)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.684375)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 133.698316 143.1 \n",
       "L 133.698316 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"133.698316\" xlink:href=\"#mc44648297a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- $\\mathdefault{10^{2}}$ -->\n",
       "      <g transform=\"translate(124.898316 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-50\"/>\n",
       "       </defs>\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-50\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 173.369775 143.1 \n",
       "L 173.369775 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"173.369775\" xlink:href=\"#mc44648297a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- $\\mathdefault{10^{3}}$ -->\n",
       "      <g transform=\"translate(164.569775 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "z\n",
       "\" id=\"DejaVuSans-51\"/>\n",
       "       </defs>\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-51\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 213.041234 143.1 \n",
       "L 213.041234 7.2 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"213.041234\" xlink:href=\"#mc44648297a\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- $\\mathdefault{10^{4}}$ -->\n",
       "      <g transform=\"translate(204.241234 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"DejaVuSans-52\"/>\n",
       "       </defs>\n",
       "       <use transform=\"translate(0 0.684375)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.684375)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-52\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 2 \n",
       "\" id=\"m5c6f975fe9\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.554334\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_7\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"48.210211\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_8\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"50.510836\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_9\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"52.540131\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_10\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"66.297697\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_11\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"73.283494\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_12\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"78.239996\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_13\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"82.084558\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_14\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"85.225793\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_15\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"87.88167\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_16\">\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"90.182295\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_17\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"92.21159\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_18\">\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"105.969156\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_19\">\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"112.954953\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_20\">\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"117.911455\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_21\">\n",
       "     <g id=\"line2d_26\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"121.756017\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_22\">\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"124.897252\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_23\">\n",
       "     <g id=\"line2d_28\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"127.553129\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_24\">\n",
       "     <g id=\"line2d_29\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"129.853754\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_25\">\n",
       "     <g id=\"line2d_30\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"131.883049\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_26\">\n",
       "     <g id=\"line2d_31\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"145.640615\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_27\">\n",
       "     <g id=\"line2d_32\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"152.626412\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_28\">\n",
       "     <g id=\"line2d_33\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"157.582914\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_29\">\n",
       "     <g id=\"line2d_34\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"161.427476\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_30\">\n",
       "     <g id=\"line2d_35\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"164.568711\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_31\">\n",
       "     <g id=\"line2d_36\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"167.224588\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_32\">\n",
       "     <g id=\"line2d_37\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"169.525213\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_33\">\n",
       "     <g id=\"line2d_38\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"171.554508\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_34\">\n",
       "     <g id=\"line2d_39\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"185.312074\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_35\">\n",
       "     <g id=\"line2d_40\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"192.297871\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_36\">\n",
       "     <g id=\"line2d_41\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"197.254373\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_37\">\n",
       "     <g id=\"line2d_42\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"201.098935\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_38\">\n",
       "     <g id=\"line2d_43\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"204.24017\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_39\">\n",
       "     <g id=\"line2d_44\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"206.896047\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_40\">\n",
       "     <g id=\"line2d_45\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"209.196672\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_41\">\n",
       "     <g id=\"line2d_46\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"211.225968\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_42\">\n",
       "     <g id=\"line2d_47\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"224.983533\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_43\">\n",
       "     <g id=\"line2d_48\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"231.96933\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_44\">\n",
       "     <g id=\"line2d_49\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"236.925832\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_45\">\n",
       "     <g id=\"line2d_50\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"240.770394\" xlink:href=\"#m5c6f975fe9\" y=\"143.1\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- token: x -->\n",
       "     <g transform=\"translate(122.916406 171.376563)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 18.3125 70.21875 \n",
       "L 18.3125 54.6875 \n",
       "L 36.8125 54.6875 \n",
       "L 36.8125 47.703125 \n",
       "L 18.3125 47.703125 \n",
       "L 18.3125 18.015625 \n",
       "Q 18.3125 11.328125 20.140625 9.421875 \n",
       "Q 21.96875 7.515625 27.59375 7.515625 \n",
       "L 36.8125 7.515625 \n",
       "L 36.8125 0 \n",
       "L 27.59375 0 \n",
       "Q 17.1875 0 13.234375 3.875 \n",
       "Q 9.28125 7.765625 9.28125 18.015625 \n",
       "L 9.28125 47.703125 \n",
       "L 2.6875 47.703125 \n",
       "L 2.6875 54.6875 \n",
       "L 9.28125 54.6875 \n",
       "L 9.28125 70.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-116\"/>\n",
       "       <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "z\n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-111\"/>\n",
       "       <path d=\"M 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 31.109375 \n",
       "L 44.921875 54.6875 \n",
       "L 56.390625 54.6875 \n",
       "L 27.390625 29.109375 \n",
       "L 57.625 0 \n",
       "L 45.90625 0 \n",
       "L 18.109375 26.703125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-107\"/>\n",
       "       <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "z\n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"DejaVuSans-101\"/>\n",
       "       <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-110\"/>\n",
       "       <path d=\"M 11.71875 12.40625 \n",
       "L 22.015625 12.40625 \n",
       "L 22.015625 0 \n",
       "L 11.71875 0 \n",
       "z\n",
       "M 11.71875 51.703125 \n",
       "L 22.015625 51.703125 \n",
       "L 22.015625 39.3125 \n",
       "L 11.71875 39.3125 \n",
       "z\n",
       "\" id=\"DejaVuSans-58\"/>\n",
       "       <path id=\"DejaVuSans-32\"/>\n",
       "       <path d=\"M 54.890625 54.6875 \n",
       "L 35.109375 28.078125 \n",
       "L 55.90625 0 \n",
       "L 45.3125 0 \n",
       "L 29.390625 21.484375 \n",
       "L 13.484375 0 \n",
       "L 2.875 0 \n",
       "L 24.125 28.609375 \n",
       "L 4.6875 54.6875 \n",
       "L 15.28125 54.6875 \n",
       "L 29.78125 35.203125 \n",
       "L 44.28125 54.6875 \n",
       "z\n",
       "\" id=\"DejaVuSans-120\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-116\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "      <use x=\"100.390625\" xlink:href=\"#DejaVuSans-107\"/>\n",
       "      <use x=\"154.675781\" xlink:href=\"#DejaVuSans-101\"/>\n",
       "      <use x=\"216.199219\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"279.578125\" xlink:href=\"#DejaVuSans-58\"/>\n",
       "      <use x=\"313.269531\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      <use x=\"345.056641\" xlink:href=\"#DejaVuSans-120\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_51\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 45.478125 136.922727 \n",
       "L 240.778125 136.922727 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_52\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"mc7b3617abf\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#mc7b3617abf\" y=\"136.922727\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- $\\mathdefault{10^{0}}$ -->\n",
       "      <g transform=\"translate(20.878125 140.721946)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_53\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 45.478125 100.09077 \n",
       "L 240.778125 100.09077 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_54\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#mc7b3617abf\" y=\"100.09077\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- $\\mathdefault{10^{1}}$ -->\n",
       "      <g transform=\"translate(20.878125 103.889989)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.684375)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.684375)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_55\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 45.478125 63.258813 \n",
       "L 240.778125 63.258813 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_56\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#mc7b3617abf\" y=\"63.258813\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- $\\mathdefault{10^{2}}$ -->\n",
       "      <g transform=\"translate(20.878125 67.058032)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-50\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_57\">\n",
       "      <path clip-path=\"url(#p6acb9c5596)\" d=\"M 45.478125 26.426856 \n",
       "L 240.778125 26.426856 \n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-width:0.8;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_58\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#mc7b3617abf\" y=\"26.426856\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- $\\mathdefault{10^{3}}$ -->\n",
       "      <g transform=\"translate(20.878125 30.226075)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-51\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_59\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -2 0 \n",
       "\" id=\"ma2f59df5df\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"142.62807\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_60\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"140.492113\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_61\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"138.608065\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_8\">\n",
       "     <g id=\"line2d_62\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"125.835203\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_9\">\n",
       "     <g id=\"line2d_63\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"119.349418\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_10\">\n",
       "     <g id=\"line2d_64\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"114.747679\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_11\">\n",
       "     <g id=\"line2d_65\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"111.178294\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_12\">\n",
       "     <g id=\"line2d_66\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"108.261894\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_13\">\n",
       "     <g id=\"line2d_67\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"105.796112\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_14\">\n",
       "     <g id=\"line2d_68\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"103.660156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_15\">\n",
       "     <g id=\"line2d_69\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"101.776108\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_16\">\n",
       "     <g id=\"line2d_70\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"89.003246\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_17\">\n",
       "     <g id=\"line2d_71\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"82.517461\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_18\">\n",
       "     <g id=\"line2d_72\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"77.915722\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_19\">\n",
       "     <g id=\"line2d_73\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"74.346337\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_20\">\n",
       "     <g id=\"line2d_74\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"71.429937\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_21\">\n",
       "     <g id=\"line2d_75\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"68.964155\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_22\">\n",
       "     <g id=\"line2d_76\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"66.828198\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_23\">\n",
       "     <g id=\"line2d_77\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"64.944151\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_24\">\n",
       "     <g id=\"line2d_78\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"52.171289\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_25\">\n",
       "     <g id=\"line2d_79\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"45.685503\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_26\">\n",
       "     <g id=\"line2d_80\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"41.083765\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_27\">\n",
       "     <g id=\"line2d_81\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"37.51438\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_28\">\n",
       "     <g id=\"line2d_82\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"34.597979\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_29\">\n",
       "     <g id=\"line2d_83\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"32.132198\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_30\">\n",
       "     <g id=\"line2d_84\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"29.996241\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_31\">\n",
       "     <g id=\"line2d_85\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"28.112194\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_32\">\n",
       "     <g id=\"line2d_86\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"15.339332\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_33\">\n",
       "     <g id=\"line2d_87\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#ma2f59df5df\" y=\"8.853546\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- frequency: n(x) -->\n",
       "     <g transform=\"translate(14.798438 113.167187)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 37.109375 75.984375 \n",
       "L 37.109375 68.5 \n",
       "L 28.515625 68.5 \n",
       "Q 23.6875 68.5 21.796875 66.546875 \n",
       "Q 19.921875 64.59375 19.921875 59.515625 \n",
       "L 19.921875 54.6875 \n",
       "L 34.71875 54.6875 \n",
       "L 34.71875 47.703125 \n",
       "L 19.921875 47.703125 \n",
       "L 19.921875 0 \n",
       "L 10.890625 0 \n",
       "L 10.890625 47.703125 \n",
       "L 2.296875 47.703125 \n",
       "L 2.296875 54.6875 \n",
       "L 10.890625 54.6875 \n",
       "L 10.890625 58.5 \n",
       "Q 10.890625 67.625 15.140625 71.796875 \n",
       "Q 19.390625 75.984375 28.609375 75.984375 \n",
       "z\n",
       "\" id=\"DejaVuSans-102\"/>\n",
       "       <path d=\"M 41.109375 46.296875 \n",
       "Q 39.59375 47.171875 37.8125 47.578125 \n",
       "Q 36.03125 48 33.890625 48 \n",
       "Q 26.265625 48 22.1875 43.046875 \n",
       "Q 18.109375 38.09375 18.109375 28.8125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 20.953125 51.171875 25.484375 53.578125 \n",
       "Q 30.03125 56 36.53125 56 \n",
       "Q 37.453125 56 38.578125 55.875 \n",
       "Q 39.703125 55.765625 41.0625 55.515625 \n",
       "z\n",
       "\" id=\"DejaVuSans-114\"/>\n",
       "       <path d=\"M 14.796875 27.296875 \n",
       "Q 14.796875 17.390625 18.875 11.75 \n",
       "Q 22.953125 6.109375 30.078125 6.109375 \n",
       "Q 37.203125 6.109375 41.296875 11.75 \n",
       "Q 45.40625 17.390625 45.40625 27.296875 \n",
       "Q 45.40625 37.203125 41.296875 42.84375 \n",
       "Q 37.203125 48.484375 30.078125 48.484375 \n",
       "Q 22.953125 48.484375 18.875 42.84375 \n",
       "Q 14.796875 37.203125 14.796875 27.296875 \n",
       "z\n",
       "M 45.40625 8.203125 \n",
       "Q 42.578125 3.328125 38.25 0.953125 \n",
       "Q 33.9375 -1.421875 27.875 -1.421875 \n",
       "Q 17.96875 -1.421875 11.734375 6.484375 \n",
       "Q 5.515625 14.40625 5.515625 27.296875 \n",
       "Q 5.515625 40.1875 11.734375 48.09375 \n",
       "Q 17.96875 56 27.875 56 \n",
       "Q 33.9375 56 38.25 53.625 \n",
       "Q 42.578125 51.265625 45.40625 46.390625 \n",
       "L 45.40625 54.6875 \n",
       "L 54.390625 54.6875 \n",
       "L 54.390625 -20.796875 \n",
       "L 45.40625 -20.796875 \n",
       "z\n",
       "\" id=\"DejaVuSans-113\"/>\n",
       "       <path d=\"M 8.5 21.578125 \n",
       "L 8.5 54.6875 \n",
       "L 17.484375 54.6875 \n",
       "L 17.484375 21.921875 \n",
       "Q 17.484375 14.15625 20.5 10.265625 \n",
       "Q 23.53125 6.390625 29.59375 6.390625 \n",
       "Q 36.859375 6.390625 41.078125 11.03125 \n",
       "Q 45.3125 15.671875 45.3125 23.6875 \n",
       "L 45.3125 54.6875 \n",
       "L 54.296875 54.6875 \n",
       "L 54.296875 0 \n",
       "L 45.3125 0 \n",
       "L 45.3125 8.40625 \n",
       "Q 42.046875 3.421875 37.71875 1 \n",
       "Q 33.40625 -1.421875 27.6875 -1.421875 \n",
       "Q 18.265625 -1.421875 13.375 4.4375 \n",
       "Q 8.5 10.296875 8.5 21.578125 \n",
       "z\n",
       "M 31.109375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-117\"/>\n",
       "       <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-99\"/>\n",
       "       <path d=\"M 32.171875 -5.078125 \n",
       "Q 28.375 -14.84375 24.75 -17.8125 \n",
       "Q 21.140625 -20.796875 15.09375 -20.796875 \n",
       "L 7.90625 -20.796875 \n",
       "L 7.90625 -13.28125 \n",
       "L 13.1875 -13.28125 \n",
       "Q 16.890625 -13.28125 18.9375 -11.515625 \n",
       "Q 21 -9.765625 23.484375 -3.21875 \n",
       "L 25.09375 0.875 \n",
       "L 2.984375 54.6875 \n",
       "L 12.5 54.6875 \n",
       "L 29.59375 11.921875 \n",
       "L 46.6875 54.6875 \n",
       "L 56.203125 54.6875 \n",
       "z\n",
       "\" id=\"DejaVuSans-121\"/>\n",
       "       <path d=\"M 31 75.875 \n",
       "Q 24.46875 64.65625 21.28125 53.65625 \n",
       "Q 18.109375 42.671875 18.109375 31.390625 \n",
       "Q 18.109375 20.125 21.3125 9.0625 \n",
       "Q 24.515625 -2 31 -13.1875 \n",
       "L 23.1875 -13.1875 \n",
       "Q 15.875 -1.703125 12.234375 9.375 \n",
       "Q 8.59375 20.453125 8.59375 31.390625 \n",
       "Q 8.59375 42.28125 12.203125 53.3125 \n",
       "Q 15.828125 64.359375 23.1875 75.875 \n",
       "z\n",
       "\" id=\"DejaVuSans-40\"/>\n",
       "       <path d=\"M 8.015625 75.875 \n",
       "L 15.828125 75.875 \n",
       "Q 23.140625 64.359375 26.78125 53.3125 \n",
       "Q 30.421875 42.28125 30.421875 31.390625 \n",
       "Q 30.421875 20.453125 26.78125 9.375 \n",
       "Q 23.140625 -1.703125 15.828125 -13.1875 \n",
       "L 8.015625 -13.1875 \n",
       "Q 14.5 -2 17.703125 9.0625 \n",
       "Q 20.90625 20.125 20.90625 31.390625 \n",
       "Q 20.90625 42.671875 17.703125 53.65625 \n",
       "Q 14.5 64.65625 8.015625 75.875 \n",
       "z\n",
       "\" id=\"DejaVuSans-41\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-102\"/>\n",
       "      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"74.068359\" xlink:href=\"#DejaVuSans-101\"/>\n",
       "      <use x=\"135.591797\" xlink:href=\"#DejaVuSans-113\"/>\n",
       "      <use x=\"199.068359\" xlink:href=\"#DejaVuSans-117\"/>\n",
       "      <use x=\"262.447266\" xlink:href=\"#DejaVuSans-101\"/>\n",
       "      <use x=\"323.970703\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"387.349609\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"442.330078\" xlink:href=\"#DejaVuSans-121\"/>\n",
       "      <use x=\"494.259766\" xlink:href=\"#DejaVuSans-58\"/>\n",
       "      <use x=\"527.951172\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      <use x=\"559.738281\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"623.117188\" xlink:href=\"#DejaVuSans-40\"/>\n",
       "      <use x=\"662.130859\" xlink:href=\"#DejaVuSans-120\"/>\n",
       "      <use x=\"721.310547\" xlink:href=\"#DejaVuSans-41\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_88\">\n",
       "    <path clip-path=\"url(#p6acb9c5596)\" d=\"M -1 22.628465 \n",
       "L 54.355398 22.641392 \n",
       "L 66.297697 22.921582 \n",
       "L 73.283494 24.121839 \n",
       "L 78.239996 29.67948 \n",
       "L 82.084558 32.246865 \n",
       "L 85.225793 35.931745 \n",
       "L 87.88167 36.253723 \n",
       "L 90.182295 39.450499 \n",
       "L 94.026857 39.668628 \n",
       "L 95.668965 43.037948 \n",
       "L 97.168092 46.732081 \n",
       "L 98.547156 47.370841 \n",
       "L 101.012654 50.574172 \n",
       "L 102.124594 50.940227 \n",
       "L 103.169102 51.854528 \n",
       "L 104.15389 52.171289 \n",
       "L 105.085419 55.941884 \n",
       "L 105.969156 56.561158 \n",
       "L 106.809766 58.223127 \n",
       "L 107.611264 58.223127 \n",
       "L 108.377128 58.577295 \n",
       "L 109.110391 59.185578 \n",
       "L 109.813718 59.435519 \n",
       "L 110.489455 59.947432 \n",
       "L 111.139687 60.078011 \n",
       "L 112.37086 60.078011 \n",
       "L 113.519892 60.611258 \n",
       "L 114.066893 61.162896 \n",
       "L 114.597061 61.162896 \n",
       "L 115.111401 61.30383 \n",
       "L 115.61083 61.30383 \n",
       "L 116.096189 61.446017 \n",
       "L 116.568248 62.326748 \n",
       "L 117.027718 62.47837 \n",
       "L 117.475252 62.785993 \n",
       "L 117.911455 64.248567 \n",
       "L 118.752065 64.592578 \n",
       "L 119.157475 64.592578 \n",
       "L 119.553563 64.767399 \n",
       "L 119.94075 65.122878 \n",
       "L 121.407942 65.858452 \n",
       "L 122.097198 65.858452 \n",
       "L 122.431754 66.047755 \n",
       "L 122.759937 66.047755 \n",
       "L 123.081986 66.433217 \n",
       "L 123.708567 67.439582 \n",
       "L 124.013515 67.860551 \n",
       "L 124.313159 68.075265 \n",
       "L 124.897252 68.075265 \n",
       "L 126.009193 68.964155 \n",
       "L 126.53936 68.964155 \n",
       "L 126.798449 69.194316 \n",
       "L 127.553129 71.165535 \n",
       "L 128.038488 71.698782 \n",
       "L 128.276134 71.698782 \n",
       "L 128.510547 72.25042 \n",
       "L 128.970017 72.25042 \n",
       "L 129.417551 73.115275 \n",
       "L 129.637033 73.115275 \n",
       "L 130.898262 74.669498 \n",
       "L 131.690544 74.669498 \n",
       "L 131.883049 74.999322 \n",
       "L 132.073428 74.999322 \n",
       "L 132.261726 75.336091 \n",
       "L 132.447988 75.336091 \n",
       "L 132.632259 75.680102 \n",
       "L 133.173531 75.680102 \n",
       "L 133.350241 76.031675 \n",
       "L 133.525158 76.031675 \n",
       "L 133.869751 76.758887 \n",
       "L 134.207587 76.758887 \n",
       "L 134.374053 77.135279 \n",
       "L 134.538926 77.135279 \n",
       "L 134.864013 77.915722 \n",
       "L 135.340424 77.915722 \n",
       "L 135.650867 78.736206 \n",
       "L 135.804015 78.736206 \n",
       "L 135.955814 79.162789 \n",
       "L 136.54998 79.162789 \n",
       "L 136.695374 79.60106 \n",
       "L 137.40449 79.60106 \n",
       "L 137.542877 80.051679 \n",
       "L 138.085571 80.051679 \n",
       "L 138.218615 80.515361 \n",
       "L 138.868847 80.515361 \n",
       "L 138.995999 80.992887 \n",
       "L 139.247523 80.992887 \n",
       "L 139.371922 81.485108 \n",
       "L 139.860723 81.485108 \n",
       "L 139.980787 81.992957 \n",
       "L 140.218433 81.992957 \n",
       "L 140.336039 82.517461 \n",
       "L 140.568868 82.517461 \n",
       "L 140.684113 83.059747 \n",
       "L 140.798592 83.059747 \n",
       "L 141.025294 83.621065 \n",
       "L 141.469941 83.621065 \n",
       "L 141.688034 84.202798 \n",
       "L 142.010082 84.202798 \n",
       "L 142.221485 84.80649 \n",
       "L 142.738914 84.80649 \n",
       "L 142.941611 85.433861 \n",
       "L 143.141952 85.433861 \n",
       "L 143.33999 86.086846 \n",
       "L 143.632843 86.086846 \n",
       "L 143.825349 86.767626 \n",
       "L 144.204025 86.767626 \n",
       "L 144.390288 87.478673 \n",
       "L 145.204412 87.478673 \n",
       "L 145.380221 88.222803 \n",
       "L 145.981796 88.222803 \n",
       "L 146.149886 89.003246 \n",
       "L 147.046164 89.003246 \n",
       "L 147.204231 89.823729 \n",
       "L 147.898113 89.823729 \n",
       "L 148.048587 90.688584 \n",
       "L 148.271861 90.688584 \n",
       "L 148.419119 91.602885 \n",
       "L 148.924831 91.602885 \n",
       "L 149.066635 92.572632 \n",
       "L 150.160914 92.572632 \n",
       "L 150.292938 93.604984 \n",
       "L 150.938298 93.604984 \n",
       "L 151.064519 94.708589 \n",
       "L 151.803022 94.708589 \n",
       "L 151.923086 95.894013 \n",
       "L 153.246839 95.894013 \n",
       "L 153.41224 97.17437 \n",
       "L 154.163784 97.17437 \n",
       "L 154.320651 98.566197 \n",
       "L 155.767648 98.566197 \n",
       "L 155.910629 100.09077 \n",
       "L 157.496553 100.09077 \n",
       "L 157.625933 101.776108 \n",
       "L 159.107149 101.776108 \n",
       "L 159.225022 103.660156 \n",
       "L 161.427476 103.660156 \n",
       "L 161.56476 105.796112 \n",
       "L 164.103032 105.796112 \n",
       "L 164.220637 108.261894 \n",
       "L 166.674411 108.261894 \n",
       "L 166.801003 111.178294 \n",
       "L 170.013563 111.178294 \n",
       "L 170.138714 114.747679 \n",
       "L 174.078613 114.747679 \n",
       "L 174.193969 119.349418 \n",
       "L 179.386992 119.349418 \n",
       "L 179.508069 125.835203 \n",
       "L 186.804739 125.835203 \n",
       "L 186.922828 136.922727 \n",
       "L 199.579747 136.922727 \n",
       "L 199.579747 136.922727 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_89\">\n",
       "    <path clip-path=\"url(#p6acb9c5596)\" d=\"M -1 54.851831 \n",
       "L 54.355398 54.8653 \n",
       "L 73.283494 61.446017 \n",
       "L 78.239996 61.880322 \n",
       "L 82.084558 62.942052 \n",
       "L 85.225793 63.419577 \n",
       "L 87.88167 65.858452 \n",
       "L 90.182295 67.23318 \n",
       "L 92.21159 68.2929 \n",
       "L 94.026857 69.427838 \n",
       "L 95.668965 69.664818 \n",
       "L 97.168092 70.905433 \n",
       "L 98.547156 71.165535 \n",
       "L 99.823969 71.165535 \n",
       "L 101.012654 71.429937 \n",
       "L 102.124594 74.029576 \n",
       "L 103.169102 74.029576 \n",
       "L 104.15389 74.669498 \n",
       "L 105.085419 74.669498 \n",
       "L 105.969156 75.336091 \n",
       "L 106.809766 75.336091 \n",
       "L 108.377128 76.031675 \n",
       "L 109.813718 76.031675 \n",
       "L 110.489455 76.391149 \n",
       "L 111.139687 76.391149 \n",
       "L 111.766268 77.520741 \n",
       "L 112.37086 77.915722 \n",
       "L 112.954953 77.915722 \n",
       "L 113.519892 78.320704 \n",
       "L 115.111401 78.320704 \n",
       "L 115.61083 79.162789 \n",
       "L 116.096189 79.60106 \n",
       "L 117.027718 79.60106 \n",
       "L 117.475252 80.515361 \n",
       "L 118.336886 80.515361 \n",
       "L 118.752065 81.485108 \n",
       "L 119.157475 81.485108 \n",
       "L 119.94075 82.517461 \n",
       "L 121.052691 82.517461 \n",
       "L 121.756017 83.621065 \n",
       "L 122.431754 83.621065 \n",
       "L 122.759937 84.202798 \n",
       "L 123.708567 84.202798 \n",
       "L 124.013515 84.80649 \n",
       "L 124.607681 84.80649 \n",
       "L 124.897252 85.433861 \n",
       "L 125.462191 85.433861 \n",
       "L 125.737863 86.086846 \n",
       "L 126.53936 86.086846 \n",
       "L 126.798449 86.767626 \n",
       "L 127.553129 86.767626 \n",
       "L 127.797518 87.478673 \n",
       "L 129.637033 87.478673 \n",
       "L 129.853754 88.222803 \n",
       "L 130.898262 88.222803 \n",
       "L 131.099774 89.003246 \n",
       "L 131.495862 89.003246 \n",
       "L 131.690544 89.823729 \n",
       "L 133.173531 89.823729 \n",
       "L 133.350241 90.688584 \n",
       "L 134.538926 90.688584 \n",
       "L 134.702236 91.602885 \n",
       "L 135.18308 91.602885 \n",
       "L 135.340424 92.572632 \n",
       "L 137.542877 92.572632 \n",
       "L 137.680162 93.604984 \n",
       "L 139.980787 93.604984 \n",
       "L 140.10002 94.708589 \n",
       "L 141.796053 94.708589 \n",
       "L 142.010082 95.894013 \n",
       "L 143.535777 95.894013 \n",
       "L 143.729364 97.17437 \n",
       "L 145.292541 97.17437 \n",
       "L 145.467457 98.566197 \n",
       "L 147.438644 98.566197 \n",
       "L 147.593166 100.09077 \n",
       "L 150.489097 100.09077 \n",
       "L 150.61864 101.776108 \n",
       "L 152.854615 101.776108 \n",
       "L 152.967594 103.660156 \n",
       "L 156.099442 103.660156 \n",
       "L 156.239707 105.796112 \n",
       "L 159.458377 105.796112 \n",
       "L 159.57388 108.261894 \n",
       "L 163.225504 108.261894 \n",
       "L 163.349233 111.178294 \n",
       "L 167.82918 111.178294 \n",
       "L 167.947593 114.747679 \n",
       "L 174.128146 114.747679 \n",
       "L 174.243171 119.349418 \n",
       "L 182.205262 119.349418 \n",
       "L 182.318375 125.835203 \n",
       "L 195.477351 125.835203 \n",
       "L 195.591578 136.922727 \n",
       "L 225.183092 136.922727 \n",
       "L 225.183092 136.922727 \n",
       "\" style=\"fill:none;stroke:#bf00bf;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_90\">\n",
       "    <path clip-path=\"url(#p6acb9c5596)\" d=\"M -1 82.502365 \n",
       "L 54.355398 82.517461 \n",
       "L 66.297697 86.086846 \n",
       "L 73.283494 92.572632 \n",
       "L 78.239996 93.604984 \n",
       "L 82.084558 93.604984 \n",
       "L 85.225793 94.708589 \n",
       "L 87.88167 94.708589 \n",
       "L 90.182295 95.894013 \n",
       "L 92.21159 95.894013 \n",
       "L 94.026857 97.17437 \n",
       "L 98.547156 97.17437 \n",
       "L 99.823969 98.566197 \n",
       "L 105.969156 98.566197 \n",
       "L 106.809766 100.09077 \n",
       "L 109.813718 100.09077 \n",
       "L 110.489455 101.776108 \n",
       "L 114.066893 101.776108 \n",
       "L 114.597061 103.660156 \n",
       "L 121.052691 103.660156 \n",
       "L 121.407942 105.796112 \n",
       "L 126.276316 105.796112 \n",
       "L 126.53936 108.261894 \n",
       "L 132.632259 108.261894 \n",
       "L 132.814579 111.178294 \n",
       "L 140.568868 111.178294 \n",
       "L 140.684113 114.747679 \n",
       "L 148.271861 114.747679 \n",
       "L 148.419119 119.349418 \n",
       "L 160.652212 119.349418 \n",
       "L 160.795788 125.835203 \n",
       "L 181.787567 125.835203 \n",
       "L 181.903446 136.922727 \n",
       "L 231.900852 136.922727 \n",
       "L 231.900852 136.922727 \n",
       "\" style=\"fill:none;stroke:#008000;stroke-dasharray:9.6,2.4,1.5,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 45.478125 143.1 \n",
       "L 45.478125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 240.778125 143.1 \n",
       "L 240.778125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 45.478125 143.1 \n",
       "L 240.778125 143.1 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 45.478125 7.2 \n",
       "L 240.778125 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 159.996875 59.234375 \n",
       "L 233.778125 59.234375 \n",
       "Q 235.778125 59.234375 235.778125 57.234375 \n",
       "L 235.778125 14.2 \n",
       "Q 235.778125 12.2 233.778125 12.2 \n",
       "L 159.996875 12.2 \n",
       "Q 157.996875 12.2 157.996875 14.2 \n",
       "L 157.996875 57.234375 \n",
       "Q 157.996875 59.234375 159.996875 59.234375 \n",
       "z\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_91\">\n",
       "     <path d=\"M 161.996875 20.298438 \n",
       "L 181.996875 20.298438 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_92\"/>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- unigram -->\n",
       "     <g transform=\"translate(189.996875 23.798438)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 9.421875 54.6875 \n",
       "L 18.40625 54.6875 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 64.59375 \n",
       "L 9.421875 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-105\"/>\n",
       "       <path d=\"M 45.40625 27.984375 \n",
       "Q 45.40625 37.75 41.375 43.109375 \n",
       "Q 37.359375 48.484375 30.078125 48.484375 \n",
       "Q 22.859375 48.484375 18.828125 43.109375 \n",
       "Q 14.796875 37.75 14.796875 27.984375 \n",
       "Q 14.796875 18.265625 18.828125 12.890625 \n",
       "Q 22.859375 7.515625 30.078125 7.515625 \n",
       "Q 37.359375 7.515625 41.375 12.890625 \n",
       "Q 45.40625 18.265625 45.40625 27.984375 \n",
       "z\n",
       "M 54.390625 6.78125 \n",
       "Q 54.390625 -7.171875 48.1875 -13.984375 \n",
       "Q 42 -20.796875 29.203125 -20.796875 \n",
       "Q 24.46875 -20.796875 20.265625 -20.09375 \n",
       "Q 16.0625 -19.390625 12.109375 -17.921875 \n",
       "L 12.109375 -9.1875 \n",
       "Q 16.0625 -11.328125 19.921875 -12.34375 \n",
       "Q 23.78125 -13.375 27.78125 -13.375 \n",
       "Q 36.625 -13.375 41.015625 -8.765625 \n",
       "Q 45.40625 -4.15625 45.40625 5.171875 \n",
       "L 45.40625 9.625 \n",
       "Q 42.625 4.78125 38.28125 2.390625 \n",
       "Q 33.9375 0 27.875 0 \n",
       "Q 17.828125 0 11.671875 7.65625 \n",
       "Q 5.515625 15.328125 5.515625 27.984375 \n",
       "Q 5.515625 40.671875 11.671875 48.328125 \n",
       "Q 17.828125 56 27.875 56 \n",
       "Q 33.9375 56 38.28125 53.609375 \n",
       "Q 42.625 51.21875 45.40625 46.390625 \n",
       "L 45.40625 54.6875 \n",
       "L 54.390625 54.6875 \n",
       "z\n",
       "\" id=\"DejaVuSans-103\"/>\n",
       "       <path d=\"M 34.28125 27.484375 \n",
       "Q 23.390625 27.484375 19.1875 25 \n",
       "Q 14.984375 22.515625 14.984375 16.5 \n",
       "Q 14.984375 11.71875 18.140625 8.90625 \n",
       "Q 21.296875 6.109375 26.703125 6.109375 \n",
       "Q 34.1875 6.109375 38.703125 11.40625 \n",
       "Q 43.21875 16.703125 43.21875 25.484375 \n",
       "L 43.21875 27.484375 \n",
       "z\n",
       "M 52.203125 31.203125 \n",
       "L 52.203125 0 \n",
       "L 43.21875 0 \n",
       "L 43.21875 8.296875 \n",
       "Q 40.140625 3.328125 35.546875 0.953125 \n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \n",
       "Q 6 8.015625 6 15.921875 \n",
       "Q 6 25.140625 12.171875 29.828125 \n",
       "Q 18.359375 34.515625 30.609375 34.515625 \n",
       "L 43.21875 34.515625 \n",
       "L 43.21875 35.40625 \n",
       "Q 43.21875 41.609375 39.140625 45 \n",
       "Q 35.0625 48.390625 27.6875 48.390625 \n",
       "Q 23 48.390625 18.546875 47.265625 \n",
       "Q 14.109375 46.140625 10.015625 43.890625 \n",
       "L 10.015625 52.203125 \n",
       "Q 14.9375 54.109375 19.578125 55.046875 \n",
       "Q 24.21875 56 28.609375 56 \n",
       "Q 40.484375 56 46.34375 49.84375 \n",
       "Q 52.203125 43.703125 52.203125 31.203125 \n",
       "z\n",
       "\" id=\"DejaVuSans-97\"/>\n",
       "       <path d=\"M 52 44.1875 \n",
       "Q 55.375 50.25 60.0625 53.125 \n",
       "Q 64.75 56 71.09375 56 \n",
       "Q 79.640625 56 84.28125 50.015625 \n",
       "Q 88.921875 44.046875 88.921875 33.015625 \n",
       "L 88.921875 0 \n",
       "L 79.890625 0 \n",
       "L 79.890625 32.71875 \n",
       "Q 79.890625 40.578125 77.09375 44.375 \n",
       "Q 74.3125 48.1875 68.609375 48.1875 \n",
       "Q 61.625 48.1875 57.5625 43.546875 \n",
       "Q 53.515625 38.921875 53.515625 30.90625 \n",
       "L 53.515625 0 \n",
       "L 44.484375 0 \n",
       "L 44.484375 32.71875 \n",
       "Q 44.484375 40.625 41.703125 44.40625 \n",
       "Q 38.921875 48.1875 33.109375 48.1875 \n",
       "Q 26.21875 48.1875 22.15625 43.53125 \n",
       "Q 18.109375 38.875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.1875 51.21875 25.484375 53.609375 \n",
       "Q 29.78125 56 35.6875 56 \n",
       "Q 41.65625 56 45.828125 52.96875 \n",
       "Q 50 49.953125 52 44.1875 \n",
       "z\n",
       "\" id=\"DejaVuSans-109\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-117\"/>\n",
       "      <use x=\"63.378906\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"126.757812\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "      <use x=\"154.541016\" xlink:href=\"#DejaVuSans-103\"/>\n",
       "      <use x=\"218.017578\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"259.130859\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"320.410156\" xlink:href=\"#DejaVuSans-109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_93\">\n",
       "     <path d=\"M 161.996875 34.976562 \n",
       "L 181.996875 34.976562 \n",
       "\" style=\"fill:none;stroke:#bf00bf;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_94\"/>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- bigram -->\n",
       "     <g transform=\"translate(189.996875 38.476562)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path d=\"M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "z\n",
       "M 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "z\n",
       "\" id=\"DejaVuSans-98\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-98\"/>\n",
       "      <use x=\"63.476562\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "      <use x=\"91.259766\" xlink:href=\"#DejaVuSans-103\"/>\n",
       "      <use x=\"154.736328\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"195.849609\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"257.128906\" xlink:href=\"#DejaVuSans-109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_95\">\n",
       "     <path d=\"M 161.996875 49.654688 \n",
       "L 181.996875 49.654688 \n",
       "\" style=\"fill:none;stroke:#008000;stroke-dasharray:9.6,2.4,1.5,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_96\"/>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- trigram -->\n",
       "     <g transform=\"translate(189.996875 53.154688)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-116\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"80.322266\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "      <use x=\"108.105469\" xlink:href=\"#DejaVuSans-103\"/>\n",
       "      <use x=\"171.582031\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"212.695312\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"273.974609\" xlink:href=\"#DejaVuSans-109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p6acb9c5596\">\n",
       "   <rect height=\"135.9\" width=\"195.3\" x=\"45.478125\" y=\"7.2\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 252x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bigram_freqs = [freq for token, freq in bigram_vocab.token_freqs]\n",
    "trigram_freqs = [freq for token, freq in trigram_vocab.token_freqs]\n",
    "d2l.plot([freqs, bigram_freqs, trigram_freqs], xlabel='token: x',\n",
    "         ylabel='frequency: n(x)', xscale='log', yscale='log',\n",
    "         legend=['unigram', 'bigram', 'trigram'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This figure is quite exciting for a number of reasons. First, beyond unigram words, sequences of words also appear to be following Zipf's law, albeit with a smaller exponent $\\alpha$ in [(8.3.6)](#eq8.3.6), depending on the sequence length.\n",
    "Second, the number of distinct $n$-grams is not that large. This gives us hope that there is quite a lot of structure in language.\n",
    "Third, many $n$-grams occur very rarely, which makes Laplace smoothing rather unsuitable for language modeling. Instead, we will use deep learning based models.\n",
    "\n",
    "\n",
    "## 8.3.4 Reading Long Sequence Data\n",
    "\n",
    "Since sequence data are by their very nature sequential, we need to address\n",
    "the issue of processing it.\n",
    "We did so in a rather ad-hoc manner in [Section 8.1](../chapter_recurrent-neural-networks/sequence.ipynb).\n",
    "When sequences get too long to be processed by models\n",
    "all at once,\n",
    "we may wish to split such sequences for reading.\n",
    "Now let us describe general strategies.\n",
    "Before introducing the model,\n",
    "let us assume that we will use a neural network to train a language model,\n",
    "where the network processes a minibatch of sequences with predefined length, say $n$ time steps, at a time.\n",
    "Now the question is how to read minibatches of features and labels at random.\n",
    "\n",
    "To begin with,\n",
    "since a text sequence can be arbitrarily long,\n",
    "such as the entire *The Time Machine* book,\n",
    "we can partition such a long sequence into subsequences\n",
    "with the same number of time steps.\n",
    "When training our neural network,\n",
    "a minibatch of such subsequences\n",
    "will be fed into the model.\n",
    "Suppose that the network processes a subsequence\n",
    "of $n$ time steps\n",
    "at a time.\n",
    "[Fig. 8.3.1](#fig8.3.1)\n",
    "shows all the different ways to obtain subsequences\n",
    "from an original text sequence, where $n=5$ and a token at each time step corresponds to a character.\n",
    "Note that we have quite some freedom since we could pick an arbitrary offset that indicates the initial position.\n",
    "\n",
    "![Different offsets lead to different subsequences when splitting up text.](../img/timemachine-5gram.svg)\n",
    "<center id=\"fig8.3.1\"><i>Fig. 8.3.1 Different offsets lead to different subsequences when splitting up text.</i></center>\n",
    "\n",
    "Hence, which one should we pick from [Fig. 8.3.1](#fig8.3.1)?\n",
    "In fact, all of them are equally good.\n",
    "However, if we pick just one offset,\n",
    "there is limited coverage of all the possible subsequences\n",
    "for training our network.\n",
    "Therefore,\n",
    "we can start with a random offset to partition a sequence\n",
    "to get both *coverage* and *randomness*.\n",
    "In the following,\n",
    "we describe how to accomplish this for both\n",
    "*random sampling* and *sequential partitioning* strategies.\n",
    "\n",
    "\n",
    "### 8.3.4.1 Random Sampling\n",
    "\n",
    "In random sampling, each example is a subsequence arbitrarily captured on the original long sequence.\n",
    "The subsequences from two adjacent random minibatches\n",
    "during iteration\n",
    "are not necessarily adjacent on the original sequence.\n",
    "For language modeling,\n",
    "the target is to predict the next token based on what tokens we have seen so far, hence the labels are the original sequence, shifted by one token.\n",
    "\n",
    "The following code randomly generates a minibatch from the data each time.\n",
    "Here, the argument `batch_size` specifies the number of subsequence examples in each minibatch\n",
    "and `num_steps` is the predefined number of time steps\n",
    "in each subsequence.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "origin_pos": 14,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def seq_data_iter_random(corpus, batch_size, num_steps):  #@save\n",
    "    \"\"\"Generate a minibatch of subsequences using random sampling.\"\"\"\n",
    "    # Start with a random offset (inclusive of `num_steps - 1`) to partition a\n",
    "    # sequence\n",
    "    corpus = corpus[random.randint(0, num_steps - 1):]\n",
    "    # Subtract 1 since we need to account for labels\n",
    "    num_subseqs = (len(corpus) - 1) // num_steps\n",
    "    # The starting indices for subsequences of length `num_steps`\n",
    "    initial_indices = list(range(0, num_subseqs * num_steps, num_steps))\n",
    "    # In random sampling, the subsequences from two adjacent random\n",
    "    # minibatches during iteration are not necessarily adjacent on the\n",
    "    # original sequence\n",
    "    random.shuffle(initial_indices)\n",
    "\n",
    "    def data(pos):\n",
    "        # Return a sequence of length `num_steps` starting from `pos`\n",
    "        return corpus[pos:pos + num_steps]\n",
    "\n",
    "    num_batches = num_subseqs // batch_size\n",
    "    for i in range(0, batch_size * num_batches, batch_size):\n",
    "        # Here, `initial_indices` contains randomized starting indices for\n",
    "        # subsequences\n",
    "        initial_indices_per_batch = initial_indices[i:i + batch_size]\n",
    "        X = [data(j) for j in initial_indices_per_batch]\n",
    "        Y = [data(j + 1) for j in initial_indices_per_batch]\n",
    "        yield torch.tensor(X), torch.tensor(Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us manually generate a sequence from 0 to 34.\n",
    "We assume that\n",
    "the batch size and numbers of time steps are 2 and 5,\n",
    "respectively.\n",
    "This means that we can generate $\\lfloor (35 - 1) / 5 \\rfloor= 6$ feature-label subsequence pairs. With a minibatch size of 2, we only get 3 minibatches.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "origin_pos": 16,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X:  tensor([[ 9, 10, 11, 12, 13],\n",
      "        [29, 30, 31, 32, 33]]) \n",
      "Y: tensor([[10, 11, 12, 13, 14],\n",
      "        [30, 31, 32, 33, 34]])\n",
      "X:  tensor([[19, 20, 21, 22, 23],\n",
      "        [14, 15, 16, 17, 18]]) \n",
      "Y: tensor([[20, 21, 22, 23, 24],\n",
      "        [15, 16, 17, 18, 19]])\n",
      "X:  tensor([[ 4,  5,  6,  7,  8],\n",
      "        [24, 25, 26, 27, 28]]) \n",
      "Y: tensor([[ 5,  6,  7,  8,  9],\n",
      "        [25, 26, 27, 28, 29]])\n"
     ]
    }
   ],
   "source": [
    "my_seq = list(range(35))\n",
    "for X, Y in seq_data_iter_random(my_seq, batch_size=2, num_steps=5):\n",
    "    print('X: ', X, '\\nY:', Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8.3.4.2 Sequential Partitioning\n",
    "\n",
    "In addition to random sampling of the original sequence, we can also ensure that \n",
    "the subsequences from two adjacent minibatches\n",
    "during iteration\n",
    "are adjacent on the original sequence.\n",
    "This strategy preserves the order of split subsequences when iterating over minibatches, hence is called sequential partitioning.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "origin_pos": 18,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def seq_data_iter_sequential(corpus, batch_size, num_steps):  #@save\n",
    "    \"\"\"Generate a minibatch of subsequences using sequential partitioning.\"\"\"\n",
    "    # Start with a random offset to partition a sequence\n",
    "    offset = random.randint(0, num_steps)\n",
    "    num_tokens = ((len(corpus) - offset - 1) // batch_size) * batch_size\n",
    "    Xs = torch.tensor(corpus[offset:offset + num_tokens])\n",
    "    Ys = torch.tensor(corpus[offset + 1:offset + 1 + num_tokens])\n",
    "    Xs, Ys = Xs.reshape(batch_size, -1), Ys.reshape(batch_size, -1)\n",
    "    num_batches = Xs.shape[1] // num_steps\n",
    "    for i in range(0, num_steps * num_batches, num_steps):\n",
    "        X = Xs[:, i:i + num_steps]\n",
    "        Y = Ys[:, i:i + num_steps]\n",
    "        yield X, Y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the same settings,\n",
    "let us print features `X` and labels `Y` for each minibatch of subsequences read by sequential partitioning.\n",
    "Note that\n",
    "the subsequences from two adjacent minibatches\n",
    "during iteration\n",
    "are indeed adjacent on the original sequence.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "origin_pos": 21,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X:  tensor([[ 4,  5,  6,  7,  8],\n",
      "        [19, 20, 21, 22, 23]]) \n",
      "Y: tensor([[ 5,  6,  7,  8,  9],\n",
      "        [20, 21, 22, 23, 24]])\n",
      "X:  tensor([[ 9, 10, 11, 12, 13],\n",
      "        [24, 25, 26, 27, 28]]) \n",
      "Y: tensor([[10, 11, 12, 13, 14],\n",
      "        [25, 26, 27, 28, 29]])\n",
      "X:  tensor([[14, 15, 16, 17, 18],\n",
      "        [29, 30, 31, 32, 33]]) \n",
      "Y: tensor([[15, 16, 17, 18, 19],\n",
      "        [30, 31, 32, 33, 34]])\n"
     ]
    }
   ],
   "source": [
    "for X, Y in seq_data_iter_sequential(my_seq, batch_size=2, num_steps=5):\n",
    "    print('X: ', X, '\\nY:', Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we wrap the above two sampling functions to a class so that we can use it as a data iterator later.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "origin_pos": 23,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "class SeqDataLoader:  #@save\n",
    "    \"\"\"An iterator to load sequence data.\"\"\"\n",
    "    def __init__(self, batch_size, num_steps, use_random_iter, max_tokens):\n",
    "        if use_random_iter:\n",
    "            self.data_iter_fn = d2l.seq_data_iter_random\n",
    "        else:\n",
    "            self.data_iter_fn = d2l.seq_data_iter_sequential\n",
    "        self.corpus, self.vocab = d2l.load_corpus_time_machine(max_tokens)\n",
    "        self.batch_size, self.num_steps = batch_size, num_steps\n",
    "\n",
    "    def __iter__(self):\n",
    "        return self.data_iter_fn(self.corpus, self.batch_size, self.num_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Last, we define a function `load_data_time_machine` that returns both the data iterator and the vocabulary, so we can use it similarly as other other functions with the `load_data` prefix, such as `d2l.load_data_fashion_mnist` defined in [Section 3.5](../chapter_linear-networks/image-classification-dataset.ipynb).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "origin_pos": 25,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def load_data_time_machine(batch_size, num_steps,  #@save\n",
    "                           use_random_iter=False, max_tokens=10000):\n",
    "    \"\"\"Return the iterator and the vocabulary of the time machine dataset.\"\"\"\n",
    "    data_iter = SeqDataLoader(batch_size, num_steps, use_random_iter,\n",
    "                              max_tokens)\n",
    "    return data_iter, data_iter.vocab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8.3.5 Summary\n",
    "\n",
    "* Language models are key to natural language processing.\n",
    "* $n$-grams provide a convenient model for dealing with long sequences by truncating the dependence.\n",
    "* Long sequences suffer from the problem that they occur very rarely or never.\n",
    "* Zipf's law governs the word distribution for not only unigrams but also the other $n$-grams.\n",
    "* There is a lot of structure but not enough frequency to deal with infrequent word combinations efficiently via Laplace smoothing.\n",
    "* The main choices for reading long sequences are random sampling and sequential partitioning. The latter can ensure that the subsequences from two adjacent minibatches during iteration are adjacent on the original sequence.\n",
    "\n",
    "\n",
    "## 8.3.6 Exercises\n",
    "\n",
    "1. Suppose there are $100,000$ words in the training dataset. How much word frequency and multi-word adjacent frequency does a four-gram need to store?\n",
    "1. How would you model a dialogue?\n",
    "1. Estimate the exponent of Zipf's law for unigrams, bigrams, and trigrams.\n",
    "1. What other methods can you think of for reading long sequence data?\n",
    "1. Consider the random offset that we use for reading long sequences.\n",
    "    1. Why is it a good idea to have a random offset?\n",
    "    1. Does it really lead to a perfectly uniform distribution over the sequences on the document?\n",
    "    1. What would you have to do to make things even more uniform?\n",
    "1. If we want a sequence example to be a complete sentence, what kind of problem does this introduce in minibatch sampling? How can we fix the problem?\n"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.g4dn.xlarge",
  "kernelspec": {
   "display_name": "d2l",
   "language": "python",
   "name": "conda-env-d2l-py"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}