diff --git a/.buildkite/pipeline.yml b/.buildkite/pipeline.yml index 001d983bc..5b46c2f0f 100644 --- a/.buildkite/pipeline.yml +++ b/.buildkite/pipeline.yml @@ -115,7 +115,7 @@ steps: <<: *timeout key: "test-notebooks" depends_on: "runner-3_6" - parallelism: 51 + parallelism: 52 command: ".buildkite/steps/test-demo-notebooks.sh" plugins: <<: *plugins diff --git a/demos/embeddings/deep-graph-infomax-embeddings.ipynb b/demos/embeddings/deep-graph-infomax-embeddings.ipynb index 3395a0044..352d874ca 100644 --- a/demos/embeddings/deep-graph-infomax-embeddings.ipynb +++ b/demos/embeddings/deep-graph-infomax-embeddings.ipynb @@ -25,7 +25,9 @@ "source": [ "This demo demonstrates how to perform unsupervised training of a GCN, GAT, APPNP, or GraphSAGE model using the Deep Graph Infomax algorithm (https://arxiv.org/pdf/1809.10341.pdf) on the CORA dataset. \n", "\n", - "As with all StellarGraph workflows: first we load the dataset, next we create our data generators, and then we train our model. We then take the embeddings created through unsupervised training and predict the node classes using logistic regression." + "As with all StellarGraph workflows: first we load the dataset, next we create our data generators, and then we train our model. We then take the embeddings created through unsupervised training and predict the node classes using logistic regression.\n", + "\n", + "> See [the GCN + Deep Graph Infomax fine-tuning demo](../node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb) for semi-supervised training using Deep Graph Infomax, by fine-tuning the base model for node classification using labelled data." ] }, { @@ -603,6 +605,17 @@ ")" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "This notebook demonstrated how to use the Deep Graph Infomax algorithm to train other algorithms to yield useful embedding vectors for nodes, without supervision. To validate the quality of these vectors, it used logistic regression to perform a supervised node classification task.\n", + "\n", + "See [the GCN + Deep Graph Infomax fine-tuning demo](../node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb) for semi-supervised training using Deep Graph Infomax, by fine-tuning the base model for node classification using labelled data." + ] + }, { "cell_type": "markdown", "metadata": { diff --git a/demos/node-classification/README.md b/demos/node-classification/README.md index 3d5826d7f..00d954341 100644 --- a/demos/node-classification/README.md +++ b/demos/node-classification/README.md @@ -11,6 +11,7 @@ These demos are displayed with detailed descriptions in the documentation: https | [Node classification with Cluster-GCN](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/cluster-gcn-node-classification.html) | [source](cluster-gcn-node-classification.ipynb) | | [Node classification with directed GraphSAGE](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/directed-graphsage-node-classification.html) | [source](directed-graphsage-node-classification.ipynb) | | [Node classification with Graph ATtention Network (GAT)](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gat-node-classification.html) | [source](gat-node-classification.ipynb) | +| [Semi-supervised node classification via GCN, Deep Graph Infomax and fine-tuning](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.html) | [source](gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb) | | [Node classification with Graph Convolutional Network (GCN)](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn-node-classification.html) | [source](gcn-node-classification.ipynb) | | [(Moved) Node classification with Graph Convolutional Network (GCN)](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn/gcn-cora-node-classification-example.html) | [source](gcn/gcn-cora-node-classification-example.ipynb) | | [Inductive node classification and representation learning using GraphSAGE](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/graphsage-inductive-node-classification.html) | [source](graphsage-inductive-node-classification.ipynb) | diff --git a/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb b/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb new file mode 100644 index 000000000..46181ea56 --- /dev/null +++ b/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb @@ -0,0 +1,858 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Semi-supervised node classification via GCN, Deep Graph Infomax and fine-tuning" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "nbsphinx": "hidden", + "tags": [ + "CloudRunner" + ] + }, + "source": [ + "
Run the latest release of this notebook:
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This demo demonstrates how to perform semi-supervised node classification, using [the Deep Graph Infomax algorithm](https://arxiv.org/pdf/1809.10341.pdf) and GCN on the Cora dataset. It uses very few labelled training examples, demonstrating the benefits of pre-training a model with Deep Graph Infomax for data scarce environments.\n", + "\n", + "> Other related demos:\n", + "> - [the GCN node classification demo](gcn-node-classification.ipynb) describes the node classification task in more detail, in a supervised context\n", + "> - [the Deep Graph Infomax embeddings demo](../embeddings/deep-graph-infomax-embeddings.ipynb) describes using Deep Graph Infomax in more detail, including applying to algorithms beyond GCN.\n", + "\n", + "This follows the usual StellarGraph workflow:\n", + "\n", + "1. load the dataset\n", + "2. create our data generators\n", + "3. train our model\n", + "\n", + "We do step 3 three times:\n", + "\n", + "1. Pre-train a GCN model using Deep Graph Infomax, without any labelled data\n", + "2. Fine-tune that GCN model using the small training set\n", + "3. Train a fresh GCN model from scratch with the training set (no pre-training)." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "nbsphinx": "hidden", + "tags": [ + "CloudRunner" + ] + }, + "outputs": [], + "source": [ + "# install StellarGraph if running on Google Colab\n", + "import sys\n", + "if 'google.colab' in sys.modules:\n", + " %pip install -q stellargraph[demos]==1.1.0b" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "nbsphinx": "hidden", + "tags": [ + "VersionCheck" + ] + }, + "outputs": [], + "source": [ + "# verify that we're using the correct version of StellarGraph for this notebook\n", + "import stellargraph as sg\n", + "\n", + "try:\n", + " sg.utils.validate_notebook_version(\"1.1.0b\")\n", + "except AttributeError:\n", + " raise ValueError(\n", + " f\"This notebook requires StellarGraph version 1.1.0b, but a different version {sg.__version__} is installed. Please see .\"\n", + " ) from None" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "import stellargraph as sg\n", + "from stellargraph.mapper import CorruptedGenerator, FullBatchNodeGenerator\n", + "from stellargraph.layer import GCN, DeepGraphInfomax\n", + "\n", + "import pandas as pd\n", + "from sklearn import model_selection, preprocessing\n", + "from IPython.display import display, HTML\n", + "\n", + "import tensorflow as tf\n", + "from tensorflow.keras import Model, layers, optimizers, callbacks" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Loading the graph" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [ + "DataLoadingLinks" + ] + }, + "source": [ + "(See [the \"Loading from Pandas\" demo](../basics/loading-pandas.ipynb) for details on how data can be loaded.)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [ + "DataLoading" + ] + }, + "outputs": [ + { + "data": { + "text/html": [ + "The Cora dataset consists of 2708 scientific publications classified into one of seven classes. The citation network consists of 5429 links. Each publication in the dataset is described by a 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 1433 unique words." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "dataset = sg.datasets.Cora()\n", + "display(HTML(dataset.description))\n", + "G, node_classes = dataset.load()" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "StellarGraph: Undirected multigraph\n", + " Nodes: 2708, Edges: 5429\n", + "\n", + " Node types:\n", + " paper: [2708]\n", + " Features: float32 vector, length 1433\n", + " Edge types: paper-cites->paper\n", + "\n", + " Edge types:\n", + " paper-cites->paper: [5429]\n", + " Weights: all 1 (default)\n" + ] + } + ], + "source": [ + "print(G.info())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Data Generators\n", + "\n", + "Now we create the data generators using `CorruptedGenerator` ([docs](https://stellargraph.readthedocs.io/en/stable/api.html#stellargraph.mapper.CorruptedGenerator)). `CorruptedGenerator` returns shuffled node features along with the regular node features and we train our model to discriminate between the two. \n", + "\n", + "Note that:\n", + "\n", + "- We typically pass all nodes to `corrupted_generator.flow` because this is an unsupervised task\n", + "- We don't pass `targets` to `corrupted_generator.flow` because these are binary labels (true nodes, false nodes) that are created by `CorruptedGenerator`" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Using GCN (local pooling) filters...\n" + ] + } + ], + "source": [ + "fullbatch_generator = FullBatchNodeGenerator(G)\n", + "\n", + "corrupted_generator = CorruptedGenerator(fullbatch_generator)\n", + "gen = corrupted_generator.flow(G.nodes())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Model pre-training with Deep Graph Infomax\n", + "\n", + "We create and train our `GCN` ([docs](https://stellargraph.readthedocs.io/en/stable/api.html#stellargraph.layer.GCN)) and `DeepGraphInfomax` ([docs](https://stellargraph.readthedocs.io/en/stable/api.html#stellargraph.layer.DeepGraphInfomax)) models. Note that the loss used here must always be `tf.nn.sigmoid_cross_entropy_with_logits`." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "def make_gcn_model():\n", + " # function because we want to create a second one with the same parameters later\n", + " return GCN(\n", + " layer_sizes=[16, 16],\n", + " activations=[\"relu\", \"relu\"],\n", + " generator=fullbatch_generator,\n", + " dropout=0.4,\n", + " )\n", + "\n", + "\n", + "pretrained_gcn_model = make_gcn_model()" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "infomax = DeepGraphInfomax(pretrained_gcn_model, corrupted_generator)\n", + "x_in, x_out = infomax.in_out_tensors()\n", + "\n", + "dgi_model = Model(inputs=x_in, outputs=x_out)\n", + "dgi_model.compile(\n", + " loss=tf.nn.sigmoid_cross_entropy_with_logits, optimizer=optimizers.Adam(lr=1e-3)\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "tags": [ + "parameters" + ] + }, + "outputs": [], + "source": [ + "epochs = 500" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "dgi_es = callbacks.EarlyStopping(monitor=\"loss\", patience=50, restore_best_weights=True)\n", + "dgi_history = dgi_model.fit(gen, epochs=epochs, verbose=0, callbacks=[dgi_es])" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAEYCAYAAACju6QJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3ib1dn48e/R9t4zTuIMZzrbZLBHKAlQRim7UCgt9AVa1o+3QIEChU5KoW/TAS0FWiirtIyGTSAQskkgO3ESJ7YT721Ztmyf3x969FjySOLEkSL7/lxXLqRHj+U7KvTWOec+91Faa4QQQggRWSzhDkAIIYQQ/ScJXAghhIhAksCFEEKICCQJXAghhIhAksCFEEKICGQLdwBHKjU1Vefm5oY7DCGEEOKoWLt2bZXWOq379YhP4Lm5uaxZsybcYQghhBBHhVJqT2/XZQpdCCGEiECSwIUQQogIJAlcCCGEiECSwIUQQogIFNIErpRaoJTappQqVErd1cvrv1VKrTf+bFdK1YUyPiGEECJShKwKXSllBRYBZwIlwGql1Bta683+e7TWtwXc/wNgRqjiE0IIISJJKLeRzQYKtda7AJRSLwLnA5v7uP9y4Cchik0IIcQh6OzspKqqirq6Ojo6OsIdTsRzuVzk5ORgt9v7/bOhTODDgOKA5yXAnN5uVEqNBEYBH/Xx+vXA9QAjRowY2CiFEEL0qaSkBKUUubm52O12lFLhDiliaa2prq6mpKSEUaNG9fvnj9VGLpcBr2qte/16p7V+EngSoKCgYMAONP94WwUvrNxLi7eDi2bmcNbkTKIc1oF6eyGEiHjNzc2MHz8ei0VqoI+UUoqUlBQqKysP6+dDmcBLgeEBz3OMa725DLjpqEfUzSfbK1lfXEes08atL60HIDvBRYzThtWiOGdKFk1t7RSMTCYtzsnG0npinTbsVgtzRicT47BJwhdCDHqSvAfOkcxghDKBrwbylFKj8CXuy4Arut+klJoAJAHLQxgbALefOY4fnz0Ri1J8vL2CjaUN7K5qptHjpc7t5Tfvbwfgz+zq8z1Gp8Uwd3QK4zPiqGluI85lI85l47QJ6aTHuahtbiPGacNhk/8AhBBCHL6QJXCtdbtS6mbgXcAKPK213qSUeghYo7V+w7j1MuBFrfWATY0fqjhXVxHB6RMyOH1CRtDrFQ0eXA4rq3bVUNHYygljU2jv1DS3tvPBlgosCr4qqefN9ft4obW9x/snRNlp9HiZOzqFP101C7vFIiN2IYQQh0WFIU8OqIKCAn2sHWbS0ampdftG3+7WDsoaPCzdXsneGjc7yptYVVQDQJTdyuOXTae+xYvDamH+pAxincdqWYIQQsCWLVuYOHFiuMM4JjzzzDN897vfpb2954CtPw72mSql1mqtC7pfl2xxFFgtitRYJwBOm5WkGAcTs+LN1z/ZXsnG0nr+tmw3N/x9rXndZbfw47Mn0t6puey4ETI6F0KIATZ//nxycnJ45plnjvi9Lr30UhYuXHjkQR0mSeBhcMq4NE4Zl8aC/Eze3rCfU8en09rewUNvbua+1zcB8NoXpVx7Qi7jM+OYnJ0Q5oiFEGLoaGtrw+FwHPS+qKgooqKiQhBR76SSKozGpMVy8+l55A9LYNbIZH5/xUwWTM7k3nMmsruqmdtf/pKr/rqKvdXucIcqhBAR75prruHDDz/k2WefRSmFUopnnnkGpRTPP/88Z599NjExMdx3331orfne977HmDFjiIqKYvTo0dxzzz20traa7/fMM89gs9l6PF+2bBkzZ84kOjqaWbNmsXr16qPy95ER+DFkeHI0f7pqFgBnT8nio60V/OSNTZz86yVcPW8k3zlhFLmpMWGOUgghgj345iY272sI+e+dlB3PT74++ZDvf+KJJ9i1axdZWVk88cQTADQ0+OL+0Y9+xC9/+UsWLVoE+JqspKen88ILL5CRkcFXX33FDTfcgN1u58EHH+zzd3R2dnL33XfzxBNPkJaWxm233cYll1zCjh07gpL9QJAEfozKToziW3NHMnNEEv9YuYfnlu/hHyv28Pfr5nDC2NRwhyeEEBEnISEBh8NBVFQUmZmZAHg8HgBuuOEGrrzyyqD7H3nkEfNxbm4uO3fu5A9/+MMBE7jWmscff5yZM2cC8MADDzB37lx27tzJ+PHjB/TvIwn8GDcpO55HLsjn6nkjuen5L7j1pfW8fctJZpGcEEKEW39Gwceq2bNn97j21FNP8Ze//IWioiKam5tpb2+ns7PzgO+jlGLatGnm8+zsbADKy8sHPIHLGngEUEoxITOe318xk/oWL3e8/CWdnZG9/U8IIY4lMTHBy5OvvPIKN910E5deeimLFy9m3bp13H///Xi93gO+j8ViwWrt2kHk77R2sMR/OCSBR5CJWfHcd+4kPtleyWVPruCNL/eFOyQhhIgoDofjkE5RW7p0KTNmzOD2229n1qxZ5OXlUVRUdPQD7AeZQo8w35ozgn11LSzesJ8f/nMdn2yrRGvNLy6aKu1ZhRDiIEaNGsWSJUvYuXMnCQkJfY6ox48fz1//+ldef/118vPzeeutt3jttddCHO2Byf/jRxilFD9aMIEPbz+FMyak868vSnhtXSmfbD+802yEEGIoueOOO0hNTWXatGmkpaWxbNmyXu+74YYbuOqqq7j22muZMWMGK1eu5IEHHghtsAchrVQjWJ27jedX7uWJD3fgtFp48uoC5o1JCXdYQohBTFqpDrzDbaUqI/AIlhjt4KbTxvK9k0bR2NrO9/+xluZeDlERQggx+EgCHwTuPGsCL10/l/oWL49/sJ1In1URQghxcJLAB4k5o1O4eFYOT326m7tf2yDbzIQQYpCTKvRB5FffnEp6vJNFS3ZywthUvj4tO9whCSGEOEpkBD6IKKW448zxpMc5+cXbW3n03W20tQ988wAhxNAmy3QD50g+S0ngg4zFopg/KYPSuhZ+v6SQB9/cFO6QhBCDiN1up6WlJdxhDBper/ewDzmRBD4IXXN8LvNGp3DmpAxeXlNMo+fArf+EEOJQpaenU1paitvtlpH4Eers7KS8vJyEhITD+nlZAx+ExmXE8c/r57JyVzXvby5nWWEVC/Kzwh2WEGIQiI+PB2Dfvn0H7QsuDi4mJobU1MM7YVIS+CA2c2QScS4bH2ypkAQuhBgw8fHxZiIX4SNT6IOY3WphYX4mb321j6qm1nCHI4QQYgBJAh/kbjhlDK3tnTz7eVG4QxFCCDGAJIEPcmPSYjkpL43X1+9jd1Uz97++UbaWCSHEICAJfAg4d0oWe2vcnPboxzy3fA9fltSFOyQhhBBHSBL4EPC1yRmkxznN51vLGsMYjRBCiIEgCXwISIx28Mmdp/HhHacQ77KxdX9DuEMSQghxhCSBDxFRDitj0mKZkBUvI3AhhBgEJIEPMRMz49i6vwGPtyPcoQghhDgCksCHmLMmZ9Lc1sG/15WGOxQhhBBHQBL4EDNvTApThiXw1NJdeDs6pZexEEJEKEngQ4xSihtOGc2uqmbyfvw2f166K9whCSGEOAySwIegBZMzyU2JBuAvn0oCF0KISCQJfAiyWS3856YTuGvhBKqa2iisaAp3SEIIIfoppAlcKbVAKbVNKVWolLqrj3suUUptVkptUkq9EMr4hpLEaAcXTB+GRcGLq/aGOxwhhBD9FLIErpSyAouAhcAk4HKl1KRu9+QBdwMnaK0nA7eGKr6hKDPBxQUzhvGPlXsornGHOxwhhBD9EMoR+GygUGu9S2vdBrwInN/tnu8Bi7TWtQBa64oQxjck3XJGHg6rhcufWkFTa3u4wxFCCHGIQpnAhwHFAc9LjGuBxgHjlFLLlFIrlFILensjpdT1Sqk1Sqk1lZWVRyncoWFkSgxPXl1ASW0LL68uPvgPCCGEOCYca0VsNiAPOBW4HHhKKZXY/Sat9ZNa6wKtdUFaWlqIQxx85o5O4bjcJJ5dXhTuUIQQQhyiUCbwUmB4wPMc41qgEuANrbVXa70b2I4voYuj7KzJmeypdlPT3BbuUIQQQhyCUCbw1UCeUmqUUsoBXAa80e2e/+AbfaOUSsU3pS4blUMgLyMOgB3lctCJEEJEgpAlcK11O3Az8C6wBXhZa71JKfWQUuo847Z3gWql1GZgCXCn1ro6VDEOZXnpsQBslz3hQggREWyh/GVa68XA4m7X7g94rIHbjT8ihLISXMQ6bSz6qJAZwxPJH5YQ7pCEEEIcwLFWxCbCRCmFy26lrMHDjc9/Ee5whBBCHIQkcGF6+ILJKAV7a9zUt3jDHY4QQogDkAQuTAvys3jp+nkAfF5YFeZohBBCHIgkcBFkxohE0uKc/Pq9bTR6ZBQuhBDHKkngIojdauGJS6ezq7KZf6/rvk1fCCHEsUISuOhh3pgUMuKdrCmqpbW9g1pp7iKEEMccSeCiB6UUBSOTWbunlsueXMGMn74f7pCEEEJ0Iwlc9GrWyCRK61pYt7cOAHebnFQmhBDHEkngolcL8jNx2bv+9SitbQljNEIIIbqTBC56lZ0YxZp7z+ThC/IBKK2TBC6EEMcSSeCiT7FOG2dMTAd8CbzB4+WGv6+huMYd5siEEEJIAhcHlB7nwmZRlNa28PyKvby7qZxnPy8Kd1hCCDHkhfQwExF5rBZFZoKLfXUtaHzT6PFR9jBHJYQQQhK4OKiRKdGs2l1De6cGoEb2hQshRNjJFLo4qNvPHE9di5c6t6+1amVja5gjEkIIISNwcVCzRibxwe2n4LRZuPH5LySBCyHEMUBG4OKQZCdGkRLrJC3OSVWTJHAhhAg3SeCiX9LinDICF0KIY4AkcNEvqbFOGlvbaWnrCHcoQggxpEkCF/2SFucEYNO++jBHIoQQQ5skcNEvc0YlE+eycd2za/B4ZRQuhBDhIglc9MvIlBjuP3cS9S1eyuo94Q5HCCGGLEngot+yE6MA2F/vodNo7iKEECK0JIGLfstMcAGwdk8N0x58j0VLCsMckRBCDD2SwEW/ZRkJ/NH3ttPY2s57m8vDHJEQQgw9ksBFv0U7bETZreZziwpjMEIIMURJAheHpcWoQI9xWKlokMYuQggRapLAxRFZOCWLikYPWksxmxBChJIkcHFYvnviKFJjnUzOjsfboak1TioTQggRGpLAxWG599xJrLl3PulxvoK2ikbZEy6EEKEkCVwckYx4X2vVzwurZRpdCCFCSBK4OCIZ8b4R+ENvbebdTbKdTAghQkUSuDgiGfEuJmTGAfD7JTt4bnmRjMSFECIEQprAlVILlFLblFKFSqm7enn9GqVUpVJqvfHnu6GMT/Sfw2bhnVtPZsHkTDaWNnD/65tYsasm3GEJIcSgF7IErpSyAouAhcAk4HKl1KRebn1Jaz3d+POXUMUnjswlx+WYj//0yc4wRiKEEENDKEfgs4FCrfUurXUb8CJwfgh/vziKTp+QwfaHF/I/p47hs8IqGjyyrUwIIY6mUCbwYUBxwPMS41p3FymlvlJKvaqUGt7bGymlrldKrVFKramsrDwasYrD4LBZOG18Oh2dms8Lq8IdjhBCDGrHWhHbm0Cu1noq8D7wbG83aa2f1FoXaK0L0tLSQhqgOLAZIxKJc9r4ZLt8sRJCiKMplAm8FAgcUecY10xa62qttb+x9l+AWSGKTQwQu9XCzJFJrC+uD3coQggxqIUyga8G8pRSo5RSDuAy4I3AG5RSWQFPzwO2hDA+MUAmZMVRWNGIt6Mz3KEIIcSgFbIErrVuB24G3sWXmF/WWm9SSj2klDrPuO2HSqlNSqkvgR8C14QqPjFwJmTG4e3Q7K5qDncoQggxaNlC+cu01ouBxd2u3R/w+G7g7lDGJAbehMx4ALaWNTIuIy7M0QghxOB0rBWxiUFgTFosVovih/9cx6c7pJhNCCGOBkngYsA5bBYeu2QaI1Oiufc/G2lrl7VwIYQYaJLAxVFx/vRhPHjeZPZUu/nJG5vCHY4QQgw6ksDFUXPq+HSuPSGXf67ay766lnCHI4QQg4okcHFUnZzna7RT1uAJcyRCCDG4SAIXR1VanBOAqsZWtNac8ZuPeX7lnjBHJYQQkU8SuDiqUmN9CbyyqZXq5jZ2Vjazbm9dmKMSQojId0QJXCkVq5Q6RymVN1ABicElJdYBQFVjG6W1vnVwWQ8XQogj168ErpR6QSn1Q+OxHViJ7wCSTUqpc49CfCLC2a0WkqLtVDZ5KK2TBC6EEAOlvyPwU4FlxuOvA3FAFvAAcN+ARSUGldRYJ5WNrV0j8HoPWuswRyWEEJGtvwk8GSg3Hp8JvKa1LgdeACYOZGBi8EiLc7J0exX/+qIEgLb2Tqqb28IclRBCRLb+JvBKYJTx+ExgifE4GpB2W6JXLruVFm8HW8sazWsyjS6EEEemvwn8FeB5pdQHQDzwvnF9OrBjIAMTg0e0w9rjmn86XQghxOHp72lk/wuUACOAO7TWbuN6NvDUQAYmBo+ffH0y35o7EotSOG0WLv7TctbuqWXhlKyD/7AQQohe9SuBG2d6P9bL9UcHLCIx6KTFOc2GLgBzx6Twt8+LyExwUZCbzPThiWGMTgghIlN/t5FNU0pNDnh+tlLqFaXUA0qpkJ4tLiLXGRPS6ejUPPzfLXz76VVSkS6EEIehv2vgfwamACilcoBXgVjge8DDAxuaGKy+MXMYPzwjj8tnj6C+xUtDS3u4QxJCiIjT3wQ+HlhnPP4GsFprvRC4Grh0IAMTg1ecy87tZ47j5LxUAErq3Af5CSGEEN31N4E7AP+xUqcCbxuPtwOZAxSTGCKyE6MAqUgXQojD0d8Evg34plJqBL594B8Y17OA2oEMTAx+w5J8CVz2hAshRP/1N4E/CPwM2A18prVeY1z/Gl1T60IckpQYBy67hQff2szH2yrCHY4QQkSUfiVwrfXr+PaAzwLOCXjpQ+DOAYxLDAFKKTzeTrSGG5//ItzhCCFEROn3caJa63Kt9XrAoZRyGdeWa603D3h0YtCblpMAgNWiwhyJEEJEln4ncKXUtUqpQqAJaFJK7VBKXTPgkYkh4bnvzOHGU8fQ6Gmnqqk13OEIIUTE6G8jl1uAPwBvABcZf94C/qCU+sHAhycGu4RoOyeM9W0n27SvIczRCCFE5OjvCPwHwC1a69u11q8bf24DbgNuGfjwxFAwITMOgG8/vYql2yvDHI0QQkSG/ibw4fgK1rr70HhNiH5LiXXyk69PAuDVtSVhjkYIISJDf/uXl+Br4LKz2/VTjdeEOCzXnjCKbWWNvPnlPjzeDlz2nkeQCiGE6NLfEfgfgd8ppX5uHGRytlLqF8AT+NbGhThsp01Ip7mtg837ZS1cCCEOpr/HiT6qlGoBfmT8Ad/I+/9prf840MGJoSUrwQVARUMrLW0dRDlkFC6EEH3p9xGgWutFwCKlVJzxvHHAoxJDUmqs78zw7/9jLQCr7jmD+Ci7TKcLIUQvDprAlVLvHeR187HW+msDEJMYopJjHEHPZ//sQy4tGM4vvzm1x71Lt1cya2QSMU45hl4IMTQdyhp4aT/+CHHYXHYrca7ghPzSmmI6OjXtHZ3moSeVja1c/fQq/vWF1E0KIYaugw5ftNbXDtQvU0otwFfwZgX+orX+RR/3XQS8ChwXcGCKGALSYp00etpZmJ/JzBFJPLJ4C2uKavi/jwr5rLCKb88byb5634m25Q2eg7ybEEIMXiGbf1RKWYFF+I4hLQFWK6Xe6N5D3VhbvwVYGarYxLHDbvVNCs0amcSlxw3n/z7awbXPrMbd1gHAy2tKaPH6Hlc3tYUtTiGECLd+90I/ArOBQq31Lq11G/AicH4v9/0U+CUgw6shqLXdl5yHJUYR57Lz8vfnkZ+dYL7uT94A1c2SwIUQQ1coE/gwoDjgeYlxzaSUmgkM11r/90BvpJS6Xim1Rim1prJSWm8OJq3tnQBkJ0YBMCEznpe/P49/33h8j3ur5fATIcQQFsoEfkBKKQvwGHDHwe7VWj+ptS7QWhekpaUd/eBEyPi3kmUae8L90uKcPe6tkRG4EGIIC+UenFKC+6XnEFy5HgfkAx8bW9MygTeUUudJIdvQ8aerZvHhlnIy4oMTuD+xB5IpdCHEUBbKEfhqIE8pNUop5QAuw3csKQBa63qtdarWOldrnQusACR5DzHDEqO4el5uj+suu5X4blvMGj3t5pq5EEIMNSFL4FrrduBm4F1gC/Cy1nqTUuohpdR5oYpDRK7eptFrm71hiEQIIcIvpG2stNaLgcXdrt3fx72nhiImETnS4pwU17RgtSjiXDYqGlupamrtsV4uhBBDwTFTxCbEweQkRZOTFMWfr5rFwxfkA/Dwfzfj7egMc2RCCBF60khaRIwfLZhAo8fL6LRYAG4/cxyPvb+dHeVNTMqOD3N0QggRWjICFxEjLc5pJm+AU8b5thCW1Lr5xdtbeX7lnnCFJoQQIScjcBGxhiX5mr0UVjbxp092AnDlnJHhDEkIIUJGRuAiYqXEOHDZLfxnXVc7gf+sK6W4xs2ra0vQWocxOiGEOLpkBC4illKKYYlRbC9vMq/d+tJ68/HMEYnmlHt9i5dFSwq542vjcNqsIY9VCCEGmozARUTrbW+439ayRvPxpzsqeXLpLr4srgfg2c+LWL6z+qjHJ4QQR4skcBHR/P3Q545O7vHa1v0N5uNa475adxsNHi8/eWMTlz+1IjRBCiHEUSBT6CKiPXR+Pu9sLCMvI5YVu2qCXtsSMAKvMTq21Ta38XlhVUhjFEKIo0ESuIhoc0enMHd0Cm9v2B903aJg876AEbjbPwL3sr64DoDUWEfoAhVCiAEmU+hiUEiMDk7GC/IzKa1robDCV+Dmn2qvaPTw3uZywHeamXRxE0JEKkngYlBIjvEl8Kk5Cbx0/VzuO3cSAO9s9I3M/SPwN7/cR01zG/MnpqM1lDd4whOwEEIcIUngYlBIirYDkB7nYs7oFLISopgxIpG3N5YBXSPwqqY24l02Lp89AoB9dR72Vrv5+/Ii2TcuhIgoksDFoOCfQk+P79pWtjA/k037Gthb7Tar0AGm5iQyMiUagP31Lfxs8Rbue30TywplW5kQInJIAheDgsNm4ZypWWZ/dICF+VkAvL1xPzXurgQ+MSuOrARfG9bSuhbz+h8+Luz1vbeWNbCnuvlohC2EEIdNqtDFoLHoiplBz4cnR5M/LJ5/ryvF4+0qVpuUHU+M00ZClJ39dR4qm1oB2Baw7SzQgsc/BaDoF+ccpciFEKL/ZAQuBrWF+VlBHdkA8tLjAMhKcLG/voWSWjfga7da3uChwePlqaW7KKv30Nkp6+JCiGOTjMDFoLYwP5Nfv7sNgN9dPoN3N5UxIdOXwLMToyiqdlPR2EqU3UqLt4M5P/vQ/NlHFm/h4lk55nOtNV/srWXmiCSUUqH9iwghRDcyAheD2ui0WH79zam8e+vJnDctm0VXzMRm9f1rn53oorCiCa1hcnZ8j5+Nd9l4ZW2J+XzV7hou+uNy1u6pDVn8QgjRF0ngYtC7uGA4441RdyB/IRv41sUDDUuM4m/XHhd0bUOp7yCUvTVu81pFowePt2MgwxVCiEMiCVwMWdmJLvNxQW7wYSjjM+OYnJ0QdG3Lft9aekltC/UtXjo6NSf9cgkT7nuHCmkII4QIMUngYsjKiPMl8HEZsYxOjQl6bUJmHC578LnhW4zTzR57fztf++0n1Ld4aW33Vbd/tLUiBBF32V3VzNVPr6LR4w3p7xVCHDskgYsha+bIJK6aO5JnvzObRKOTG8DJ49K4cMYwAP52zXHceOoYwLcf3K+8oZWNxpQ6QEMfibSzU+Nuazefd3Rq/r68yJx2b25tp6Wt7yn4PdXNvXaIu/u1r1i6vVLW44UYwiSBiyHLZbfy0wvyyUqIIiGqK4H/7rLp5GX41sxPm5DOD07PA6D7jrLlu7o6tzW0tNObh/+7hUn3v2semvLupjLue30Tj3+wA4Dv/2MtP/7Phl5/dn99C6f8+mN+tnhLj9d2lPsOabFapBpeiKFKErgQQKzThtWisFoU8S570GtRDqt5WEqg5Tu7Enh9S+8j8KeX7Qag0eNL8P7BtP+UtOIaNzuNx91VNvoazDz16e6g62v31FBttIZtbu39i4MQYvCTBC4EoJQi3mUjOcaBpZdR7ZRhCT2u+c8VV8qXbP1r5L3xr1W3d/pG4jXNvuTc1NpuJuru6txdXwqKqnytXJtb27noj8sD3lcSuBBDlSRwIQyJ0Q5SehlpA0wfnhj0PC891nw8MjmadzaVsfCJT9kX0Fs9cHuZP9H6/1nR2Mr++haaWtupamrrdZ27LmBUv7/eV+Xu792eGe8rwGvqxwi8o9t6vBAiskkCF8KQEe9kWGJUr6/595EfPyaF//7wRE6fmG6+Niyp62fW7fWNyrXWPL9yr3m9ewIvqW1h3s8/wuPtpK2js9c19PqAA1j8XwZKa30J/LeXTge6ptB//e5W3jGOTu3L35bt5vRHP5H2sEIMEtJKVQjDY5dM77Mo7PgxKcS5bNx46lgmZyewaV/XdHliVNeofc2eGnJTo6lze/npW5vN6/6Rcl/bviqbPCREB6+9B06hu41K9RJjBD4qNQaHzUKj8b6LluwEYPfPz+6zzWtxjZuyBg/ljZ6gJjZCiMgkCVwIQ3Yfo2/wTa9veOAs8/mYtK594/FRXf8ZrSmq5b1N5eb+8EcuzOfH/97IXf/6ig82Z+Cw9T7pVdHYytj04G5xgVPoLQEjcLtVkR7nJNZpo7m13axwB1i7p7ZHUxq/ZuNLwO6qZkngQgwCMoUuxGEYndq1Bh4fsAVtR0Uj5Q0eqowjSvONbm7VzW28tKaY/fUeRiRH89dvFwS9n7+QrcHjZUe5r+NbfYvXTPgtxtp1aV0LWQlRWCyKWKeNJk87DQGJfnVR3/vC/fvNi6rcfd4jhIgcksCFOAxJAcVusY6uEbjH20l7wBrz6LTgDm8fbCknzmULahwDvgRe3uBh6gPvceZvl9LZqalze8lK8BWrdY3A3eQYa+4xThtNre1BW9j6aigD0Gx8Cdhd1fu2NSFEZJEELsRh+uVFU/j7dbPNKewRydFBrydF24nrtqccIM5lIyEquNq9sqmVV00+m1QAACAASURBVANOPqtsaqW+pY0Mo9rcvwZe3tBqVqDHGQk8cKq9oY/96IHvsVtG4EIMCiFN4EqpBUqpbUqpQqXUXb28/n2l1Aal1Hql1GdKqUmhjE+I/rj0uBGclJdmJsaRKcEJ3J98u4tz2Xsdga/b2zX9XVzjps7tJSXGgdNmMUfgjR4vcS7fiD/W1XMEfqB94eYUenXzof4VAV872Jue/4LVRTX9+jkhxNEVsgSulLICi4CFwCTg8l4S9Ata6yla6+nAr4DHQhWfEIfrzEkZAFxSMDzoelqcM+i5f4TuG4F3JfD0OCeVja2sL65jxgjffvPiWjd1LV4So+1EOay0tHWgtaa5rYMYpy+BxzhtNLd2UG9Uq8c5bYc0hb632k1HP7aS1bjb+O+G/Xz32TWH/DNCiKMvlCPw2UCh1nqX1roNeBE4P/AGrXVgK6sYQDasimPenNEpFP3iHE7OSwu63n0EPm90CuAbCdutXf/p5abGsL64jqqmNs6dmg3Anmo39W4vCVEOou2+BN7a3klHpzYTeKzTRqOnawQ+PDn6gFPoLW0d2CyKto7OoIYzvakL2IPeZlTUB1a7+1U0ePjmHz+npFam5YUItVAm8GFAccDzEuNaEKXUTUqpnfhG4D8MUWxCHLH4KFvQNrGM+OAR+JzRvu1dFd1ap45KiTGnvmeNTCIj3slnO6po6+hkdFoMLocVt7fD3EseayZwK02tXnO/eE5SFA3dptDvfm0Db2/Yzwsr99LQ4mWs0UFul9GadVdlEw++uSloRP7B5nKmP/S+OWXun74PTOBFVc2U1LpZuqOKNXtq+TygL7wQIjSOuX3gWutFwCKl1BXAvcC3u9+jlLoeuB5gxIgRoQ1QiD4o5duf7e3o5KKZOeZo2m/miCSgZ2IPfD46LYbhSdGsMY4JnZQVT7TDiqetw+y61jUCt+PxdlLT3EqMceCKvz87QE1zG/9ctZd/rurqCDc5O4GtZY0UVTVzyrg07nz1K9buqeUbM3KYkuPb8rbN2Mb236/2c1xusrl27u3oSvJ3vvolCVF2cpJ8ywI7K6WyXYhQC2UCLwUCFwlzjGt9eRH4Y28vaK2fBJ4EKCgokGl2ccxIj3PS2t7J/y6Y0OO13NQYnr6mgFkjgxutBK6Vx7vsTB+eaCbwcRlxRNmtuNsCR+BWALOYrbi2hcRoB/FRdnMkv3ZPLS8EtHI1Y0iJJsZhZbcxArcaXdtW7q7GYvEleP8If0dFI4UVjWw3EnqgqqY2Ojo1NcapaDsr+lcYJ4Q4cqFM4KuBPKXUKHyJ+zLgisAblFJ5WusdxtNzgB0IEUFumT+O9m5rxZ/ceSrlDb5p89MnZJjXE6Pt1Lm9ZgL3J86LC4bzl898R4g6bBaiHDaWbq/koTd9rVn9I3D/EadF1c3ER9mJd9lo8XbwjxV7eOitzebadaAoh5WRKTFmJXqs8SXg4f/6zhwv+sU55jr6ssJq5j+2NOjnL3tyOU9eXUCjpx2rRZlr37uqmthR3kiDx9vjC4oQ4ugIWQLXWrcrpW4G3gWswNNa601KqYeANVrrN4CblVLzAS9QSy/T50Icy04Zl9bj2siUGEamxPS4/smdp+HxdphJ0J/Ax2fGcfaUTCYbXdyi7L519ZW7fWvS/gTu34pWXOOmYGSy2RHu3v9s7DO+aIeNEcnR7Kjwjaod1p5lMP7+6lF2q7n+7bdiVw0bS+tp9HhpaPHS1tFJcoyDvdVufvnOVnZXNfPhHaf2+fuFEAMnpGvgWuvFwOJu1+4PeHxLKOMRIpwSouwkBEx7jw04ovQPV84yH0c7gv8z9Sf6pGjfCNzboUmLc5pT6oFunZ9HQ0s7Ty/zjehjnFaGJ0fx0bYKOjt1r9vOGlp8swKrfzyf219az2vrgle6dlY2m73ewXdW+ifbK9lQWm+ul/vi6mRPdXOPHu/90d7Ryeb9DUzNSTz4zUIMMdKJTYgwG5MWw4PnTeaJy6b3+rrLbg16HtMtgYNvHT2+W9e3m04bw63zx3H/1yfhNKrjo+xWRiRH09beSWVTK7Xu4ATe0alp9LQTH9AsprvN++qDnucaDWzKG1pp8HQdrvLnT3Yy/7GlbN7XEHT/5zuruPhPn/d5MlugdzaVcf6iZQfd9ibEUCQJXIgwU0rx7eNzSYl1HvxmunqvJ8YEN4Pp3pwlcB96tMNq/NPGcKOhzB0vf9mjQK2hxUuDx2u2gO1tVL+xNDghD+/WQrbWKGzbVu6rTH99ffAI/sVVxawuquWvxjr/gVQ2tqJ1z613QghJ4EIc82qag5NXjL8K3WnDZpxfnhbn5MS8VL45K4coY8SeHheYwH2JONppNTvCfVZYRUen5puzcvifU8cAviNMGzztXe1anT17uW8rC0763RN4VZMvgTcZI+zX1+8Let0fX+D2tr40GcsLgY1lDuaBNzZx9hOfHvL9QkQqSeBCHOP8CdHPZhSeKaXMQra0OCfRDhuPXjyNacN9xW+B+8v9I3CnzcKwpOCzwHNTojku17dH/dnPi9i6v8EsiOttCr2tW5X98KTgBO7fWran2lecV9bgoaLRg8coiKs0jlotb2iltT24SK47f0Hde5vLe4zke+Nua+eZz4vYvL+h1yr8I9Hg8fLhlvLD+tkL/7CMZz8vGtB4hJAELsQxbkJm30VgicY6eOBoOzvRl6ADp9BHpfqq4BUKp83KH66cSZxZze4wT0d75vMiWts7zTXw+F4SeHfDkqIwtpMDUN3cSntHJ3tr3OZxqKf9+mN++pZvG1xFo8e8t6LhwFPj/gK/F1bu5Z7XNvTazjXQW1/t73pv4/dorc1+8f2lteb0Rz/mhZV7eXVNCdc9u8Y86/1QdXZqviyuY2Np/cFvFqIfJIELcYy779xJ/PeHJ/b6WlLACNxvVEoMLruF1IA19V9fPI2ff2MKE7N8XwbOnpLFLGPU3f1wFcAsiPNXvANcPW8kv7poatB9Svmm8gN/vqa5jU+2V9LeqZk3xtf/vbmtgw1GAqtoaDUTe1mDh75sLWsIWj5obutg3V5fpzmte+/ftHR7pfl4f73vvZcVVjPtofd4f3P/R88t3g52VTWzaV+9uQ5f3XTo0/ngm0Xo1AQd+yrEQJAELsQxzmW3mnvCu0uKdmCzKBIDEuh3ThzFGzefGNSXPSHKzuWzR6AChsojjbVrj7ejx/GmsQEHpvg9dH4+00ck9rjPYlFBFfGri2q4zji57LTx6eb13VXNdHRqqppamTLM9/fxJ9nuKho8LHj8U97dFJx0P91RSWFFI6PuXsyKXcH917XWrNhVQ/6weACzcn3zft8Xhztf/ZLOfpzCBphV+tVNbVQbI2//EsGh8jfGqZcELgaYJHAhIsRFM3M4Z0pW0LXc1BhGp8VgsXQl5hinjXEZB997ffuZ47lyzgjOnZrdYwTuX3vuvgY+olvBmn8aPvDnF28oA+DZ78zmrMmZXe/paaewoolODdOG+74IlAck8LJ6D/8x9pwX93G62dIdVWYV/M8Xbwl6rbCiiaqmVi6ckWO+H2Ae9lLn9lJ9CMlX66798f6K+urmVvNn+5vA/Yn7cKfxheiLJHAhIsRvLpnGoitnBl27/cxxvHzDvMN6v4RoO49cOIUYpy3oeFMAu9X3haD73nKX3RpUHOffbuafyo8xiuVOn5DOKePSgmYBAFbt9o2ax6TFEGW3Bk2hP/7Bdm59aT01zW0U1/Tc950R7+SrkjozkX5ZUh9UBOc/UGXOqGTinDZ+/vZW3t1UZraxha6kfiBPLyti6gPvUd7gMZN/0Ai8HxXxEJDA+xiB765q5qRffcT+etnrLvpHErgQEcxlt5qFbAPlsUumcfNpeUDwFLrfyOQYs2jNv93MP4X+1NUF3DZ/HHct7HmYC8Anxhp1eryL1DgHf/1sN3/5dBdVTa28Z6xRby9vZG9NzxH4BTOGoTW8u7HMvOY/lAW6prtTYh3mDMINf19LaZ0bqzFDcaA1dz9/tfjOyiZqjWRd1dRq7gao7TYC93gPXEnv/xLQVwL/62e7KK5pMWcuhDhUksCFEAC8+v15LPl/p/KNmTlEGSPp3raRHTcqienGFLg/gfu/RMwelcwt8/OCpvDvOHMcV8wZgdNm4YMtFaTEOMjPTmBCpm+t+uH/buF7z60xp6Z39JHAvzYpA5fdwirjnHKAoqqu+/zJNinawUUzc8zrK3bVMM04KrXsEEa5/g5xxTVuc/95g6fdTP5LtlWwyuhL/8Hmcibc9w6bArrTNbW2B621+xN3i7cjaMags1Pz6Lvb+GxHFdA1e9GbfXUt0o1O9CAJXAgBQEFusrndzK/71DrAnWdN4F/fPx6AWGMK/cIZw7jzrPHmHvVAPzgjj59dOIXrTx4NwFn5mThsFh6/dDqrfnwGM0Yksm5vHS67hVinje3lTeyt7pnAsxKigrbLAewxTlV7fX0pG0vrcdktuOxWHr14KoWPLDSn+ydlx2OzKHZXuQ84Ym5t7zATblG1O6jVrL/T3bq9ddz92leArxkOwOeFvqWB+hYv8372YVD/+MCRd+Dj51ft5fdLCiky/q7+42J7c9drG/jRv77q83UxNEkCF0IcVPcqdYtFEeOwmvvEp+QkcNNpYw/4HjedNpYfnpHHrfN90/MxThvpcS5ONyrVTxmXxriMWN76ah9byxp6/HxqrNM8QjUz3kVyjIOiajf1LV5ueXE9izeUmVP5SilsVgtzR/u2saXFusiId/H0st184w+fA75itN++vz2oBe2Ocl+RHfi+HNT1UXi2u6oZj7fD/Pv7R/8rd1XT2Noe1KI2MGk3BDz+66e7gt7zQAV25fUeKkPUTtbj7TDrCcSxTRK4EOKA3r31ZN6/7ZQe1x++MJ+r5o085Pdx2a3cfua4HqPo0yb4Evi5U7OZMzqFWrcXi0Vx6njf0axXzBnB09cU4LBZSI31Jej4KBsjU6LZU93Mlv1dyb57PcA04xSz1vYOmtt8I9zN+xvQWvP2xv088eGOoOlv/znpwxKjKKpy99nCtVP71ur9p7IVVjRRXOPm852+kXhZvYfOTo3H20F9S9d7+L8QtLR1sKfGHVTk131tPVB9i/eAI/SBdNe/vuKM33xCc4h+nzh8ksCFEAc0PjMuqFGM34Uzcsx17CORPyyBj+44hXOnZvG/Z43nvdtO5qM7TqVgpK/RTLzLzukTMgDMEXi8y05uSgzbyhp5J6CoLanbTMEVc0Zw3Ymj+M6Jo3AHHHW6r95jrikv31nNF3trAczq9xPGprCnuplad5v5O7vbur/RHHm/t7mck361hOVGAi9v8PDLd7cy4b53glrh+kfjOyub0BrOzu/aZnegEXh9i+/89dtfXm/GerSs2OVb39/TyzLGQGpr7+SSPy1nmbEMIfovpOeBCyFEb0andZ2F7i+AyzP+Gbj33H9iW5zLxrlTs1i8YT/PBPQYT+o2AnfZrdx37iQAXvjuHF5aXcwra0vYVFrPvjpfUdrP394K+IrktpQ1kBRtJy89jua2EvbWuJmcHc8tZ+QxJi2WVUU1fLK9kn9/Ucrm/Q3UNAdPsW8zps4rGlt5Zpkvrg0l9aTGOqhqaqOktoV/rS3BYgydrj1hFLVuL1+V1PU5Am9t76DF20GLF177opQ3v9zHjkfONl//z7pSJmTFDciXKfDt6S9r8LCnuplJ2QPznr3ZW+NmVVENH22t4ISxqUft9wxmMgIXQhyTzpqcyUvXz+Xy2cPNaykxXWvcZ0zM4LUbjw/6md62vfkV5Cbz4PmTUQqW7qiktDa4qvu9zeUU17SQkxRNptHqdXdVMykxDgpyk0mKcXDW5Ex+duEURqZEU1LrNkfggWaOSKS8wWPGUtbgYWJWPBbl2+t+xytf8sLKvdgsiolZ8Tz7ndkcPza1zwYx3befeTu61uy11tz60noWPD5wp6/5m/IUHeURuL9Zj6y3Hz5J4EKIY9ac0SlB7V9TjDVw/6Emk7MTeO47s7nm+FwAPAc53SzaYeP8adn8Y8XeoO1o95w9gUsLfF8UshNdZgLv1DAiJabH+2QmuChvaKXW3cY5U7PY/vBC4lw2Yp025k/KwN3WQUdAv/YTx6aSPyzBrGpfXVTL2PRYcw08JcbRo0GM1pqiquagwrfuApcFDtfH2yqCTlnznzbnr/A/EHdbO5/uqKS6qbXf29xKjK2CuyoP/nt89zXJunw3ksCFEBEjOcY3hR54VOjJ49I4LjcZOLSE9tgl0xmZEtwSdnJ2AgXG4S7utg4yA05y694+FnxV8GUNHmqb20iKtuOwWTh3ahZfn5bNMOM0uMAK9uNGJZsHu/j5z2AH39R/ndtLe0cn72ws44E3NnH/65s49dGP+XRHzzVif6vXwINVOrr1eT/QtLyf1pp7/7OR+/6z0Twgxj+rENgkpy/Pr9jLVX9dxayHP+Dr//cZHm8Ht7+0nh3ljb3ev2RrBRf/6XPztDrwjcQDt/Z5vB3c95+NFFZ0jcy11pz+m0/49tOrDhqTn7ejk8fe327u6x9ohRVNXPrn5WaHvnCQBC6EiBhRdl+zk+7Hivr3r/sbthyIxaLMRjR+E7PimWkUzeUPSwg6irV7sgdfJ7nKxlZq3V6SjXX3n39jKj//xpSgn/XLz07glDxfVf0PTh/LzaeN5bxp2ebr/iLBsgYPd732Fc98XsTfV+wBCKqy9/Mnt6qA09oCR8wdnZpL/7yCP32y80AfBTsrmyipbWFfvYeiajdaazPpF/fSTOe372/niqdWmM8bA0bE1c1tbN7fwGvrSvna40t7PTFuWWEVq4tqKaltMQsGtQ7+svCs8Xf/62e7zWv+WYE1e4IL+LTW3Pj8WrMZTqD1xXX87sMdfLytssdr3XV0an77/nYqDqFTn98ra4pZubuGpz7dffCbjxIpYhNCRIycJN/o9tSAU87A16jlnVtPIi/94Ie4AHxzVg6vr9/HfedOItphJTnGQXKMg3duPYnRqbHmlrWqpjbz1LZAgSP0pG5V6hMDiskuO24404cn4rBZmDcmhTduPoEpwxKClgUA8wvF6+v39dh73ttIeHdlM1kJLj7d3pW4Tv/NJ7x584m8sGovW8saaPF2mNvi+rJka1dye3HVXv68tGtvenVzG1rroFif+HAH4OsMl50YRb0xWj8uN4l1e+vMLxFaw9ayRiZmBRfB7TcS5O7qZvbWuBmdFsOeaje3vbSef994Ao8s3szzK/cCBI2cW/qYWalubmPxhjKyEqI4MS+4EM6/b763OoXuVu2u4YkPd7C9vJE/fmvWQe8H8H89Wbq9ss/WwUebjMCFEBEjOzGKVfecwc29NI2ZkBlv9jw/mJPy0lh1zxlcd+IoLp89Iug9/OvSGfEuXHZLr1voMhO6rnXfZpYQbTdbzJ43PZvLjPdXSjE1J7FH8gaYlBVPQpSdp41R523zx5mv7ajoWeRVXOvmF29v5bcfbAe62rC+vKaYf67aa56bXlLbc126vMHD+HvfZsWuaraVN5IZ7yI9zslzy/eY92QnuGht7zSXJD7YXM5xj3xgvu4/W73G7SU3JZoF+Vm0d2q+KunaUx/4u59fuYfiGjf7jXXyoqpmimvcnDg2lUcvnsrWskae+byIf6zYy9enZjMtJyFoCr2vpRH/4TS99biv7Mf57ZXGNHj3ZYgD8RdBbt7fELI9+t1JAhdCRJT0eFfQ8alH8j4HMio1hnEZcb0m3MBp8jmjUnq8/udvzSI3JZrJWQef0gfftP7sUcnmXvDvnjSK3146jWiHNWhEnhzjIDPeRXFNC9vLuxLc2vvOpGBkkjnt7lfaS2HZR1sraG3v5JllRdS520iKcTA6LYaWgHXoMem+bX01xij8u8+tCeoE5y9cq2luJTnGYTbY+WJPLf7/afz3F9e4+fG/N3LZkyvMhPtlcR2Nre0MT4rmjIkZKAW/fGcrcU4bP70gn7mjU9hlnB8PwQk8cL3c/369TX0faATuP9r1V+9s5abnvzAL8GKctqDGPp8XVvXYHdDa3kFHp6Yk4Mjb8n5MvQ8kSeBCCNGLn56fz1NXF/T6WmAC91esBzp+bCof33kaCd0ayxzItSfkAr6tcDFOGxfOyDEL6PzfIRKj7AxPjmJvTTNFAVPrLruV2aN8hXz+OgHwFdIFjg5b2zvYWOpLUE67hTq3l6RoOyOTgyvtRxs1BdXNbUFfFAAsyneU67xffMSywmqSYxykGMWFX5bUUzDSF4c/ga40Dn4prfOttQNmYd7w5GjiXXYyjO58F84cRkKUnTHpsbS1d1Jc4+anb23me8+tMX///oAjYf1T8gccgXdLwB9uKWfaQ++xuqiG9zeXs2RbhflZ/ntdKef9fhk1zW14Ozq5+ulVPLe8KOjzO+u3S7n/9Y2U1rWQa9RH+BN4e0cna/fU9mskfyRkDVwIIXrRfW07UEqMg3OmZplbzwbC8WNSee+2k+kMKP5Kj3extayR8RlxbC1rJDHazvDkaF77orTHz1934iiiHVbOmZrNaY9+bF4vrW1hfGYcnZ2aCxd9zmajKK6o2k2Tx8v4zDhGdCvU889O1DT3rLA+KS/NPBYWfBX0/u19AGMzYtlR0ciiJYW8vKa4xyyAUl1JdXiyr6Yh1mWDBsxT5CZk+moZNpTWBxWzAfx9+R7uO3ciSinKjWRe3tDaY73ePy1e020K3b/Ov7WskZ2Vvt73/i8Z4JtG9yVkF+2dOqiS/+U1JRRVuymq9q3TnzwujaJqt5nA1xXXcfGflvOHK2dy9pSsHp/dQJMRuBBC9JNSikVXzOTkcWkD+r7jMoI7qvnXtk+fkI7DZiEx2sHwpJ5FdeDrUnfz6XnkpkQT47Bit/qSWWmdb6p3+a5qM3lbLYpdFU3Uub2+9zRG+pOy4rlt/jgz+VQ3tZmtYEen+Ubl3bfDJcc6zAY74Eu+aXFO2jo6zeR9wtiunzlhTFexmf/3PnHZdG6bP46pxi6CSVnxxDptLN9V3ePv+fSy3dzz7438v1e+NEfjbe2dPYr/uk+h/23Zbv731S/No2A376s3D67pXihY09xmFtGtL67jlF8voazew6trioNO7PPPNpQ3+H7XJ9sqsVpUyDrLyQhcCCGOUf5uaMePSeXlNcUkRtnJTe1K4N+clWOOVv2UUhTkJhPnsrF4w35WF9Vy+oQMnl+5h6RoO8vvPoN/rtrLg29uhlbftLy/0n5iVjy3zM8zp91rmtvMJPfKDfNo8LSzrSx4j3dytCNotmLOqBTe3VTG9vImLpiezTUnjCIz3sW/vijh1+9u48o5I8xjWOON42gnZycwOburXsBmtTB7VDJvrN8X9LsWXTGTm174gn+u8o2A89K7WvCWNXiC4ug+hf67D3dQ6/YS47DS3NZh9q3vzYuri0mP61oWAHhpdTGb9zfwnRNHcdHMHD7ZVskFM7L52eIt5gj84+0VzByRaHazO9okgQshxDHq3nMm8qt3tlKQm8RPz89nZEoMY9Nj+bK4nqqmVn510dReC/qeufY4wDcyfWl1MVfMHsF7m8q59oRcXHYrYwJ6zydFO8y1dv82vRiHFYfNQk1zGx2dmii7lZRYJymxzh7d0GKctqBz4/PSY3HafDMH4zPjzS1yN546hvOndzW6OZjjx6Tw0daKoGv5w+KZMiyBDcY6/o6KJjLinZQ3tLK3xk1uSgyNrV7sFgtVxhR6bXMbDR4vtW4vd541nhtPHcO4e9+mqNqNy27B4/XtMf/6tGze/NL3hcH/z0DLdlbh7dBMz0lkXEac2bM/Pd5JRUMrG0vr2VjawJ1njT+kv99AkAQuhBDHqBPGpvL6zScCsDBgTfWB8yYf8Of8a8FXzRvJe5vLufmf62jv1OaWOf90OPi2vSXFOFh0xUyOG5Vk/rzLZjHXi/2JvftjICihZyf4dgj4K7cDZweUUuQY0/9PXV2AzXrgnQTnTc/m4f9uCboW5bByXG6ymcABfnhGHo9/sIN7XttAdXMbGfFO0uKcdGjNSXmpfLqjigsWLQNgvLGrIN5lp7q5jbQ4J7NzU4hz2cyz3fvin3qf2q0JUEaci/IGD/f8ewPpcU6+NefQj9g9UpLAhRBikDp+TCopMQ6+LK5j3ugU89S37IQoc/TpP8HtnKnBRVcNnq7EHLjGnRBlJycpigWTM9lb4+Zio5Dvy/u/ht3mS8ojkqNZX1wXNNIPdOakjIPGnh7nYu7oZPN4U/D1sr989nDaOzvZVtbI5v0NXH7cCOaOTuGuf31FdXMb5Q2tVDa28j+njCEvI5ZPd1SZ/dbHG18oEqJ8CTw11slvLpkGwKIlhQeNKSPeSXa3XQdZiS7e2ViGu62De8+Z2K+dB0dKitiEEGKQsloUX5vsO3P8yrldDWssFsWoVF9y7X6Gut+fvjUTp/+wldiuxjVKKT64/RTuPnsiT15dYDaySYi2E+3wjQkfvjCfZ649rkd1e3899505/OIbU8znUXYreRlxPHR+Ps9dN5tV98zHYlGMSYvlle8fz8+Nezu1r0DuzEmZPHJhvvnz/un7OGONOjXg7xXYX78v154wqkdfgFPGpZn71E8Z4KLGg5ERuBBCDGLXnTgKu1XxtUmZQdfHpMWwZX8DiX0k8AX5Wdx+ppufv721R4c7V8Be897Eu+w92t0eDofNYjaVAYLi8K+zB8oOWF/PSnAR67Rx5ZyRxDhs7K5qNusFEnpJ4HEHmEJXypecr57Xc3p8/sQMXHYLCVF2xqb3PuNwtEgCF0KIQWxseiwPnZ/f47p/Oj0xuu/97v4p56ownrh1oMTaXeD0dmAyv2DGsKD7Yp2+5J8WsH/96nm5OGwW7n99U4/3XZifyR+u7L1HeozTxi1njCPaYe21a9/RFNIpdKXUAqXUNqVUoVLqrl5ev10ptVkp9ZVS6kOlVOiqAYQQYgi5fPZwHjxvctAotLs8o9K6+1a1UIpzHfqacmBXpgAb0gAACPRJREFUvKxeOuT5+TulBfa5d9gsXD0vN6iTnV/3fvfd/c+pY/i2cSZ9KIUsgSulrMAiYCEwCbhcKTWp223rgAKt9VTgVeBXoYpPCCGGkqyEqIMmnWGJUbxx8wncf+6Bq96Ppv6MwONcduKcNt+fAyR+/3p3b19e3vzBidxzdvDpYv5z6I81oRyBzwYKtda7tNZtwIvA+YE3aK2XaK39HeJXADkhjE8IIUQ3U3MSiXIceM37aIp19G+lNyvRRVbigQ+q8Z8vHu3s+d5j02O5am5u0LXkEFaW90co18CHAcUBz0uAOQe4/zrg7aMakRBCiGNaf0+e+/rU7IPeMyI5hmVUB22PCxT4hcVhtTD2EM+ZD7VjsohNKfUtoAA4pY/XrweuBxgxYkRvtwghhBiCfnBG3kHvuf/cSZw2Po38YQc/7nXNffPNlq/HmlAm8FIg8OieHONaEKXUfODHwCla615LH7XWTwJPAhQUFITm3DYhhBBhMTw5ivzsQztb/VBEOazm/vi+vPr9eSTHOI7Z5A2hTeCrgTyl1Ch8ifsy4IrAG5RSM4A/Awu01hU930IIIcRQ8+n/nh7y31mQmxzy39lfISti01q3AzcD7wJbgJe11puUUg8ppc4zbvs1EAu8opRar5R6I1TxCSGEEJEkpGvgWuvFwOJu1+4PeDw/lPEIIYQQkUp6oQshhBARSBK4EEIIEYEkgQshhBARSBK4EEIIEYEkgQshhBARSBK4EEIIEYEkgQshhBARSGkd2Z1IlVKVwJ4BfMtUoGoA30/IZ3q0yOc68OQzHXjymR65kVrrtO4XIz6BDzSl1BqtdUG44xhM5DM9OuRzHXjymQ48+UyPHplCF0IIISKQJHAhhBAiAkkC7+nJcAcwCMlnenTI5zrw5DMdePKZHiWyBi6EEEJEIBmBCyGEEBFIErgQQggRgSSBB1BKLVBKbVNKFSql7gp3PJFCKfW0UqpCKbUx4FqyUup9pdQO459JxnWllPqd8Rl/pZSaGb7Ij11KqeFKqSVKqc1KqU1KqVuM6/K5HiallEsptUop9aXxmT5oXB+llFppfHYvKaUcxnWn8bzQeD03nPEfy5RSVqXUOqXUW8Zz+UxDQBK4QSllBRYBC4FJwOVKqUnhjSpiPAMs6HbtLuBDrXUe8KHxHHyfb57x53rgjyGKMdK0A3dorScBc4GbjH8f5XM9fK3A6VrracB0YIFSai7wS+C3WuuxQC1wnXH/dUCtcf23xn2id7cAWwKey2caApLAu8wGCrXWu7TWbcCLwPlhjikiaK2XAjXdLp8PPGs8fha4IOD6c9pnBZColMoKTaSRQ2u9X2v9hfG4Ed//OQ5DPtfDZnw2TcZTu/FHA6cDrxrXu3+m/s/6VeAMpZQKUbgRQymVA5wD/MV4rpDPNCQkgXcZBhQHPC8xronDk6G13m88LgMyjMfyOfeTMc04A1iJfK5HxJjqXQ9UAO8DO4E6rXW7cUvg52Z+psbr9UBKaCOOCI8D/wt0Gs9TkM80JCSBi6NO+/Yqyn7Fw6CUigX+BdyqtW4IfE0+1/7TWndoracDOfhm3SaEOaSIppQ6F6jQWq8NdyxDkSTwLqXA8IDnOcY1cXjK/VO4xj8rjOvyOR8ipf5/e/cWomURx3H8+0sLjTIJzJvKEjuoIUEEdrKl80G6KCtJU7QugpK6CEIjksqlM1gWBmZBhWUUKRF1o2REF1kJllmWSAdCMisJD4X8unhm8/FlIZH1XZ93fx9YeHZmnveZGdj97zMzO6MjqYL3a7bfLsnp1z5g+w9gNXAe1XTD4JJV77f/+rTkHwf81uaqHu4uAK6TtIVq2vESYCHp07ZIAN/nU+C0snryKGAqsLKf69RkK4GZ5XomsKKWPqOsmp4I/FkbEo6izAu+CHxt++laVvr1IEkaIWl4uR4KXE61tmA1MKUUa+3Tnr6eAqxydr7aj+25tk+0fQrV78xVtqeRPm2L7MRWI+kaqvmcQcBS2wv6uUqNIGkZ0EV1bOBW4EHgHWA5cDLVca832d5eAtMiqlXrO4FZttf2R70PZ5IuBD4C1rNvbnEe1Tx4+vUgSJpAtYBqENXLy3LbD0kaTfX2eDzwBTDd9h5JQ4BXqNYfbAem2t7cP7U//EnqAu61PTl92h4J4BEREQ2UIfSIiIgGSgCPiIhooATwiIiIBkoAj4iIaKAE8IiIiAZKAI+IQ05SlySXfbMjog8kgEdERDRQAnhEREQDJYBHDACS5kjaKGm3pE2S7u/Zq1rSFkkLJC2RtEPSNkndko6o3X+spBck/Sppj6S1kq5oecYJkl6StLU85xtJs1uqMlbSGkk7JW2QdHUbmh/RkQb/f5GIaDJJ84FZwD3AOmAssBgYAjxQis2h2kb4XKpTuhZTbYu7sOQvLXnTgR+AO4B3JU2wvbHsLf4hsAuYBmwGxlBtpVn3JHAf1TGe84A3JI2y/Xvftjqi82Ur1YgOJuloYBtwve33a+kzgGdsDy8nSf1o+6Jafjdwq+2TJI0BNgHX2n6vVuZzYJ3t2ZJuA54Dxtj+qZd6dFEdcHFDz8lqkkZSnWl+le0P+rrtEZ0ub+ARnW08MBR4S1L9r/VBwBBJI8r3n7Tc9zEwV9IwYFxJW9NSZg3VcZwA5wAbegveLdb1XNjeKmkvMPKAWhIR+0kAj+hsPfPYNwLf9pK/vY11Afi7l7SsxYk4CPnBiehsXwG7gdG2v+vla28pN7HlvvOBn23vKJ8BMKmlzCTgy3L9GTAu/+cd0T4J4BEdzPZfQDfQLelOSWdIGi9pqqTHakXPljRf0umSbgHuBp4qn/E98CbwvKQrJZ0paSFwFvBEuX8Z1fnkKyVdJulUSZdKurldbY0YaDKEHtHhbD8s6RfgLqqgvItqOP3lWrFngVHAWuAfYBH7VqAD3E4VrF8FhgHrgcm2N5Zn7JR0MfA48DpwDLAFePRQtStioMsq9IgBrqxCX2L7kf6uS0QcuAyhR0RENFACeERERANlCD0iIqKB8gYeERHRQAngERERDZQAHhER0UAJ4BEREQ2UAB4REdFA/wLmvUELiS1nawAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "sg.utils.plot_history(dgi_history)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Node classification\n", + "\n", + "We've now initialised the weights of the model to capture useful properties of the graph structure and node structure. We can now further train the model to perform a node classification prediction task. To emphasise the value of the unsupervised weights, we will use a very small amount of labelled data for training.\n", + "\n", + "> See [the GCN node classification demo](gcn-node-classification.ipynb) for more details on this task.\n", + "\n", + "### Data preparation\n", + "\n", + "The Cora dataset labels academic papers into one of 7 subjects:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
subject
Neural_Networks818
Probabilistic_Methods426
Genetic_Algorithms418
Theory351
Case_Based298
Reinforcement_Learning217
Rule_Learning180
\n", + "
" + ], + "text/plain": [ + " subject\n", + "Neural_Networks 818\n", + "Probabilistic_Methods 426\n", + "Genetic_Algorithms 418\n", + "Theory 351\n", + "Case_Based 298\n", + "Reinforcement_Learning 217\n", + "Rule_Learning 180" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "node_classes.value_counts().to_frame()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To simulate a data-poor environment, we will split the data into a train set of size 8, along with test and validation sets." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "train_classes, test_classes = model_selection.train_test_split(\n", + " node_classes, train_size=8, stratify=node_classes, random_state=1\n", + ")\n", + "val_classes, test_classes = model_selection.train_test_split(\n", + " test_classes, train_size=500, stratify=test_classes\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The train set has only one or two observations of each class." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
subject
Neural_Networks2
Probabilistic_Methods1
Rule_Learning1
Reinforcement_Learning1
Genetic_Algorithms1
Theory1
Case_Based1
\n", + "
" + ], + "text/plain": [ + " subject\n", + "Neural_Networks 2\n", + "Probabilistic_Methods 1\n", + "Rule_Learning 1\n", + "Reinforcement_Learning 1\n", + "Genetic_Algorithms 1\n", + "Theory 1\n", + "Case_Based 1" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_classes.value_counts().to_frame()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For a categorical task, the categories need to be one hot encoded." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "target_encoding = preprocessing.LabelBinarizer()\n", + "\n", + "train_targets = target_encoding.fit_transform(train_classes)\n", + "val_targets = target_encoding.transform(val_classes)\n", + "test_targets = target_encoding.transform(test_classes)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "train_gen = fullbatch_generator.flow(train_classes.index, train_targets)\n", + "test_gen = fullbatch_generator.flow(test_classes.index, test_targets)\n", + "val_gen = fullbatch_generator.flow(val_classes.index, val_targets)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Fine-tuning model\n", + "\n", + "We now have the required pieces to finalise our GCN model for node classification:\n", + "\n", + "- a GCN model with weights pre-trained with Deep Graph Infomax to capture the graph structure\n", + "- a small train set\n", + "\n", + "We use the same GCN model as before but train it for a supervised categorical prediction task. See [the fully-supervised GCN node classification](gcn-node-classification.ipynb) demo for more details." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "pretrained_x_in, pretrained_x_out = pretrained_gcn_model.in_out_tensors()\n", + "\n", + "pretrained_predictions = tf.keras.layers.Dense(\n", + " units=train_targets.shape[1], activation=\"softmax\"\n", + ")(pretrained_x_out)" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "pretrained_model = Model(inputs=pretrained_x_in, outputs=pretrained_predictions)\n", + "pretrained_model.compile(\n", + " optimizer=optimizers.Adam(lr=0.01), loss=\"categorical_crossentropy\", metrics=[\"acc\"],\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [], + "source": [ + "prediction_es = callbacks.EarlyStopping(\n", + " monitor=\"val_acc\", patience=50, restore_best_weights=True\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [], + "source": [ + "pretrained_history = pretrained_model.fit(\n", + " train_gen,\n", + " epochs=epochs,\n", + " verbose=0,\n", + " validation_data=val_gen,\n", + " callbacks=[prediction_es],\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAI4CAYAAACV/7uiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXycVdn/8c/Jvm/N0qZpurd2oTRpKYssioCISH1cAB8QUISigCj4e1RQHlBReVCkKqgggkUQAVFRURREEBSknS7YjTalSdMmTdJO9j1zfn/cM9maZSaZzJJ836/XvDLLnXuuTKdzzTnnuq/bWGsRERGR6BIT7gBEREQkcErgIiIiUUgJXEREJAopgYuIiEQhJXAREZEoFBfuAMYrNzfXzpkzJ9xhiIiITIhNmzbVWWvzBt8f9Ql8zpw5bNy4MdxhiIiITAhjTPlQ92sKXUREJAopgYuIiEQhJXAREZEopAQuIiIShZTARUREopASuIiISBRSAhcREYlCIUvgxpifGWNqjDH/GeZxY4z5vjFmrzFmmzGmNFSxiYiIRJtQjsAfBs4d4fH3AQu9l6uBH4UgJhERkagUsk5s1tqXjTFzRthkLbDBWmuB14wxWcaYGdbaqpAEOMVUHGnlyp+/QVtXz5CPn7tsOl85f6nf+3tm6yHuem4X1gYWR25aIo9ddSIpCeN/Kz706ts8+Mrb496P+G95YSY//vgqv7d/Y/9R/uepbXT1eCYwqtHNz0vj4U+cgDHGr+3/c7CBz/5yM51hjluiw10fOZ6T50+b8OeJpFaqM4ED/W5Xeu87JoEbY67GGaVTXFwckuAmm00VR9lT08z7lk8nOSF2wGM7DjXy+BsH+PJ5S4iN8e8D7mlXJa0dPZyx+Jh2vcNyt3Ty4u5aXOX1nLowN6D4h/KrN5y3z5q5OePel4yu/Egrf95eTU1jO/kZSX79zh+3VXGovo33r5gxwdEN76C7jZfeqmX/kVbm5qb69Tt/+k8V5UdbWbuycIKjk8kgKyU+JM8TSQncb9ba+4H7AVavXh3gmE8ADtW3A3D3hSuPSeBPuyq58Ymt7Klp4h3TM0bdl8dj2VxRz7nLpnPnR1b4HUNjexfH3/4XXBXucSfw5o5u3jrcxPVnLuTzZy8a177EP64KNx+675+4Ktycu9y/hLy5ws3KWVncfeHKCY5ueLurm3jvPS/jKnf7ncBd5fUsnZER1rhFBoukKvSDwKx+t4u898kEqG5oJysl/pjkDVBanA04H1r+2FfXQkNbF6WzswKKISMpnoX5abgq3AH93lC2HqjHY6F0dva49yX+WVaYQUJsDK4K/94n7V09bD/UGPZ/o4X5aaQnxvn9vuvu8bC1sp7S4sDe3yITLZIS+DPAZd5q9JOABq1/T5yqhjamDzPtOXtaCjmpCWwq9+8DzuXdbtUYPphXzc7GVe7G4xnfRMqmcjfGwMpZ+pANlcS4WJbPzPD7fbKtsoFuj+39ghguMTGGlcVZfse9q7qJ1s6esH/xEBkslIeR/RL4F7DYGFNpjLnSGHONMeYa7ybPAvuAvcADwGdCFdtUVNXQTmFW8pCPGWMoLc5ms58jFFeFm4ykOOblpgUcR0lxNo3t3eyraw74dwfHsDA/jczk0Kw9iWPV7GzePNhAZ/foxV2+EW8kjGRLi7N563ATzR3do267uTduJXCJLCFL4Nbaj1lrZ1hr4621RdbaB621P7bW/tj7uLXWXmutnW+tPc5aq5N8T6CqhnZmZA5feFQ6O4t9dS0cbekcdV+uCjclxdnE+FnwNuB5ApyuH4pvDV4fsKFXWpxNZ7eH7YcaRt3WVe5mzrQUpqUlhiCykZXOzsZjnaWX0bgq6slLT6Qoe+gvvCLhEklT6BIi7V09HG3pHDmBe5PhaKPwhrYu9tQ0jzl5zstNJTM53u/pzKHsq2t21uCVwEPON6082jq4tRZXhTti/o18Sy3+vO+cuLP8PuRMJFSUwKeg6ganAn1G5vAjiuOLsoiNMaMW+mw5UI+1Y1v/Bmc9srQ4a1yFbL7Ru9YoQ68gI4mZWcm9dRDDOXC0jbrmzoj5N8pMjmdRwegFlHXNHZQfaR3z+1tkIimBT0GHGtoARhyBJyfEsnRGxqhT2y5v8djxszLHHE9pcTZ7apxR9Fi4KtxkJsczz89DgiS4Smdnj5oIXRG4juzUedSPWEDp+2ISSXGL+CiBT0FV3mPAZwxTxOZTWpzF1sp6ukfoPuWqcLO4IJ30pLEXj/lGZVv8WI8cLoaS4qwxrcHL+JUWZ1HV0E6V94vhUFwVblITYlk8PT2EkY2stDibhrYu9tW1DLuNq6Ke+FjD8plj/4IqMlGUwKeg6kbfFPrI3bNKZ2fT2tnDruqmIR/3eCxbKurHPS16/KwsYox/65GDNbR18dbhZlZphBQ2/hQibip3c/ysLL87+4WCr2/BSNP/rgo3SwszSYo/tl+CSLgpgU9Bh+rbyE6JH/VDabRCtj01zTR1dI97ejEtMY7F0zP8PmytP9+oPVLWVqeipYUZJMXHDPsFrLWzm13VTRG3jjwv1znscLjp/64eD9sq6/XlUCKWEvgUVN3QPmIBm09RdjJ56YnDVhgH87je0uIstlTU0xNgQxdXuZsY44ziJTziY2NYMXP4QsStBxroiYAGLoPFxBhKRiig3FnVSHuXJ+AOgyKhogQ+BR1qaKcwa/STTzgNXYbvWOUqd5OdEu93P+mRlBZn09TRzZ6aoafrh+OqcLOoIJ20xKhs6z9plMzOYvuhBtqHOLudL0GWREADl8Gchi5DF1CqgE0inRL4FFTV0Mb0Uda/fUqLs6k42kpdc8cxj/mO6w3G8bG+6dVAGrr41uAjbWp2Kiotzqarxw7Z0MVV7mZeXipZKQlhiGxkvuQ8VAGlq6Ke6RlJw3YsFAk3JfAppq2zh/rWLr+m0KF/Yh04Cq9v7aSstiVoa8++/uuBHA8erDV4GT/fv8Hg2RprLZsPRO468vGzMokxQxeybSp368uhRDQl8Cmmyo9jwPtbPjOT+FhzzDr4Zu/tYE2L+qbrA0ngvWvw+pANu7z0RIpzUo6ZQdl/pJWjLZHTwGWw9KR4FhWkH/O+q2ls52B9W0RO+4v4KIFPMVV+dGHrLyk+lqWFmceMUFwVbmJjDMcXBe8DrnR2NvtqW3D70X8dnBFSTmoCc6alBC0GGTvfFzBr+woRN0XBOnLp7Gy2DGrooi+HEg2UwKcYXwL3p4jNp7Q4i20H6+nq19DFVeHmHdPTSQ1i8VjvYWsH/D8LmnpUR47S2dnUNHVwsL6voYurwk16YhwL8wM/U12o9BVQ9p0Rz1VRT0JsDMsKM8IYmcjIlMCnmCrvh2vBMOcCH8qq2dm0d3nYWdUIQI+vgUuQR1UrijKd/ut+FLK5WzrZV9tCSQSP7Kaa3oYu/ZZbXOVuVkZ4l7zeOo9+0+iucjfHFWWSGKcGLhK5lMCnmEMN7UxLTQios9TgAqXd1U20dPYE/fjYlIQ4lsxI96sjm2+UHslTs1PNO6ank5IQ27vc0tTexe7DTRH/bzTHW0Dpe991dnvYdrAhIs5bLjISJfAppjqAQ8h8CrOSmZ6R1Duy8o1UVhXnBD2+VcXZo/ZfB+dws9gYM66TqEhwxcXGsKIos/f9sfVAA9ZG/jqyMYaSWX0FlNsPNdDZ7Yn4Lx4iSuBTTJWfXdgGK52d1TuyclW4yU1LYFZO8I+P9fVf33145IYurgo3S2akk5KgBi6RpLQ4mx2HGmnv6ulNiCujoEuer4CyvrWz94tqpH/xEFECn2Kq/OzCNlhpcTYH69s43NjO5op6SoLUwGWo5wGGbd8K0N3jYeuB4K/By/itmp1Nt8eyrbLB2yXP6Tce6fr6/tfjqnAzMys5oDoRkXBQAp9CWju7aWjrCngKHfpGI8/vPMzbdS0T1uCit//6COvguw87a/BqshF5fEWFG8uP4ip3R82XrONnOQWUm8rdTtx6b0kUUAKfQg55zwNeOIYp9GWFGSTExvCzV94GJq54zJ+GLr1TnFGSHKaSnNQE5uam8utNlTS2R0+XvJSEON4xPZ0//aeKqoZ2FbBJVFACn0KqG/w7D/hQEuNiWT4zg7LaFuJiDCuKJq54rLQ4m/IjQ/dfB9hc7iY3LZGibPWojkQlxVmU1bYARNWZvEqLs/vijpIvHjK1KYFPIYd626iOLfH5pqyd8z9P3PGxpcP0X/fZpAYuEc33PslMjmdebuQ2cBnMF3dSfAxL1cBFooBKeKPIH7dVUXG0dcjHZmQm8cGSmSP+fpV3Cr0gM3FMz++MSt6e8NHJcd7+64+8Vt47IvLp7vFQfqSV/15TPKExyNj53h8lEd7AZTBf3CtmZhEfq7GNRD4l8ChR29TBtY+5Rtxm1exsZuUM3xe8urGN3LSEMXeXWjM3h9y0RM5eWjCm3/dXUnwspy7I5cXdtfxjT90xj8fHGk5bmDehMcjYLSpIZ35e6oS/T4JtVk4yy2dmcM6y6Ipbpi4l8CjhK+r65VUnHXOGpF3VTXzw3ldxVbhHTOCH6sd2DLjPtLRENn7lrDH/fiAevPwEOodp5hJjDAlxGiFFqtgYwws3vSvcYQTMGMMfrj8t3GGI+E2fglHCVeEmPtZQUpxFUnzsgMvywowBLSyHU93QPqYCtnCIiTHH/J2+i5K3iIgSeNTYXF7PssLMIYvH4mJjOL4oa8TmJ+AUsUVLAhcRkZEpgUeBzm4PWyvrR2xcsmp2NjuqGmnt7B7y8eaObprau5mRpUOvREQmAyXwKLCzqpGOUU6uUDo7ix5vC8uhVPceQqYRuIjIZKAEHgV8BWwjNcUomXXsOY3783VhG08Rm4iIRA4l8CjgqqhnRmbSiMk3OzWBebmpwxayVWkELiIyqSiBRwF/T65QOjsbV0U91tpjHqtqaMcYdIYlEZFJQgk8wh1ubOdgfZtf3c9Ki7M52tJJ+ZFju7VV1beTm5aoQ7BERCYJfZpHON+UuD9nR/KtkW8aYhq9qrGdQk2fi4hMGkrgEW5TuZuEuBiWFY5+9q+F+emkJcYNWchWVd82pvOAi4hIZFICj3CuCjcrZmb6NfUdG+N0ahuqoUtVw/jaqIqISGRRAo9gHd09/Odgo18FbD4lxdnsrm6kuaOvoUtTexfNHd2qQBcRmUSUwCPY9kONdPZ4/Fr/9iktzsJjYeuBvlF4VYP3GHB1YRMRmTRCmsCNMecaY3YbY/YaY740xOPFxpgXjTGbjTHbjDHnhTK+SNNXwBbACNzX0KVfIduheucYcBWxiYhMHiFL4MaYWOBe4H3AUuBjxpilgzb7CvCEtbYEuBi4L1TxRSJXhZui7GTyAzh2OzMlnoX5aQMK2aq9I3AVsYmITB6hHIGvAfZaa/dZazuBx4G1g7axQIb3eiZwKITxRRRrLZvK3QGNvn1Ki52GLh6P09DlkJq4iIhMOqFM4DOBA/1uV3rv6+824FJjTCXwLHD9UDsyxlxtjNlojNlYW1s7EbGG3aGGdg43dgS0/u1TOjuLhrYu9tW1AM6JTPLTE4mPVcmDiMhkEWmf6B8DHrbWFgHnAY8YY46J0Vp7v7V2tbV2dV5eXsiDDIXe9e8AKtB9fKN23zR6VUM703UImYjIpBLKBH4QmNXvdpH3vv6uBJ4AsNb+C0gCckMSXYRxVbhJio9hyYyM0TceZH5eGhlJcb1fAg7Vt6mATURkkgllAn8DWGiMmWuMScApUntm0DYVwHsAjDFLcBL45JwjH4Wrop4VRVljmvaOiTGUFGfjqnBjrfWOwJXARUQmk5AlcGttN3Ad8BywE6fafLsx5mvGmAu8m90EXGWM2Qr8ErjCDnVqrUmuvauH7QcbxlTA5lNanM2emmYO1rfR2tlDoabQRUQmlbhQPpm19lmc4rT+993a7/oO4J2hjCkSvXmwgW6PHVMBm0/p7CyshT+9WQ3AjCyNwEVEJpNIK2ITxlfA5rNyVhbGwB/erAJQG1URkUlGCTwCuSrczJ6WQm5a4pj3kZ4Uz+KC9N6WqjqRiYjI5KIEHmGcBi7141r/9inx7iPGQH762L8MiIhI5FECjzCV7jbqmsfWwGUw3z7y05OIUxMXEZFJRZ/qEcbXfGU8698+q7z7UAGbiMjkowQeYTaVu0lJiGVxQfq49zU3N5VpqQnM1GlERUQmnZAeRiajc1W4Ob4oKyhT3sYYHrziBHJSEoIQmYiIRBKNwCNIa2c3O6uaeqe+g2HlrCyKp6UEbX8iIhIZlMAjyLbKBno8ltLZ4y9gExGRyU0JPIJs8jZwKZkVvBG4iIhMTkrgEWRzhZt5ualkp2rNWkRERqYEHiGstbgq6oNy+JiIiEx+SuARovxIK0dbOoPSgU1ERCY/JfAIsan3BCYqYBMRkdEpgUcIV4WbtMQ4FuaPv4GLiIhMfkrgEcJVUU9JcRaxMSbcoYiISBRQAo8AzR3d7K5u7D17mIiIyGiUwCPA1gP1eCxBOQOZiIhMDUrgEcClBi4iIhIgJfAI4KpwszA/jcyU+HCHIiIiUUIJPMw8Hm8DF61/i4hIAJTAw2xfXQsNbV06/ltERAKiBB5mrgpn/TuYpxAVEZHJTwk8zDZXuMlIimNeblq4QxERkSiiBB5mm8rdlBRnE6MGLiIiEgAl8DBqbO9iT02zCthERCRgSuBhtKWiHmu1/i0iIoFTAg8jV4UbY+D4WZnhDkVERKKMEngYbSp3s7ggnfQkNXAREZHAKIGHicdj2XKgXicwERGRMVECD5O9tc00tXdr/VtERMZECTxMfCcw0RnIRERkLJTAw2RTuZvslHjm5qaGOxQREYlCSuBh4qpwGrgYowYuIiISOCXwMKhv7aSstkXr3yIiMmZK4GGwuaIegBKtf4uIyBgpgYeBq8JNjIHji5TARURkbJTAw8BV4WbJjAxSE+PCHYqIiESpkCZwY8y5xpjdxpi9xpgvDbPNhcaYHcaY7caYx0IZXyj0eCxbKup1AhMRERmXkA0BjTGxwL3A2UAl8IYx5hlr7Y5+2ywEvgy801rrNsbkhyq+UNld3URLZw+lszV9LiIiYxfKEfgaYK+1dp+1thN4HFg7aJurgHuttW4Aa21NCOMLCVeFr4GLRuAiIjJ2oUzgM4ED/W5Xeu/rbxGwyBjzqjHmNWPMuUPtyBhztTFmozFmY21t7QSFOzFcFW5y0xIozkkJdygiIhLFIq2ILQ5YCLwL+BjwgDHmmLlma+391trV1trVeXl5IQ5xfDZX1KuBi4iIjFsoE/hBYFa/20Xe+/qrBJ6x1nZZa98G3sJJ6JPCkeYO3q5r0fS5iIiMWygT+BvAQmPMXGNMAnAx8MygbX6LM/rGGJOLM6W+L4QxTihfA5eo7MBmLVRtg398F5oOhzsaEZEpL2RV6NbabmPMdcBzQCzwM2vtdmPM14CN1tpnvI+dY4zZAfQA/89aeyRUMU40V4WbuBjDiqLMcIfiv/oKePNJ2PYE1O5y7tv9J7jiWYhLCG9sIiJTWEg7iVhrnwWeHXTfrf2uW+BG72XScVW4WVqYQVJ8bLhDGVl7I/zn107Srvinc9+sk+D9d0NsAjxzHfzlFjjvrvDGKSIyhakVWIh093jYeqCBi06YNfrG4dTmhgffC3W7IXcRnPkVOO6jkD2nb5vaXfCvH0LRCbDiwrCFKiIylSmBh8iu6ibaunoojeT17+4OePxScL8N//0kLDwbhqqWP+t2OLQZfn8DFCyHgqWhj1VEZIqLtMPIJq2+Bi4R2oHN44HffgbKX4G198Gic4ZO3gCxcfCRn0FiOvzqUmhvCG2sIiKiBB4qm8rd5KcnMjMrOdyhDO1vX4f/PAXvuRVWfHT07dOnw0cfBvd+J/FbO9ERiohIP0rgIeKqcLNqdoQ2cNn4ELxyN6y6Ak4NoH5w9ilwztdh1x/g1fUTFp6IiBxLCTwEaps6OHC0LTIbuLz1F/jjTbDwHDjvu8NPmw/npM/Asv+CF26Ht1+emBhFROQYfhexGWOuAFqttU8Muv9CIMlauyHIsU0avevfkXYGskNb4MkrYPpy+MhDztp2oIyBC34Ah7fDIx9yRuULz4YFZ0Pe4sC/EIiIiF8CGYF/ETg6xP11wJDn9haHq9xNfKxhWWEENXCp2wuPXQgpOfDfT0Bi2tj3lZgOH/8NnHQNNNfAX74C950I9xwHv/8c7PojdLUHL3YREQnoMLI5wN4h7t/nfUyG4apws3xmZuQ0cNn/KvzqEjAxcNnvnIK08cosgnO+4VzqD8Dev8Ke550ubpsegpRpzhr76ishc/BJ6EREJFCBjMAbgLlD3D8faA5OOJNPZ7eHbZUNkbP+vfVXsGEtpObBp16A/CXBf46sWbD6k/Cxx+B/3oZLfw3FJ8Mr33NG5U9cDuX/UuW6iMg4BDIC/xNwlzHmA9baKgBjTCFwJ4Pao0qfnVWNdHR7wp/ArYW/fxte+jbMOQ0uegSSQxBTXAIsOMu5uPfDGz8F1wbY8VuYvgJO+jSsuAhiImR2QkQkSgQyAv8fIBUoM8ZsNMZsxJlST/U+JkPYVB4BBWzdHfCbdU7yXnkJXPp0aJL3YNlznCn2G3fC+fdATxf89tPw49Ng30uhj0dEJIr5ncCttbVACXA98Lr3ch1Qaq2tmZjwop+rwk1hZhIzMsPUwKX1KGz4IGz7ldPXfO294T+LWEIqrP4EfOZf8NGfQ2cTbLgAHr8EjpSFNzYRkSgR0HFD1tp24EHvRfywuaKeknD0P+/ugM2PwMvfhdYj8OEH4biPhD6OkRgDyz4Ii86F1+6Ff9wN957oTKuf/gVIiqCqfRGRCOP3CNwY8yVjzJVD3H+lMUZT6EOobmjnYH2IG7h0dzjrzN8vcRq0ZM2CTzwbecm7v/gkOO0muH4THH8R/PMH8P1S2Pn7cEcmIhKxAhmBXw1cNsT9O4FHgP8LSkSTiK+By6pQjMB9I+5/3A2NB2HWibD2hzDv3dHTTCV9ujPFf8JV8IfPwVOfhMv/AMUnhjsyEQmltnpoqATbE7x9dnc6p0secDnqPFdsAiRnOX0xkrP7XfrdTkiNuM/SQBJ4IVA5xP2HAB3YOwRXuZvEuBiWzsgY/8662p1uZ01Vg9583uuVG53EXbQm+hL3YIUrnUK7n77HOV79qhedmQQRiQ7WQkdT32dUR9Pw27XUOrUvR8vg6D7nettQPcMmQFKWk7h7upx6oe624beNTRiY3ONHqGs68yswc1Xw4x0kkAReAxwH7B90/wrgSLACmkzePNjAssIMEuICbDnv8Thv5oObnMR8cCNU/wc8XQO3i4nr+5aYvyT6E3d/KTnwscfhp2fB4x+DTz7nfAMWkbHxeKCjceAX/9gEyJkP6TMgJoDPqc6WvmR7dJ83+e6Hlpq+fXu6A4svYybkzIMlH4Bp8yGr2IkvWGLinc9L3yg7KfPYw1e72pwRuW+A1HoU2r23W48OHL0P96UEwBPEmYMRBJLAnwa+Z4yptNZuBjDGlALfBZ6aiOCi3cH6NlYHOn3+5lPwxxv7zrGdkAaFJXDKdVBY6rypfW/AhLTJkayHk7fYKb577ELnlKUffXhy/70iwWItVG2FbU/A3uedUW57PVjP0NvHJUPOXCeB5syD7NnOstzgKefWo9BUDc3VA38/Nd/5vfylfSPU3unoHKdVsxnmC0JyjvPcI41oQyU+2blkzAh3JH4JJIHfAqwENhljfPMbOcA/gJuDHVi083gshxvbmRHI+b+thZf+D9IKnOOlZ652kthUbnKy6Bw4+2vw16/Cy3fBGaqXnHSsdUZxBzc5CSJQiWnexDMf0vIn35e8jmZwv+2MdtvrIWu28/dmFh372eDe77Qv3vYE1L3ljDrnvxvmnjb0um53W79R9D6o2wN7/gI9nc7+TIx3mtm7fVo+FCxznn/a/L6En5ge8pdFAkjg1toW4F3GmDMB3+T+Jmvt3yYksihX19JBV4+lMDPJ/1+q2Ql1u+G870DpUPWCU9Qp1zvr/y/eAXnvgKUXhDsiGY/Wo31LQ5UbncTdXh+cfSekeUeS8/slGe91f5K7x+NsM9FfAqyFzuaBI9v+1+v3wxFvUh082vWJTYBs76g5ew4c2gwHXnMem/1O51S/S9c6I+GRzHvXwNueHmg+7IxEEzMDm1qXkAroOHBjTDZQAMQCCcCpxphTAay1Xwt+eNGrqt45+9b0QBq47PgtYGCJEtQAxsAH1sORvU5HuZy5MP248MVTtRV2/xnyFjmFKpmzRv/Atxa62yNjmjAUOluc0Vzv+ujbfYVKLbXejYxTu7HkA1C02plxyigM/Lna3M7+fc91pAyq34Rdfxi4DpuQ3jdNnJrnLFO1DUqc7Q0QGz/0aPWYKuV+1+NTho7N0wXu8oGx+Ua7HY3D/01pBU6cC85yYvaNdpOzvfsbtK99f3emvd9zKxz3UWepbaxiYsf27yAhF8j5wE8A/gwYIAOoBfKBVqAKUALvp6rBSeAzAhmBb/+t8805vWCCoopi8Ulw8aNw/7vhsYth+YeGWGvLdj64JqoBTNU2eOlOJzH0l5rvTUCrnJ/xKf0KfHyVtfugowEWvQ/e9UWnrmEysRZqdsCevzprrhX/Gpg806Y7SWjRe2HaQphZ6rwGwZh6Tclx9j1YTxfUV3iTe7+EV73NSdbJWX1JOHtu33uop5MBhxjVl0PVltGrlEdjYpz3Z858KDrBObKi/5eA3uKqLOf9PpysYmdKvD9rJ9/SgYwqkBH4XcCvgWtwzkz2TqATeAz4XvBDi25VDc5/9EJ/18B90+drrprAqKJc+nTnDGe//hT8+35nRDuYiXEOpVt4Fiw42zlhyninAPsn7sRMeNeXndOiNhwYeKTA7kHn9DExzug8Zx6s+Kgz+nY9Ave/a3Ik8tajUP5qX9JuPOjcX7AcTr7O+dumzXeS43jONz9WsfHO80+bD5wVnH32VikPGrl3dwy9fYzvPeCtqp6oNsZK3lOSsX6e0tEYUw+cbK3daYxpAE601u4yxpwM/MxaOwHnpRzd6tWr7caNG8Px1CP65rM7+fk/97Pr6+di/PnP9eK3nCRx026NwP3V1TZo/fCoc7jdnr84IyZwpiIXnAXzz1BGPS0AACAASURBVIS4xEGHgnivd3c4o57B06PxybD1l32J++TPwInXOCO3obQehUMu6OnuOwwmLnHgNu2N8O+fwD9/6Kz7Lj4Pzviic+x7JOvudKalD27s+9Jy1Nu3PjHDWUddeLbzWmv6VSSojDGbrLWrB98fyAi8B/AdiFwDzAJ2AXXA7HFHOMlUNbQzIzPJv+QNsP03mj4PVO8hH/0SxtK1cOYt0FzjjAr3/BV2/RG2PDrwd3ubMuQ4o6K6t6DV7Uxz9+cbcY+UuH1ScpwENpKkDDj9/8GadX2J/P4znOn3tALvlGq/qt+UaTD39NELkYLN0+MURe35K5T9zflC5KtMTpvuLBWUXOp0/Ju1xhntikhIBZLAt+EcRrYXeA242RgTA1wF7J6A2KJaVX0b0/1d/9b0efCl5cPK/3YuPd1w+D/OlLZvlB2fMvS0Y0+3t7jJ7YyQcxdOzJp6byK/2lkO2PeSs15btdV57q7Wvm0T0uGka5yq4olM5C1HoOwFb9J+wTkJDsb5cnHiOqfIrGi103BDU7YiYRdIAr8D8C1kfRX4I/AnnGK2CD5TRnhUNbRz4lw/P2y3q/p8QsXG+T9FHRsHqdOcSygkZTqJ/PT/N/D+rnYnkddXOGdqe/kueO3H40/kHg80Vg6sYPYV3NW9BVhn1L/AW0Mw/8zQvRYiEpBAjgN/vt/1/cAyY0wO4Lb+LqRPET29TVz8HIHv+C3MPkXT59InPgniZzgdoYpPdI6Df+lOJ5G//hNnSv+kT4+cyK11GnsMaMn75sDiv7gkp8gudyEs/7BT/DejRMf+ikSBgI4DH8xaG6KO89GlrrmDbo/17xjwml1Qu8tp3iIynIJlcOEGp0jvpTvh5f9zLomZfWvm/Q9Daqj0djarc34/LtmZhVh9pXP8uq9zWaA9sEUkYowrgcvQDtV7DyHzZw1czVskENOXw0WPOIl897POOnX/Snx3uVNdn1YAi86FolXOGnb+UhWaiUwySuAToLq3iYsfI/Dtv9H0uQRu+nLnIiJTlubOJsAhf7uw+abPl34wBFGJiMhkogQ+Aaob2kiKjyErZZQpS9/0uU7OISIiAVICnwCHGtopzEwevYnL9t9C8clOi1AREZEAKIFPAL+auNTsgtqdsOy/QhOUiIhMKkrgE6C6oX30AjZNn4uIyDgogQdZj8dyuKlj9AI2TZ+LiMg4hDSBG2PONcbsNsbsNcZ8aYTtPmyMscaYY86+Eulqmtrp8diRu7DV7PROn6v6XERExiZkCdwYEwvcC7wPWAp8zBizdIjt0oEbgNdDFVswVXkPISscaQp9y6MQEwfLPhSiqEREZLIJ5Qh8DbDXWrvPWtsJPA6sHWK7rwN3Au1DPBbxquqdsIctYuvphq2/goXvhbS8EEYmIiKTSSgT+EzgQL/bld77ehljSoFZ1to/jrQjY8zVxpiNxpiNtbW1wY90HKoafG1UhxmB730eWmqg5JIQRiUiIpNNxBSxec8tfjdw02jbWmvvt9auttauzsuLrFFsVUM7KQmxZCQP06V2yy8gJRcWnhPawEREZFIJZQI/CMzqd7vIe59POrAc+LsxZj9wEvBMtBWyVTU4x4AP2cSl5Qjs/jOsuEgnlhARkXEJZQJ/A1hojJlrjEkALgae8T1orW2w1uZaa+dYa+cArwEXWGs3hjDGcavydmEb0ptPgqdL0+ciIjJuIUvg1tpu4DrgOWAn8IS1drsx5mvGmEnTzaSqvn34Y8C3/AJmrHTO7SwiIjIOIT2dqLX2WeDZQffdOsy27wpFTMHU3eOhpmmYBF61DarfhPO+E/rARERk0omYIrbJoKapA4+FGVlDTKFveRRiE2D5h0MfmIiITDpK4EHkO4TsmGPAuzth2xOw+DxIyQlDZCIiMtkogQfRofphurC99WdoOwoll4YhKhERmYyUwIOo2ttG9Zg+6FsehfQZMP/MMEQlIiKTkRJ4EB1qaCM1IZb0xH61gU2HYc9f4fiLISY2fMGJiMikogQeRFX17czISh7YxGXbr8D2wEod+y0iIsGjBB5EVY2DDiGz1pk+L1oDuQvDF5iIiEw6SuBBVFXfNjCBH3RB7S51XhMRkaBTAg+Srh4Ptc0dzOhfgb7lUYhL1nm/RUQk6JTAg+RwYzvWQqGvAt1a2PVHWPReSMoIb3AiIjLpKIEHSZX3ELLpvhF4zU5oroYFZ4UxKhERmayUwIPEl8ALfWvgZS84P+e/O0wRiYjIZKYEHiRV9YPaqJb9DXIXQ2ZRGKMSEZHJSgk8SKoa2klPjCM9KR662qD8n7DgPeEOS0REJikl8CCpamjra6Fa8S/oblfrVBERmTBK4EFS1dDeV8BW9jfn1KGzTwlvUCIiMmkpgQfJofr2vgK2vX+D4pMgITW8QYmIyKSlBB4End0e6nxNXJqqoWa7ps9FRGRCKYEHweFG72lEM5Og7EXnTiVwERGZQErgQXDIewjZjKwkZ/07NQ8KjgtzVCIiMpkpgQdBtW8EnpEI+16Eee+GGL20IiIycZRlguBQvZPAZ3aUQUutps9FRGTCKYEHQXVDGxlJcSRX/N25Q+1TRURkgimBB8GhhnanAr3sb5C/DNKnhzskERGZ5JTAg6CqoY3ZGRYqXtPoW0REQkIJPAiqG9o5OXY39HSq/7mIiISEEvg4dXT3UNfcycpOF8QlQfHJ4Q5JRESmACXwcar2ngd8fuPrTu/z+OQwRyQiIlOBEvg4VTW0M4MjZDTv0+FjIiISMkrg4/R2XQunxr7p3FACFxGREFECHydXuZuzEv6DTZsO+UvDHY6IiEwRSuDjtLm8jneaNzHzzwRjwh2OiIhMEUrg41Df2knykR2keZo0fS4iIiGlBD4OmyvqKYnZ49yYfUp4gxERkSlFCXwcXBVulsRUYpOyIKMw3OGIiMgUogQ+DpvK3RyfeAiTv1Tr3yIiElJK4GPU47FsPeBmnqcCClR9LiIioaUEPka7q5vI6jxMkqdFh4+JiEjIhTSBG2PONcbsNsbsNcZ8aYjHbzTG7DDGbDPGvGCMmR3K+ALhqnCzOOaAc6NgWXiDERGRKSdkCdwYEwvcC7wPWAp8zBgzeOi6GVhtrV0BPAX8X6jiC5Sr3E1JUpVzI+8d4Q1GRESmnLgQPtcaYK+1dh+AMeZxYC2ww7eBtfbFftu/BlwawvgC4qpwc1FKNcQUQXJWuMMRkUnI4/FQWVlJS0tLuEORCZKamkpRURExMYGPp0OZwGcCB/rdrgROHGH7K4E/DfWAMeZq4GqA4uLiYMXntyPNHew/0sqCaeUqYBORCVNXV4cxhsWLF4/pA14im8fj4eDBg9TV1ZGfnx/w70fkO8IYcymwGrhrqMettfdba1dba1fn5eWFNjicBi5xdJPdVq4CNhGZMPX19RQUFCh5T1IxMTEUFBTQ0NAwpt8P5Qj8IDCr3+0i730DGGPOAm4BzrDWdoQotoBsqnCzMLaaGE+XEriITJienh7i4+PDHYZMoPj4eLq7u8f0u6H8WvcGsNAYM9cYkwBcDDzTfwNjTAnwE+ACa21NCGMLiKvczXty6pwbmkIXkQlk1CRqUhvPv2/IEri1thu4DngO2Ak8Ya3dboz5mjHmAu9mdwFpwJPGmC3GmGeG2V3YdPV42FbZwJqUajCxkLso3CGJiMgUFMopdKy1zwLPDrrv1n7XzwplPGOxq6qJtq4eFppKyF0IcYnhDklERKYgVUYEyFXhBiCvdS/kLwlzNCIik9/DDz9MXFxIx5tRQQk8QK4KN3PTPcQ1VkC+OrCJiAzlrLPO4oorrgjKvi666CIOHjym5nnKUwIPkKvCzfsK6p0bKmATERmzzs5Ov7ZLTk6moKBggqOJPkrgAahpaufA0TZOSjvs3KEpdBGRY1xxxRW88MIL/PznP8cYgzGGhx9+GGMMjz76KOeddx6pqal89atfxVrLVVddxfz580lOTmbevHncfPPNdHT0HUU8eArdd/vVV1+ltLSUlJQUVq1axRtvvBGOPzdstKgQAFe5M/JebA5AfCpkzQlvQCIypdz+++3sONQY8uddWpjB/37A/yXD9evXs2/fPmbMmMH69esBaGx04v7iF7/InXfeyb333guAtZb8/Hwee+wxCgoK2LZtG+vWrSM+Pp7bb7992OfweDx8+ctfZv369eTl5fH5z3+eCy+8kD179kyZ9fKp8VcGyeYKNwmxMeS1lkH+O0DdkUREjpGZmUlCQgLJyclMnz4dgPb2dgDWrVvHJZdcMmD7O+64o/f6nDlzKCsr47777hsxgVtrueeeeygtLQXgtttu46STTqKsrIzFixcH+0+KSErgAXBVuFlWmE5M7Q5YfF64wxGRKSaQUXCkWrNmzTH3PfDAA/z0pz9l//79tLS00N3djcfjGXE/xhiOP/743tuFhYUAHD58eMokcA0h/dTZ7WFrZQOnFwKtR9RCVURkDFJTUwfcfvLJJ7n22mu56KKLePbZZ9m8eTO33norXV1dI+4nJiaG2NjY3tu+jmajJf7JRCNwP+2oaqSz28Mp6d4CNlWgi4gMKyEhgZ6enlG3e/nllykpKeHGG2/svW///v0TGNnkoRG4n1zlTgOXJbHeM6LqGHARkWHNnTuXTZs2UVZWRl1d3bAj6sWLF/Pmm2/yu9/9jrKyMtavX8/TTz8d4mijkxJ4P109Hto6e4a8bCp3U5iZREbjHkjNg7TQn8ZURCRa3HTTTeTm5nL88ceTl5fHq6++OuR269at4+Mf/zif+MQnKCkp4fXXX+e2224LbbBRylhrwx3DuKxevdpu3LgxKPv6/gt7uPuvbw37+PkrZvDD5psgMQ0u/31QnlNEZDg7d+5kyRL1m5jsRvt3NsZsstauHny/1sD7eeeCaSTEvWPIxwzw3qX5cP8uKL08tIGJiIgMogTez6rZOayanTP8BkfKoKtVBWwiIhJ2WgMPRM1O56cK2EREJMyUwANRs8P5mTc1mgSIiEjkUgIPxOHtkD3HKWITEREJIyXwQNTs0PS5iIhEBCVwf3W1O0VsOoWoiIhEACVwf9W9BbZHFegiIhIRlMD95Stg0xS6iIhEACVwf9XsgNgEmDY/3JGIiEx6Dz/8MHFxfa1K/v73v2OMobKycsTfM8bwi1/8YtzPf8UVV3DWWWeNez8TSQncX4d3QO4iiI0PdyQiIlPOKaecQlVVVe95v4PlF7/4Re+pSPtbv349Tz75ZFCfK9jUic0fXe1waDPMPzPckYiITEkJCQlMnz49ZM+XmZkZsucaK43A/fH6j6C1DkouDXckIiIR74EHHiAzM5P29vYB9995550UFxfT09PDVVddxfz580lOTmbevHncfPPNdHR0DLvPoabQX3zxRVasWEFSUhIrVqzgxRdfPOb3brnlFpYsWUJKSgqzZs3immuuoaGhoXefH//4xwFn6t0YwxVXXAEcO4VureU73/kO8+bNIyEhgfnz53PPPfcMeK45c+Zw6623csMNN5CTk0NBQQGf//zn6e7uDuwF9JNG4KNpqYN/3A2LzoV5Z4Q7GhGZyv70Jah+M/TPO/04eN+3/d78wgsv5LOf/Sy/+93vuOiii3rv37BhA5deeinGGPLz83nssccoKChg27ZtrFu3jvj4eG6//Xa/nuPQoUOcf/75XHjhhTz++OMcPHiQG2644ZjtkpOTuf/++5k1axZlZWVce+21fPazn+XnP/85p5xyCj/84Q+57rrrqKqq6t1+KPfddx9f/epXWb9+Pe9+97t54YUX+NznPkd6ejpXXnll73Y/+MEP+OIXv8jrr7/O5s2bueSSS1i+fPmAbYJFCXw0f/8WdLbA2V8PdyQiIlEhMzOTtWvXsmHDht4EvnHjRnbs2MHTTz9NTEwMd9xxR+/2c+bMoaysjPvuu8/vBH7fffeRm5vLAw88QFxcHEuXLuWb3/wmH/jABwZs95WvfGXA83zrW9/i4osv5qGHHiIhIaF3qny06flvf/vbXH/99Vx99dUALFy4kN27d3PHHXcMSM6nnXYaX/rSl3q3eeihh3j++eeVwEOudjdsfAhWfxLyFoU7GhGZ6gIYBYfb5ZdfzgUXXEBNTQ35+fls2LCBNWvWsHixcy6JBx54gJ/+9Kfs37+flpYWuru78Xg8fu9/x44drFmzZkCl+qmnnnrMdk8//TT33HMPe/fupbGxEY/HQ2dnJ9XV1X4XxDU2NlJZWcnpp58+4P4zzjiD9evX09raSkpKCgArV64csE1hYSFvv/22339XILQGPpK/fBUSUuFdXwp3JCIiUeWcc84hNzeXxx57jK6uLh5//HEuv/xyAJ588kmuvfZaLrroIp599lk2b97MrbfeSldXV1BjeP311/noRz/K6aefzm9+8xtcLhc//vGPAejs7Azqc/kkJCQMuG2MCeiLSSA0Ah/Ovr/DnufgrNshNTfc0YiIRJXY2FguueQSHnnkEebNm0dDQwMXX3wxAC+//DIlJSXceOONvdvv378/oP0vXbqURx55hJ6eHmJjYwF49dVXB2zzyiuvkJubyze+8Y3e+5566qkB2/gSbv/9DJaRkUFRUREvv/wy559/fu/9L730EnPnzu0dfYeaRuBD8fTAc1+BzGI48ZpwRyMiEpUuu+wyXC4X//u//8v5559PTk4OAIsXL+bNN9/kd7/7HWVlZaxfv56nn346oH1/+tOfpra2lquvvpqdO3fywgsvcMsttwzYZvHixdTW1vLggw+yb98+NmzYwH333Tdgm7lz5wLwzDPPUFtbS3Nz85DP9+Uvf5kf/OAHPPDAA+zZs4ef/OQn/OhHP+Lmm28OKO5gUgIfytZfwuE34az/hfikcEcjIhKVVqxYwcqVK9myZQuXXXZZ7/3r1q3j4x//OJ/4xCcoKSnh9ddf57bbbgto3zNnzuT3v/89//73v1m5ciU33HADd99994Btzj//fG655RZuvvlmjjvuOB5//HHuuuuuAduccMIJ3HDDDaxbt478/Hyuu+66IZ/v05/+NF/72tf45je/ydKlS7nzzjv59re/PSHFaf4y1tqwPXkwrF692m7cuDF4O+xsge+XQmYRfOp5GKJDj4hIKOzcuZMlS3QGxMlutH9nY8wma+3qwfdrDXywV78PzdVw4QYlbxERiViaQu+vsQr++X1Y+kEoPjHc0YiIiAxLCby/f/0QPN1w1m3hjkRERGREmkLv78yvwqL3Qs7ccEciIiIyIo3A+4tPgrmnj76diEiIRHuhsYxsPP++IU3gxphzjTG7jTF7jTHHtDczxiQaY37lffx1Y8ycUMYnIhJJYmNjg96dTCJLV1fXgHawgQhZAjfGxAL3Au8DlgIfM8YsHbTZlYDbWrsA+B5wZ6jiExGJNFlZWRw+fHjCWnFKeHk8Hg4fPjzmc4+Hcg18DbDXWrsPwBjzOLAW2NFvm7XAbd7rTwE/NMYYqzkkEZmCcnNzqaysZPfu3eEORSZIamoqublja9cdygQ+EzjQ73YlMPhYrd5trLXdxpgGYBpQ138jY8zVwNUAxcXFExWviEhYxcTE6DNOhhWVRWzW2vuttauttavz8vLCHY6IiEjIhTKBHwRm9btd5L1vyG2MMXFAJnAkJNGJiIhEkVAm8DeAhcaYucaYBOBi4JlB2zwDXO69/hHgb1r/FhEROVbI1sC9a9rXAc8BscDPrLXbjTFfAzZaa58BHgQeMcbsBY7iJHkREREZJKSd2Ky1zwLPDrrv1n7X24GPhjImERGRaBT1pxM1xtQC5UHcZS6Dqt6nKL0ODr0ODr0Oeg189Do4Qvk6zLbWHlOxHfUJPNiMMRuHOu/qVKPXwaHXwaHXQa+Bj14HRyS8DlF5GJmIiMhUpwQuIiIShZTAj3V/uAOIEHodHHodHHod9Br46HVwhP110Bq4iIhIFNIIXEREJAopgYuIiEQhJXAREZEopAQuIiIShZTARUREopASuIiISBRSAhcREYlCSuAiIiJRKKSnE50Iubm5ds6cOeEOQ0REZEJs2rSpbqizkUV9Ap8zZw4bN24MdxgiIiITwhgz5CmzNYUuIiIShZTARUREopASuIiISBRSAhcREYlCSuAiIiJRSAlcREQkCimBi4iIRCElcBERkSikBD5IU3tXuEMQEREZlRJ4P9/761us+vrzdPd4wh2KiIjIiJTA+ynKTqazx8MBd1u4QxERERmREng/C/LTANhb0xzmSEREREamBN7PfCVwERGJEkrg/WQkxVOQkagELiIiEU8JfJD5eWnsrVUCFxGRyKYEPsiC/DTKapqx1oY7FBERkWEpgQ+yID+N5o5uapo6wh2KiIjIsJTAB1mQp0I2ERGJfErgg+hQMhERiQZK4IPkpSeSnhSnBC4iIhFNCXwQY4xTia4ELiIiEUwJfAgL8nUomYiIRDYl8P46W2DvCyzIT6O2qYOGNp2ZTEREIpMSeH8vfhMeu5DlyW4AyjQKFxGRCKUE3t/J14KJZeW+nwCqRBcRkcgVsgRujJlljHnRGLPDGLPdGHPDENsYY8z3jTF7jTHbjDGloYoPgIxCWHMVqbufYklsFWVK4CIiEqFCOQLvBm6y1i4FTgKuNcYsHbTN+4CF3svVwI9CGJ/j1M9j4lP4cvLTGoGLiEjEClkCt9ZWWWtd3utNwE5g5qDN1gIbrOM1IMsYMyNUMQKQmgsnfYbTu1/FVG8N6VOLiIj4Kyxr4MaYOUAJ8Pqgh2YCB/rdruTYJI8x5mpjzEZjzMba2trgB3jKdbTHZXBpywbau3qCv38REZFxCnkCN8akAb8GPmetbRzLPqy191trV1trV+fl5QU3QICkTPYt/hTvit3K4f+8GPz9i4iIjFNIE7gxJh4neT9qrX16iE0OArP63S7y3hdyZs06amwWaa9+C3RqURERiTChrEI3wIPATmvt3cNs9gxwmbca/SSgwVpbFaoY+5tbmMe93WuZVrcRyv4WjhBERESGFcoR+DuBjwNnGmO2eC/nGWOuMcZc493mWWAfsBd4APhMCOMbICk+llcyzudIXAH87esahYuISESJC9UTWWtfAcwo21jg2tBENLri/CwerrmYmw6th11/gCUfCHdIIiIigDqxjWhBfhoPNq3BTlsIf/sGeFSRLiIikUEJfAQL8tNo7TYcOeELULsLtv4y3CGJiIgASuAjWpCfBsC2zDOgaA388Qtw4I0wRyUiIqIEPqIFeekA7K1thYsfg/Tp8NiFULcnzJGJiMhUpwQ+gsyUeHLTEp2e6Gl5cOmvwcTALz4ETYfDHZ6IiExhSuCjmJ+X2ndSk2nz4ZInoKUOHv0IdDSFNzgREZmylMBHsSA/jbLaFqzvOPCZq+DCDXB4OzxxGXR3hjdAERGZkpTAR7EgP42Gti7qmvsl6oVnwwXfdzq0PXO9mryIiEjIhayRS7TyVaLvrWkmLz2x74GSS6GxCl78hlPcdvbtYYpQRESmIo3AR9GbwGubj33w9C/Aqk/Aq/fAr6+C9jGdXE1ERCRgSuCjmJ6RRGpCLGU1QyRwY+D934V33wL/+TX8+FQdJy4iIiGhBD4KYwzz89P6KtEHi4mFM/4HPvEnZy38Z++Fl7+jtqsiIjKhlMD9sCAvjbKhptD7Kz4RrvkHLF3rnL1sw1poPBSaAEVEZMpRAvfD/Pw0qhraae7oHnnD5Cz4yM9g7b1w0AU/OsWZWleVuoiIBJkSuB98hWxDroMPZoxTob7uZciaDU99En52LlRunOAoRURkKlEC90P/Q8n8lrsAPvUCfGA9HN0HP30PPPkJcO+fmCBFRGRKUQL3Q3FOCvGxhg2vlfPEGwc4WN/m3y/GxsGqK+CzLjj9f2D3n+CHJ8Bfvgpt9RMas4iITG7GRvn67OrVq+3GjRM/Pf2d53bz+BsHqGvuAGDOtBTeuSCXUxfkcvL8aWSlJIy+k4aD8OIdsOUxSM6Gk6+FEz7lrJ2LSPhYC21uaD7snOOgswW62qCrdeBPT7dzhImn23vpcm6bGIiNh9iEfj8TnesJaZCQ6r2kQUKKcz0xw7nEqp+WjMwYs8lau/qY+5XA/Wet5a3Dzbyyt45/7q3jtX1HaOnsITk+ll+tO4kVRX4m4qqt8MLXYO/zzn/g1Z90knla/sT+ASJTXXsj7H4Wyv/pJOvmw9Bc41w8XYHtKyYeYuKcCxZ6Op1LoBLSICnTuSRm9F1PyoSkjKEfS8xwHkvMgPhkp/ZGJi0l8AnQ1eNh64F6PvOoi5zUBH5//anExwawKlG1FV75Hmz/rfONveRSeOdnIXvOhMUsMuV0tcFbzzlHhOz5C3S3Q3IOZM6EtALvJR/SpjunDU7KhPgUJzHGp3p/pkB8Ur+kPcz/c2uhp6svmXd3OKP3zpZ+l2bnZ0ej84WivcG5dHh/ttV7H2twHrej9JSIifcm+ixIyYGUac7fl5LjzPT5fiZlObN9vuuJGcP/HRJRlMAn0F+2V3P1I5v4wjmLuO7MhYHv4EiZ0451yy/BemD5h+HUz0PB0uAHKzIVdHfCvr87SXvXH6GzCVLzYdl/wXEfgaITomPUam2/ZN/Ql9Q7GqG9vt/1RmcJoO0otHovbUedLw/DMTGQkN43pR/f72dimnMUTe4iyF3o/EzJCd3fLQMogU+wax918dcdh3n2htN6q9YD1ngI/nUvbHwIulrgHefDqTdC0argBisyGXW1wd4XYOczsPvPzog2KROWXOAk7TmnOZ0Tp5KudieRt9U7Cb/NPfB673p/v1mCrlbnC0F9+cAlgeQcJ5FPmw+ZRZA5q9/Pmc5MhUwIJfAJVtvUwVl3v8TC/DSeWHcyMTHj+HbfehRe/wm8/mPnP9q8d8FpNzkfQNEwahAJlY4m2PNXJ2m/9Rfni29yNix+Pyy9AOa9G+L8KDCVY3l6nCRetxeO7IG6t5zrR/dBUxUwKHek5DrLEanTnGn8lFznZ2qu82+SmO6sP/PpVAAAIABJREFU9yemeYv5vNdjE53ZAE3nD0sJPASe2lTJF57cytfWLuOyk+eMf4cdTc5o/F8/dIptZq52zoC26Fwlcpm6jpQ5a9lv/Rn2v+oUn6Xmw5LzndH2nFOd6m+ZOD1dzoxhQyU0HPBeKqG5FlqPQGud87PNHdh+TYz3EuvMlsTEOf+WMfHe6v5+130FhL3bxDqPJaT0fUHwVf8npjtLAwP2keAcAeDbn/E+X0y/5zaxzrKm9Ti1CL7rHu/t3qMRfEcneO+btSaoRclK4CFgreWyn/0bV7mbv9x4BjOzgjSl1NUOWx511snrK6DgOCeRL7lA31pl8uvphop/OoVobz3njAYBchfDovfC4vfBrBOn3vR4NOjp7luX72xxahE6mgde7+kalBy9131JsafT+ZLW0z3weu9hfL7tupzbna0D9z9aEeBEuOQpWHh20HanBB4iB4628t57XmbN3BweuuIETDBHyj1d8OaT8I/vwpG9zgfY6V+AZR/SsaQyufR0wdsvwY7fwc4/OEkgNsEZXS86FxaeAzlzwx2lRDprnSMBOluc5ZUeb8Lv6fQeLeA9YqB3NN1vFG17+o7x911iYgde943Se2cDvNez5zhHBgSJEngIPfTq29z++x3cc9FKPlgyM/hP4OmBHb91TltaswOy5zpV6ysucg51EYlG3R1O5fiO3zmV4+31TpX04nOd2ab5ZzprpiJTjBJ4CPV4LB/58T/ZX9fC8zeewbS0xIl5Io/HaUrx8l1QtQVS85zObquvdI5nFYlknh6nF8LbL8HbL0P5v6C7DRIz4R3nOafmnfdufSmVKU8JPMT2HG7i/d9/hfcsyee+S0qDO5U+mLXOh+C/7oM9zzlVnSsudLq75S+ZuOcVCUR3J9RshwP/hn0vwf5XnEO9APKWwNzTYcFZzlEXqhwX6TVcAtfC6QRZWJDOjecs4tt/2sWDr7zNp06bN3FPZozzoTfvXVD7Frz+I6cpzOZHnGnHU653RjKqXJdQsdY5897BTc6pdA9uckbbPc65BMieA8s+6CTtOadBekE4oxWJShqBTyBrLdf8YhPP76zhsU+dyInzpoXuyVuPwsafwb8fgOZqKCxxjiVf/H5VrkvwtTfCIRdUvuEk7Mo3nMOIAOKSoXAlzFwFRaudwyGzZoU3XpEooin0MGlq72LtD1+lsb2bP372VAoyQrye190BW38Jr9wD7rch7x1Od7flH1bluoyNpwdqd/WNris3OsWUvsYeuYudVqVFq52knb9U7zWRcVACD6M9h5tYe++rvGN6Oo9ffTIJcWEYAfd0O5Xr//iu82GbNRveeYNzApW4CSqyk+jn8TjduA5tdhL2QZdTMOnrsZ2U6U3W3svMVTo9rkiQKYGH2R+2HeK6xzZz+cmzuX3t8vAF4vE4hW4vfwcOboSMIudY8pWXqHBoKuvpdhJ17S7v5S3nZ91bfck6NhFmrHCStO+SM0+1FSITTEVsYXb+ikK2VNTz01feZmVxFv9VUhSeQGJinM5Vi86FfS/Ci9+EP3wOXrkbTv8fOP5itaGcrHq6nJaXR/Y5/ayP7oOjZc5Pd/nA82FnzHROXFF6OeQtdtaw85fpS55IBNEIPIS6ezxc8tPX2VpZz9OffidLC4PXqWfMrHXO4PTiHU4RUvZcOOOLcNxHtW4ZrY6UOVPe7v3Opb7c+dlQ6bSo9IlPdUbQOXOdM0xNW+DUSOQudKbGRSQiaAo9QtQ2dXD+D/5BYlwsv7/+VP5/e3ceH1V973/89ZnJvi8ECJCw7yogiKKiiIJ7vdpF7aKttl7barXa2/X+WmtXe7W1tGq1arXXWq3b1briDiiKYRHZtyQkyJKQEJKQfb6/P85EIntgkpNh3s/HYx5z5szJzGeODu/5fs85329mcg9p7TrnjTP95q9gy1LvH/ZTbvRa5DpG3vOFQt4EH+/f441m1i61t3fJVvZA7z5roBfWOUO8maPU/S3S4ynAe5CFpdV84d75fH7iAH772eP8LufTnPOGsWwf3S29H5x8ndeVqmEse57GnbDkUVhwr9cVnt4PTrgaRp7nhXZCqt8VisgRUoD3ML95cSX3ztnAY9ecxEndeX34oXLOO0Y+9/dQMtebz/fEb8Lkb0BKjt/VydYVsOhhWPwPb9alAZPhpGu9McN1DoPIUUUB3sPsam7l7DvnEB8M8OJ3ppIU34OnQiz7wDvJbfWL3nHTCV+GyddAr2F+VxZbdmyEj570btuWe/MYj73YC+7+E/2uTkS6iAK8B3p7TQVXPriA75w5nJtmjPC7nIPbuhzemQXLnvLOWB52Fpx4LQw9U6O7dZX6Su/6/aVPQNl73rqCE72TDMf8hyatEYkBCvAe6sbHFvPCR5t56YapDOud7nc5h6ZuGyx8CD54wBumNWeo1yIf/8WIzoEbkxprvFm5SuZ6E9RsWQY4b7KP4z7vjaCXPcjvKkWkGynAe6jKuibO+v3bDO+dxuPXTCEQiKKzglubYeVz8P69UL4AEtK8OclPuBr6jPW7up4v1AZVxd7IeJuKoHiud+KgC3mDphRM9ib6GHU+9PVx8B8R8ZUCvAf7V1EZ339yKb+++Fi+eGKh3+Ucnk0LvRb5R096M04VnuwF+ejPxPbgH6EQ7KqE2s1Qs8kb3WzbSqhY6Y121j47VyDeGzt80FQYPNU7KU3zYIsIPSDAzexB4AJgm3Nur+aEmU0DngWKw6ueds7derDXPRoC3DnH5X99j+Uf7+T1m06nd3dPeBJJu6pg8SNQ9IA3eEhqb5h4pde9ntOFU6ruj3PQUO0NZlK/3VtuqArfV3v1NtdBqNVrEYdavRZw+3IgzjurO5jgXQ/fvtzxTO9PfYUcNO2EnZuhdot3iCHU+umaMvp787TnjfIm+mhfTkjpjj0iIlGmJwT4aUAd8PcDBPj3nHMXdOZ1j4YAB9hQUcc5f5zLjDF9uOuLx/tdzpELhWD96/DB/d4AMTjIH+/NAT3mP7zRvyLFOaiv8Fq329d5PxyqisMjkZVCU82+/y4x05t4IzEDAkEvrANBsGD4cdAL8rYWr6Xc1uLN7tbW7C1/ahCUDssJqZCR712Tnd4X0vPDj/M1ypmIdJrvY6E75+aY2aDuer9oMyQvjevPGMYdr67hkglbOXN0H79LOjKBAAyf4d12lHlnUi9/Bl67xbv1m+BdAjXyfMgqPLRu9tYmbzjQHaVQsTo86Ub4vqF693bBBG/EsexB3hnbOYO9x2m9vevZk3O8ENVQsSISxbr1GHg4wJ8/QAv8KaAc+BivNb58P69zDXANQGFh4cTS0tIuqrh7NbeGOH/WXJpaQ7z1vWnRdULboaou3R3mHy/evT6l1+5WanurtbXJm3xjR5l3DXTdlk+/VlJWuPt5pNcF3T6Od3o/XdYmIkcN37vQw0UMYv8BngGEnHN1ZnYe8Efn3PCDvebR0oXe7qmF5dz8xIc8/a2TOb4w2+9yulZVsXe51M7NUPuxd8y4tv3Y8TavSztzAGQVQGZh+L7Aa7HnjYTUPI3lLSJHPd+70A/GObezw/KLZna3mfVyzlX6WVd3O2tMHxKCAV5cuvnoD/Ccwfs/Ft7WChZQS1pEZD96zL+OZtbXzGtOmdlkvNq2+1tV98tMjmfq8F68+NFmQqHovsTviATjFN4iIgfQbf9Cmtk/gfnASDMrN7OrzexaM7s2vMnngGVm9iEwC7jMRftF6ofp/OPy+bimkSXlO/wuRUREeqjuPAv98oM8/2fgz91UTo/W3o3+Qix0o4uIyGFRH2UPlJEUz2kj1I0uIiL7pwDvoc4/Lp/NNY0sLlM3uoiI7E0B3kOdOXp3N7qIiMieFOA9lNeNnsdLy9SNLiIie1OA92DnH9c33I1effCNRUQkpijAe7CzRvchIS7AC0u3HHxjERGJKQrwHiw9KZ7ThufpbHQREdmLAryHu+C4fLbsVDe6iIh8mgK8hztzdG8S4gI8r7PRRUSkAwV4D5eeFM/pI/J46aMt6kYXEZFPKMCjwPnHet3oizaqG11ERDwK8CjQ3o3+wkfqRhcREY8CPAqkJ8UzbYTORhcRkd0U4FHi/OPy2bqzSd3oIiICKMCjxpmj+5AcH+TR9zf6XYqIiPQACvAokZYYx5dOLOTZDz+mpLLe73JERMRnCvAocs1pQ4gLGHe9uc7vUkRExGcK8CjSOyOJyycX8vTiTZRV7fK7HBER8ZECPMp8c9pQggHj7rfUChcRiWUK8CjTJyOJy04o4Imicsqr1QoXEYlVCvAodO3pQzGDe95a73cpIiLiEwV4FOqXlcwXJhXwr6IyPt7R4Hc5IiLiAwV4lPrmtKEA/OVttcJFRGKRAjxKDchO4XMTB/DYgjK21DT6XY6IiHQzBXgU+9a0YbQ5x71z1AoXEYk1CvAoVpCTwiUT+vPo+xvZVqtWuIhILFGAR7nrpg+jNeS47+0NfpciIiLdKO5I/tjM0oDTgTXOubWRKUk6Y2BuKheN78eD7xQzb10l4wZkcVxBJuMGZDGybzrxQf1GExE5GnUqwM3sUeA959wsM4sH3gdGA61mdolz7vmuKFIO7GcXjGVAdgpLynbwyootPF5UBkBCXIAx+RlcP30YZ47u43OVIiISSZ1tgU8D7ggvXwikA/nA1cD/AxTgPshMieemGSMAcM5RVtXAh+U7WFq+g9krtnLzEx/y9n+dQWZyvM+ViohIpHS2fzUH2BpengE87ZzbCjyK1xIXn5kZhbkpXDiuHz85fwx3f+l4ahpauFszmImIHFU6G+AVwODw8gzgzfByChCKVFESOWP7ZXLJhAH87Z0SzWAmInIU6WyAPwH8w8xeAzKAV8PrxwM6ia2H+t7ZIzCD22ev9rsUERGJkM4G+PeBO4FlwAznXHuTrh/w10gWJpGTn5nMN6YO4dklH/Nh2Q6/yxERkQjoVIA751qdc793zt3onPuww/rbnXP3Rb48iZRrpw2lV1oCv3pxJc45v8sREZEj1KkAN7NxZja2w+PzzOwJM7vFzI7omnLpWmmJcdx41ggWFFfx6oqtB/8DERHp0TrbhX4vcCyAmQ0AngTSgG8Av4xsaRJpl51QwNC8VH770ipa2nTOoYhINOtsgI8EFoeXLwE+cM6dC1wBXBrJwiTy4oIBfnzeaDZU1vPYgo1+lyMiIkegswGeALTPmjENeCm8vAboG6GapAtNH9Wbk4bk8IfX1rKzscXvckRE5DB1NsBXA58zs0K868BfC6/PB6ojWZh0DTPjJ+eNoaq+mb+8pWlIRUSiVWcD/OfAr4FiYJ5zrii8fia7u9alhzt2QCYXT+jPA/OKKams97scERE5DJ29jOxZoBCYCJzf4anXgf+KYF3Sxb5/zkiSE4Jc+8hCGprb/C5HREQ6qdNzTTrntjrnlgAJZpYUXjffObci4tVJl8nPTObOS8ezemstP3nmI10bLiISZTod4Gb2NTNbB9QBdWa21sy+GvHKpMtNG9mbG88cwdOLN/HI+zorXUQkmnR2IJcbgLuB54DPhm/PA3eb2fUH+dsHzWybmS3bz/NmZrPMbJ2ZLTWz4ztTmxye66cP44yRedz67+Us2qjzEEVEokVnW+DXAzc4525yzj0bvn0X+C5ww0H+9iHgnAM8fy4wPHy7Brink7XJYQgEjDsvnUB+ZjLfemQRlXVNfpckIiKHoLMBXoB3wtqeXg8/t1/OuTlA1QE2uQj4u/O8B2SZWX4n65PDkJkSzz1fPp7qXc1c/+hiWjVKm4hIj9fZAC/HG8BlT9PCzx2J/kDZHu/Vf18bmtk1ZlZkZkUVFRVH+LYC3rzhv7r4WOZv2M7ts9f4XY6IiBxEZycguQeYZWbDgLnhdafhda3/NJKFHUh45rP7ACZNmqTTpyPkcxMHsHhjNX95ez3jCzI55xh1gIiI9FSdCnDn3O1m1gD8IHwDr6X8PefckR6z3sSnu+EHhNdJN/rphWNYtqmGnzyzjFOH55GWqEnmRER6osO5Dvwu51whkAlkOucKIxDe4J3ZfkX4bPSTgBrn3OYIvK50QmJckJ9fdAzb65t5YG6x3+WIiMh+HLR5ZWazD/L8J8vOuZkH2O6feMfKe5lZOfAzID78d38BXgTOA9YBu4CvHbR66RLjC7I4Z2xf7puzni+fVEhuWqLfJYmIyB4OpX80It3YzrnLD/K8A74difeSI/e9s0cye8UW7npzPT+9cIzf5YiIyB4OGuDOObWEY9Cw3ml8fmIBj7xXylWnDmJAdorfJYmISAedPgYusePGGcPB4A+vrvW7FBER2YMCXPYrPzOZr548iKcXl7N6S63f5YiISAcKcDmgb54+lLSEOP7nldV+lyIiIh0owOWAslMTuHbaUF5buZWFpQcaCVdERLqTAlwO6munDCIvPZHbXlqtecNFRHoIBbgcVEpCHN85czgLSqp4a7XGnhcR6QkU4HJILjuhgIG5Kdz28ipCIbXCRUT8pgCXQxIfDHDTjBGs2lLLv5d+7Hc5IiIxTwEuh+zC4/oxqm86f3h1jeYMFxHxmQJcDlkgYNw8cyQl23fx9CJNFCci4icFuHTKWaN7M25AJn98fS3NrWqFi4j4RQEunWJm3DRzJJt2NPB4UZnf5YiIxCwFuHTaacN7ccKgbP78xloaW9r8LkdEJCYpwKXTzIybZoxk684mHnmv1O9yRERikgJcDsuUobmcMiyXv7y9nvqmVr/LERGJOQpwOWw3zRhJZV0zD88v8bsUEZGYowCXwzZxYDbTR/Xm3rc3sLOxxe9yRERiigJcjshNM0ZQ09DCA3OL/S5FRCSmKMDliBzTP5NzxvblwXnFVNc3+12OiEjMUIDLEfvujBHUNbdy39wNfpciIhIzFOByxEb2TeeC4/rx93dLNDqbiEg3UYBLRJx/bF/qm9tYWr7D71JERGKCAlwi4sTBuZjB/PXb/S5FRCQmKMAlIrJTExjVN4P5GxTgIiLdQQEuEXPy0FyKSqs1PrqISDdQgEvETBmSS3NriMUbdRxcRKSrKcAlYiYPySFgqBtdRKQbKMAlYjKS4jmmfybz11f6XYqIyFFPAS4RNWVoLkvKdtDQrOPgIiJdSQEuETVlSC4tbY6i0iq/SxEROaopwCWiThiUQ1zAdD24iEgXU4BLRKUmxjGuIIt3FeAiIl1KAS4RN2VILh9tqqGuqdXvUkREjloKcIm4KUNzaQs5PijWcXARka6iAJeImzgwm4RgQNeDi4h0IQW4RFxSfJAJhVm8q+vBRUS6jAJcusSUobks/3gnNbta/C5FROSopACXLjFlSC7OwfvF6kYXEekKCnDpEuMLs0iKD+hyMhGRLqIAly6RGBdk0sAc3tOJbCIiXUIBLl1mytBcVm2pZXtdk9+liIgcdRTg0mWmDM0F4H1dDy4iEnHdGuBmdo6ZrTazdWb2w308/1UzqzCzJeHb17uzPomsY/tnkpoQ1OVkIiJdoNsC3MyCwF3AucAY4HIzG7OPTR93zo0P3+7vrvok8uKDAU4YnBOxiU0aW9q45631nHPnHIor6yPymiIi0ao7W+CTgXXOuQ3OuWbgMeCibnx/8cHJQ3NZX1HPtp2Nh/0aoZDjmcXlTL/9LW57eRWrttTy9KLyCFYpIhJ9ujPA+wNlHR6Xh9ft6bNmttTMnjSzgu4pTbrKlCG9ALj7rfW8snwLRSVVbKioo2ZXC865g/79u+sr+cxd8/ju4x+Sm5bIP79xEicOzmH28q1dXbqISI8W53cBe/g38E/nXJOZ/SfwMDB9z43M7BrgGoDCwsLurVA6ZUy/DAZkJ/PQuyU89G7Jp56LCxg5qQnkpSeSl55I7/REeqcn0TsjkeyUBJ5dsonXVm6jf1Yyd146ns+M60cgYKzc3Jdbn19BSWU9g3ql+vPBRER8ZofSCorIG5lNAW5xzp0dfvwjAOfcb/azfRCocs5lHuh1J02a5IqKiiJdrkRQS1uIyromttc1s72+map6b7mqvpntdc1sq22koq6JbTubqKxrIhT+XzI9MY5vnTGMr50yiKT44CevV1a1i6m/e5MfnzeKa04b6tOnEhHpHma20Dk3ac/13dkC/wAYbmaDgU3AZcAXO25gZvnOuc3hh58BVnZjfdJF4oMB8jOTyc9MPui2bSFHVX0zFbVN9M9KJjMlfq9tCnJSGJOfwezlWxXgIhKzuu0YuHOuFbgOeAUvmP/lnFtuZrea2WfCm33HzJab2YfAd4Cvdld90jMEA0ZeeiJj+mXsM7zbnT22Lws3VlNRq0FiRCQ2det14M65F51zI5xzQ51zvwqv+6lz7rnw8o+cc2Odc+Occ2c451Z1Z30SPWaO7YNz8NpKncwmIrFJI7FJVBrVN52CnGRmL9/idykiIr5QgEtUMjPOHtOXd9Ztp7ZRc46LSOxRgEvUmjm2L81tId5eU+F3KSIi3U4BLlFr4sBsclMTeEWDuohIDFKAS9QKBoyzRvfhzVXbaGpt87scEZFupQCXqDZzbB/qmlp5b4OmLBWR2KIAl6h2yrBepCQEeUVno4tIjFGAS1RLig8ybWQer67YSijUPcMCi4j0BApwiXozx/SloraJxWU7/C5FRKTbKMAl6p0xqjdxAWP2CnWji0jsUIBL1MtMjmfK0FxmL996SHOMi4gcDRTgclSYOaYPxZX1rNtW53cpIiLdQgEuR4UZY/oCMHuFBnURkdjQnfOBi3SZvplJjCvI4g+vruGBecUkxQVIig+SGB8kKT5ASkKQ4b3TOX5gNhMKshiQnYyZ+V22iMhhU4DLUeOWC8fwwtLNNLa20dgSorHFu29qbaO2sZXHPyjjoXdLAMhLT2RCQRbHD8zm9BF5jM7P8Ld4EZFOsmg/6WfSpEmuqKjI7zIkCrS2hVi1pZbFZTtYXFrNoo3VlGzfRVzAePKbJzO+IMvvEkVE9mJmC51zk/ZarwCXWLZ1ZyOX3P0ucUHjhe9MJS1RnVIi0rPsL8B1EpvEtD4ZSdx52XjKqnbxs2eX+12OiMghU4BLzDthUA7XTx/OU4vKeXbJJr/LERE5JApwEeD66cOYODCb/35mGWVVu/wuR0TkoBTgIkBcMMCdl44H4IbHFtPaFvK5IhGRA1OAi4QV5KTwy4uPYdHGHcx6Y90+t9lS08idr63hqoc+YMeu5m6uUERkN51yK9LBReP78/aaCv78xlpOHdaLyYNzCIUc89ZV8sh7pby+ahsh53AOHphXzM0zR/pdsojEKAW4yB5uvegYikqqufGxxXxlyiAe+2Ajpdt3kZOawDemDuGLkwu57eVVPPROCV8/dQiZKfF+lywiMUhd6CJ7SEuMY9blE9hW28RtL6+iT3oSf7xsPPN/NJ0fnjuKwtwUrps+jNqmVh58p9jvckUkRqkFLrIP4wuyeOLaKaQmxjGiT/pez4/Oz+DssX148J1irp46mIwktcJFpHupBS6yHxMKs/cZ3u2unz6c2sZWHn6npPuKEhEJU4CLHKZj+mdy1uje3D+vmLqmVr/LEZEYowAXOQLXTx9OTUMLf59f4ncpIhJjFOAiR2BcQRbTRuZx/9xi6tUKF5FupAAXOULXTx9OVX0z/3i/1O9SDqisahfXPbqIqnoNQCNyNFCAixyhiQOzmTq8F/fN2UBDc5vf5ezXb19exfNLN/PAvA1+lyIiEaAAF4mA75w5nMq6ntsKX7aphheWbiYlIcjf55dS29jid0kicoQU4CIRcMKgHE4emsu9czbQ2NLzWuF3zF5NZnI8f71iErWNrfzj/Y1d+n5z1lSwbWdjl76HSKxTgItEyHfOHE5FbROPdnE4dtYHJVW8ubqCa08fyinDenHqsF48MK+4y35oFJVUccWDC7jiwQU98sfMgTwwr5hFG6v9LkPkkCjARSLkpCG5nDw0l9+9sooPSqr8LgcA5xz/8/Jq8tITufLkgQB8a9pQKmqbeHrRpoi/X1vIccu/l5ORFMeqLbX85sWVEX+PrvLaiq384vkVfOuRRbquX6KCAlwkgmZdPoF+mclc9dAHLP+4xu9ymLO2kgUlVVw/fRgpCd7IyVOG5jJuQCb3zlkf8XnPnygqY9mmnfzy4mO56pTBPDy/lNnLt0T0PbpCY0sbt/x7OfmZSWzZ2cis19f6XZLIQSnARSKoV1oi//v1E0lLjOPKBxdQXFnvWy3OOf7nlVUMyE7mshMKP1lvZnxz2lBKt+/ipWWRC9eahhZ+98pqJg/K4cLj8vnBuSMZ2y+D7z+1lM01DRF7n65wz1vrKa9u4I4vjOPSSQU8OK+YNVtr/S5L5IAU4CIR1j8rmf+9+kRCDr58//sHDK8VH+/kqoc+4MI/zeONVVsjWsfLy7awbNNObjxrBAlxn/6qzxzTlyF5qdzz1nqccxF5vztfW8OOXc387DNjMDMS44L86fIJNLeGuOGxJbSFIvM+kVa6vZ573l7PheP6cfLQXvzg3FGkJcXx3/+3LGL7RqQrKMBFusCw3mk8/LXJ1DS08JUHFuw1eMqmHQ3c9K8lnP+nuSwsraa2sYWrHiriq39bwPqKuiN+/7aQ4/bZqxnWO42LJ/Tf6/lAwLj29KGs2LyTOWsrj/j91myt5e/zS7l8ciFj+2V+sn5IXhq/uOgYFhRX8ac3ema39K3/XkF8wPjJeaMByElN4Ptnj2JBcRX/tyTy5wmIRIoCXKSLHDsgk/uvnMTGql189W8LqGtqpaahhd+8tJIzbn+L55du5prThjDnv85g9ndP57/PH83CkmrO/sMcfvn8Cnbuca12W8ixeGM1s15fy+fueZdTb3uD219Zvc8W/jOLN7G+op6bZ4wgGLB91vcf4/uTn5nE3W+uO6LP6Zzj5/9eTlpiHDfPHLnX85+dOICLJ/Rn1utrWVDcM07ua/faiq28vmobN5w1nL6ZSZ+sv+yEAsYVZPGrF1ZR06Br5qVnsmjvIpo0aZIrKiryuwyR/Xq2WhJsAAAQK0lEQVRtxVb+85GFjOyTzsc1DdQ0tHDxhP7cPHMk/bOSP7VtRW0Td8xezeNFZeSkJHDTzBHEBYw5ayqZt66SmoYWzODY/plkpSQwd20FATNmjunDFVMGcdKQHFraHNPveIvslASeu+4UzPYd4OBdNvWL51fw1DdPZuLA7MP6fC8v28K1jyzk1ovGcsWUQfvcpq6plQtmzaW5NcSLN0wlKyXhsN4rkhpb2pjxh7dJigvy4g1TiQ9+uj3zUXkNF901j6+cNJCfX3SMT1WKgJktdM5N2mu9Alyk6z2zuJyb//Uhpw7P44fnjGJMv4wDbv9ReQ0///dyikq9a5L7ZCQydXgep43I49RhvchJ9QKwrGoXj7xXyuNFZezY1cKIPmmMzs/g2SUf8/BVkzl9RN4B36e+qZVTbnuDEwbl8Ncr9vr34aAaW9o46/dvk5YYx/PXn0pccP+dekvLd/DZe97l9BF5zLp8widnxfvlztfWcOdra3n0Gydy8tBe+9zmp88u45H3SnnuulM5pn/mPrfZl7Vba7l/bjGvrdzKReP7c9PMEaQl+vt5JXopwEV8VtvYQnpS/CFv75xj/obt5KYmMqJP2gFb0o0tbTz34cf8fX4JyzbtZPLgHB6/5qQD/k279iCb/d3TGJaXxuadjZRW1lOyfRel2+v5uKaRwbkpTBiYzfEF2WSm7P4Ms15fy+9fXXPAEOzowXnF3Pr8CnJSE/j61MFcMWXQYQVbKOT4uKbhk7P8xxVkkdGJfbtx+y7O+sPbnD22L3+6fMJ+t6tpaOHMO95iQHYKT3/zZAL7ORwB3n+v9zZU8de5G3hj1TYS4wJMHpzDvHWV9ElP4qcXjuHcY/oe0n8TkY56RICb2TnAH4EgcL9z7rd7PJ8I/B2YCGwHLnXOlRzoNRXgIrs551ixeSf5mcmftNIPprq+mZN/+waJ8QF2NbfR3Lr72vCEuAB9MhLZVN1A+0nkw3qncXxhFsf0z+TXL67kzFF9uOtLxx9yjQtLq5j1+jreXlNBVko8V58ymCtPGbTPAG5obmPN1lpWb62luLKe4op6iivrKdleT1OHOs1gRO90jh+YxfGF2Rw/MJshvVL3G5Zff/gD5q/fzus3T/vUse99eWphOTc/8SG/veRYLptcuNfzLW0hXl62hb/O3cDS8hpyUxO4YsogvnxSIblpiSzaWM1/P7OMFZt3cvqIPG69aCwDc1MPeX/1FBsq6nhp2Ra27mxk8uAcpgzJJTct0e+yYoLvAW5mQWANMAMoBz4ALnfOreiwzbeA45xz15rZZcDFzrlLD/S6CnCRI/fo+xt5e802BuWmMjA3lUG5KQzslUp+RhKBgFHf1MqH5TtYVFrNwtJqFm3cQU1DC0nxAV676XQGZKd0+j2XlO3gz2+s5bWV28hIiuNrpwxmdH46q7bUsmqzF9ol2+tp/ycqPmgU5qQwuFcaQ/JSGdzLu7W0hVhUuoNFG6tZtLGa2kZvFLXM5HhyUxNIiAuQGBcgMS5IQlwAM5i7tpIfnzeKa04betA6nXNceu97fLSphn5ZSTS2hGhqbfvkvqXNK3BIr1S+PnUIlxzfn6T44Kdeo7UtxMPzS/n97NW0hhzXnTGMq6cOpqahhW07m9hW28S22ka27Wyisq6J+GCA1MQgaYnxpCXFkRZezkiKo1d6Ir3SEslIitvvD5Taxha27mxkc00jO3a10CcjiYKcZPqkJx2wF2HPz712Wx0vfrSZl5dtYdUW77r45PggDeEhckfnZ3DK0FxOGdaLyYNzSO1hhwlCIcf2+ma21TbS1BqiX2YyvdMTD3kf9BQ9IcCnALc4584OP/4RgHPuNx22eSW8zXwziwO2AHnuAEUqwEW6Xyjk2BDuvh7WO+2IXmvZphr+9MZaXlnuXQdvBoNyUxnVN52RfdPD9xkUZCcf8Bh7e13rK+pYWFrNh+U11Da20NQaornVC9v25aF5adzxhXF7nbi2PyWV9fz+1TW0hRyJ8QGS4oMkxQVJCi+Pyc9g+qjeBw2GLTWN/OL5Fbzw0eb9bpOdEk9byFHX1MqBLp1PiAuQl5ZIr/RE8tISaGoNsbmmkS01jfsdCjYhGKB/djIDspMZkJ1Cdko8ZmB4dXvLsKu5jTdXb2N9RT1mMGlgNucek885x/Sld3oiSzfV8O66St5Zt52FpdU0t4UIBoyUhA4/XDrWbpDYYX8lxQdIiguSGB8gEP4REnKOUMi7dw5c+AUMryj7pD4jGDDig0ZcMODdBwLEBY2gGdW7Wj75MVRR17TX+APxQSM/M5n+Wcn0y0qmf3YyqQnBT167/TeRmeGco76pjfrmVuqaWqkP39r/2yTGBUgIBkiM9+4T4rzbV04axMi+6fv/j9dJPSHAPwec45z7evjxV4ATnXPXddhmWXib8vDj9eFtKvd4rWuAawAKCwsnlpb2zCkcReTQra+oo66xlRF90klOCB78D6LYvLWVFJVWkZeeSO/0JHqnJ5IXblm3D7rjnKOxJURtUwv1TW3UNXqXIVbWNVFR2/TJfUX4Pik+SH5mEn0yksjPTKJvZhJ9M5LISklgy85Gyqp2UVa9i/LqBsqrdlFW3UBtY0s4LPlk0BoHBM04YVAO5x3bl7PH9qV3xv4PMzS2tFFUUs37xdv3+uHQ/sMg5BxNrSGaWtpoDPdeNLa00djShsML54AZAfMCtP2+PcjbayT8uC3kaA05mltDtIYcrW0hWtq89Vkp8fTOSKJPeiK9MxLpk+Ht38S4IJt2NHi36t33W2sbOVgMJsUHSEuMIzUxjtSEONIS4zCD5rb2H4fefXNriOa2EH+6fAKnDDv4OSGHan8B3rP6Ow6Rc+4+4D7wWuA+lyMiETA078ha8tHk1OG9OHX4gf+BNzOSE4Lej5kjbMxFsjW4p6T44CF9np6qJRzC7T9ivHto/2WRmhA8aM+PX7ozwDcBBR0eDwiv29c25eEu9Ey8k9lEREQiLj4YOORDKT1Nd1b9ATDczAabWQJwGfDcHts8B1wZXv4c8MaBjn+LiIjEqm5rgTvnWs3sOuAVvMvIHnTOLTezW4Ei59xzwAPA/5rZOqAKL+RFRERkD916DNw59yLw4h7rftphuRH4fHfWJCIiEo2is+NfREQkxinARUREopACXEREJAopwEVERKKQAlxERCQKKcBFRESiUNTPB25mFUAkB0PvBVQedKujn/aDR/vBo/2gfdBO+8HTnfthoHMub8+VUR/gkWZmRfsaND7WaD94tB882g/aB+20Hzw9YT+oC11ERCQKKcBFRESikAJ8b/f5XUAPof3g0X7waD9oH7TTfvD4vh90DFxERCQKqQUuIiIShRTgIiIiUUgB3oGZnWNmq81snZn90O96uouZPWhm28xsWYd1OWb2qpmtDd9n+1ljVzOzAjN708xWmNlyM7shvD7W9kOSmS0wsw/D++Hn4fWDzez98HfjcTNL8LvW7mBmQTNbbGbPhx/H3H4wsxIz+8jMlphZUXhdrH0vsszsSTNbZWYrzWxKT9gHCvAwMwsCdwHnAmOAy81sjL9VdZuHgHP2WPdD4HXn3HDg9fDjo1krcLNzbgxwEvDt8H//WNsPTcB059w4YDxwjpmdBNwG/ME5NwyoBq72scbudAOwssPjWN0PZzjnxne47jnWvhd/BF52zo0CxuH9P+H7PlCA7zYZWOec2+CcawYeAy7yuaZu4ZybA1Ttsfoi4OHw8sPAf3RrUd3MObfZObcovFyL9wXtT+ztB+ecqws/jA/fHDAdeDK8/qjfDwBmNgA4H7g//NiIwf2wHzHzvTCzTOA04AEA51yzc24HPWAfKMB36w+UdXhcHl4Xq/o45zaHl7cAffwspjuZ2SBgAvA+Mbgfwt3GS4BtwKvAemCHc641vEmsfDfuBL4PhMKPc4nN/eCA2Wa20MyuCa+Lpe/FYKAC+Fv4cMr9ZpZKD9gHCnA5KOddaxgT1xuaWRrwFHCjc25nx+diZT8459qcc+OBAXg9U6N8LqnbmdkFwDbn3EK/a+kBTnXOHY93ePHbZnZaxydj4HsRBxwP3OOcmwDUs0d3uV/7QAG+2yagoMPjAeF1sWqrmeUDhO+3+VxPlzOzeLzw/odz7unw6pjbD+3C3YRvAlOALDOLCz8VC9+NU4DPmFkJ3uG06XjHQWNtP+Cc2xS+3wY8g/ejLpa+F+VAuXPu/fDjJ/EC3fd9oADf7QNgePgs0wTgMuA5n2vy03PAleHlK4Fnfayly4WPbz4ArHTO/b7DU7G2H/LMLCu8nAzMwDsf4E3gc+HNjvr94Jz7kXNugHNuEN6/BW84575EjO0HM0s1s/T2ZWAmsIwY+l4457YAZWY2MrzqTGAFPWAfaCS2DszsPLzjXkHgQefcr3wuqVuY2T+BaXjT420Ffgb8H/AvoBBvutYvOOf2PNHtqGFmpwJzgY/Yfczzx3jHwWNpPxyHd0JOEO8H/r+cc7ea2RC8lmgOsBj4snOuyb9Ku4+ZTQO+55y7INb2Q/jzPhN+GAc86pz7lZnlElvfi/F4JzMmABuArxH+fuDjPlCAi4iIRCF1oYuIiEQhBbiIiEgUUoCLiIhEIQW4iIhIFFKAi4iIRCEFuIh0OTObZmYuPL64iESAAlxERCQKKcBFRESikAJcJAaY2fVmtsrMGs1srZn9pH1MbzMrMbNfhWdZ2mlmlWb2azMLdPj7dDO718wqzKzJzIrMbOYe79HbzP5mZlvD77PazK7ao5TRZjbHzHaZ2QozO7cbPr7IUSnu4JuISDQzs1vwhn68EVgCjAb+AiQB/y+82fV4wwifgDdZxV/whtX9Y/j5B8PPfRnYCFwLPG9mxznnVoXHTX8baAC+hDfc5DC8IUc7uh34Ad4UpT8GHjezgc656sh+apGjn4ZSFTmKmVkKUAlc4px7ucP6K4BZzrms8IxbZc65qR2e/zXwFedcgZkNA9YC5zvnXuywzSJgiXPuKjO7GrgLGOacK99HHdPwJgL5bPtMb2bWB28e5XOcc69E+rOLHO3UAhc5uo0FkoGnzKzjr/UgkGRmeeHH8/f4u3eAH5lZBjAmvG7OHtvMwZtqFGAisGJf4b2HJe0LzrmtZtYG9DmkTyIin6IAFzm6tR/H/jywZh/Pd/cMUs37WKdzcUQOg744Ike35UAjMMQ5t24ft7bwdift8XcnA5ucczvDrwFw2h7bnIY3NzTAQmCMrvMW6T4KcJGjmHOuDvg18Gsz+7aZjTSzsWZ2mZnd1mHT8WZ2i5mNMLMvAjcAd4RfYz3wBHC3mZ1tZqPM7I/AMcD/hP/+n3hzIj9nZmeZ2WAzO9PMLu2uzyoSa9SFLnKUc879wsw2A9fhhXIDXnf6Qx02+xMwECgCWoA/s/sMdICv44X1I0AG8BFwgXNuVfg9dpnZ6cDvgMeANKAE+G1XfS6RWKez0EViXPgs9Pudc7/0uxYROXTqQhcREYlCCnAREZEopC50ERGRKKQWuIiISBRSgIuIiEQhBbiIiEgUUoCLiIhEIQW4iIhIFPr/4uR4gAO5Qt0AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "sg.utils.plot_history(pretrained_history)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We've now fine-tuned our model for node classification. Observe that the accuracy in the first few epochs was very poor, but it quickly improved. (The train accuracy plot is quantised because the training set is so small.)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1/1 [==============================] - 0s 920us/step - loss: 1.5896 - acc: 0.5632\n", + "{'loss': 1.5896106958389282, 'acc': 0.5631818175315857}\n" + ] + } + ], + "source": [ + "pretrained_test_metrics = dict(\n", + " zip(pretrained_model.metrics_names, pretrained_model.evaluate(test_gen))\n", + ")\n", + "print(pretrained_test_metrics)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Model without Deep Graph Infomax pre-training\n", + "\n", + "Let's also train an equivalent GCN model in a fully supervised manner, starting with the " + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "direct_gcn_model = make_gcn_model()\n", + "direct_x_in, direct_x_out = direct_gcn_model.in_out_tensors()\n", + "direct_predictions = tf.keras.layers.Dense(\n", + " units=train_targets.shape[1], activation=\"softmax\"\n", + ")(direct_x_out)" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [], + "source": [ + "direct_model = Model(inputs=direct_x_in, outputs=direct_predictions)\n", + "direct_model.compile(\n", + " optimizer=optimizers.Adam(lr=0.01), loss=\"categorical_crossentropy\", metrics=[\"acc\"],\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "direct_history = direct_model.fit(\n", + " train_gen,\n", + " epochs=epochs,\n", + " verbose=0,\n", + " validation_data=val_gen,\n", + " callbacks=[prediction_es],\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAI4CAYAAACV/7uiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3iUVdrH8e+ZmTSS0HuTIiIoVcCOuvbeFbsuq6yrrm13dXXXturquusruqKC7tp1LdjBhthQULCAgCAgIEhHIAkkmXLeP85MMgkTSJnMk0l+n+uaK5knT2buDMzczznnPucYay0iIiKSXnxeByAiIiI1pwQuIiKShpTARURE0pASuIiISBpSAhcREUlDAa8DqKu2bdvaHj16eB2GiIhIvZg1a9Z6a227ysfTPoH36NGDmTNneh2GiIhIvTDGLEt0XF3oIiIiaUgJXEREJA0pgYuIiKQhJXAREZE0pAQuIiKShpTARURE0pASuIiISBpKWQI3xvzHGLPWGPNdFT83xpj7jTGLjDGzjTFDUxWbiIhIukllC/xx4Kgd/PxooE/0dgnwUApiEhERSUspW4nNWvuxMabHDk45EXjSWmuB6caYlsaYTtbaVSkJMEnCEculT8/iNwf2YkTP1mXH75w0n0lzyv+UFjkZPPObvWnZLJNb35jLe/PWeBGuiKShzICPB88eSr9OzXny86WM/3iJ1yFJnHtOG8S+vdvU+/M0pKVUuwA/xd1fET22XQI3xlyCa6XTvXv3lARXXZu3BXl33hoGdWtZIYG/P98l6BE9W7N2SwmfLlrPkvVFDO2eyZT5a/EZw7AerbwKW0TSRHEwzKQ5q5m9YhP9OjXnowXrKCoJccju7b0OTaJaNstIyfM0pARebdba8cB4gGHDhlmPw6mgqCQEQEkost3xg3drz92nDeTLpRv5dNH6snOLSkIcPaAjt580IOXxikh62bw1yKQ5qyksCQNQWBKiT4d87j1jsMeRSao1pCr0lUC3uPtdo8fSSkGxS8qllRJ4YXGIvGx3vZSXFSg7BlBQEiIvKzVXbCKS3nKz/ED550dhSYj8rLRsi0kdNaQE/jpwfrQafR9gc7qNfwMUlW6fwCMRS1FpmNysigm8oCREaShCaShCXvRNKSKyIwG/j+wMH4UlQcD14OUqgTdJKftXN8Y8BxwMtDXGrABuBjIArLUPA5OAY4BFwFbgolTFlkyxq+JguDyBx5J6fqUEXlQSKutGz9MbUESqKS8ro0IXeqx3T5qWVFahn7WTn1vgshSFU28KSrZvgRfGknT0TZYb14Ve/jN1oYtI9eRnB8o+OwqK1YXeVDWkLvRGIdaiLo1vgUePxRJ3ZsBHZsBHYWlcAlcXuohUU26Wn6KSEMFwhJJQRF3oTZQSeJIVJihiixW2xV8l52cFKrbAVcQmItWUF/380BBc06YEnmQFCaaRVe5CB9caLywJlSX8XLXARaSa8rICFJTENwCUwJsiJfAki10RBxN1oWeWv8nysgIUxb0B81WEIiLVVPnzQ0VsTZMSeJLtsAs97k2Wlx2gQF3oIlILedEiNnWhN21K4ElWWLp9EVthpSI2iI5hqQtdRGohNzoGXlC8/WeLNB1K4EmWqAVeXoVenqQrd4HFd6+LiOxIflaA0nCEX7aWuvvqQm+SlMCTrDDBPPCCkhCZAR9ZgbgEHu0CKywJkZcVwOczKY9VRNJTrMt89eaSCvelaVECT7Kqitgqv8FiXehuGUR1n4tI9cW6zNdsKa5wX5oWJfAki41JVZhGVpw4gRcHXReYrp5FpCZiXearN7sErs+QpkkJPMmKqihiq/wGK7+CLtGbT0RqJPb5sXpLMTkZfvwagmuSlMCTyFqbsIgtUQLPj+sC0xxOEamJPH1+CErgSVUSihCKWCBBAq/0JovdX1ugFriI1Ex+3OeHNjJpupTAkyh+VbX4IrbC4u33643dD0esClBEpEb0+SGgBJ5Use7z1rmZhCKWSLQ1XlgSTljEFqMraBGpifjPD/XgNV1K4EkUa4G3zs0EygvZCkuC2y20UHlZVRGR6qqwr4I+P5osJfAkiiXwNtEEXhKKEApHKA5GtltpLb7bS11gIlITPp8hN9OtH6EWeNOlBJ5EsS70Vs2iLfBQhKKSMLD9VbK60EWkLmKfKUrgTZf+5ZMoNgc81oUeDEcoCbkEXjlJx66eQS1wEak597lRos+PJkwt8CSKrcLWKnf7FnjlN1nA7yMnQ11gIlI7sUaBNjJpupTAk6goQRFbYUkQSFxoUtYFpjegiNSQutBFCTyJCktC+Ay0yMkAXAs81irPS7BhSeyNpzegiNRUrDBWXehNlxJ4EhVEF2zJCriXtSS+iC0rY7vzlcBFpLbUAhcl8CQqKgmRnxUgM5rAgzvpQo9tI6o3oIjUlBoAogSeRIUlFVvgpaEIhbEWeGaCMfBoq1xj4CJSU2UJXJ8fTZYSeBLFNi3J8Mcl8OgYeG6CMfD87AA+Q1k1uohIdakLXZTA62DR2gL2uOltftq4FSjfNjTWhR6rQs/J8BPwb/9St2yWQatmmRijvXxFpGZiC0bFimal6dGlWx0sXb+VotIwSzcU0a11MwqLQ3Rsnk2mP34MPFxlleilB/Xm+EGdUxmyiDQSJw7uTOeWObTLz/I6FPGIEngdxLYMjXWTF1VqgZeEIhSWhKpcaKF982zaN89OTbAi0qg0ywxw0G7tvA5DPKQu9DqI7TZWEF3ApSBaxJYZX8RWHEw4/i0iIlIXSuB1UBJyCbyoJIS11k0jyw6UdaHHllJVkYmIiCSbEngdlIbKu9C3BcNELNsVsRWUhBIu4iIiIlIXSuB1UJbAS0Jx08XKW+DBUCQ6Lq4udBERSS4l8DqIjYEXloQojI6D52cHCPh9+ExsGllICy2IiEjSKYHXQTC0fQKPbTCQGfCVLeSiLnQREUk2JfA6iLXAi+K60GOt7Uy/j8KSEKXhiLrQRUQk6ZTA6yA2Bl5QXN4Cj1WcZwZ8bCwqrXBMREQkWZTA66AkQRd6WQL3lydw7dcrIiLJpgReB/Fd6EUllbrQAz5+2eoSeFUrsYmIiNSWEngdxBexFeywC11FbCIiklxK4HVQtpRqsStiC/hM2V7grgUeBBJvJSoiIlIXSuB1ECtiKwlF2LQtSF52oGxr0Ay/j3DEAupCFxGR5FMCr4NYAgdYu6W4bA44ULYaG6iITUREki+lCdwYc5QxZoExZpEx5voEP+9ujJlqjPnaGDPbGHNMKuOrqVgXOsDqLcUVWtqx9dBB08hERCT5UpbAjTF+4EHgaKA/cJYxpn+l0/4CvGCtHQKMAsalKr7aiG+Br95cXCFRZ8Ul8PiWuYiISDKksgU+AlhkrV1irS0FngdOrHSOBZpHv28B/JzC+GqsNBwhOuTN+sLSCl3lsRZ4bqYfn894EZ6IiDRiqUzgXYCf4u6viB6LdwtwrjFmBTAJuCLRAxljLjHGzDTGzFy3bl19xFotpaEILXPKp4jFb1oSGwPXRiYiIlIfGloR21nA49barsAxwFPGmO1itNaOt9YOs9YOa9euXcqDjCkNRWidm1l2Pz+uBZ4RS+Aa/xYRkXqQygS+EugWd79r9Fi80cALANbaz4FsoG1KoquF0nDFBJ6oC10JXERE6kMqE/iXQB9jTE9jTCauSO31SucsBw4FMMb0wyVw7/rId6I0FKFVs/IEnpcogasLXURE6kHKEri1NgRcDrwDzMdVm881xtxmjDkhetq1wMXGmG+B54ALrbU2VTHWVLBSCzxRAlcFuoiI1IeUZhdr7SRccVr8sZvivp8H7J/KmOqiJBQhO8NPToafbcFwhdZ2lorYRESkHjW0Ira0UhqKkBXwlSXpvARFbPkaAxcRkXqgBF5L1lpKwxEyA76yxJ2wC10JXERE6oESeC2FIhZrXUu7LIEnWEpVXegiIlIflMBrKRhdB31nLXB1oYuISH1QAq+l2DromX5fWTd5hQTuVxe6iIjUHyXwWipL4AFf2S5kCeeBK4GLiEg9UAKvpuJgmJWbtpXdL4lL4LlZfqDSSmxaSlVEROqREng1Pfn5Uo6+72Ni68rE9gLPCvjo0rIZbfOyKuwB3r55NsZAp5Y5XoQrIiKNnJqH1bShsJQtxSFCEUuG35R1oWf4ffz6gB6cPqxrhfP32qUVn13/Kzq1UAIXEZHkUwKvpliXeWkoQobfV16F7veRFfCTleff7neUvEVEpL6oC72aYl3mpaGKX+O7zUVERFJF2aeagtGEHQwrgYuIiPeUfaop1gKPdaWXhJXARUTEO8o+1VTWdV65Be7XSygiIqmn7FNNlce+g2qBi4iIh5R9qqnKIja1wEVExAPKPtVUUlUXulrgIiLiAWWfaop1mQcrJXIlcBER8YKyTzWVVqo+j1+JTUREJNWUfaqp8th3rEs9Sy1wERHxgLJPNVUe+45fSlVERCTVlH2qqXILvDQUIeAz+HzGy7BERKSJUgKvprIitriWuArYRETEK8pA1bTdNLKwEriIiHhHGaiaEnWhqwJdRES8ogxUDdba7TYzKQ1HVMAmIiKeUQaqhlDEYq37Pr4FrilkIiLiFWWgaoglbag4nUxj4CIi4hVloGqIVZ5DxaVUlcBFRMQrykDVUFULXEVsIiLiFWWgaiiJT+BxK7GpiE1ERLyiDFQNpeHtE7jGwEVExEvKQNUQ34VeEjedTAlcRES8ogxUDSpiExGRhkYZqBqqKmLL0hi4iIh4RBmoGkoTFLGpCl1ERLykDFQNsXHvnAx/xSp0daGLiIhHlIGqIZa0c7MCWolNREQaBGWgaogl8PzsQHkXulrgIiLiIWWgHfho4TrWFZSUVaHnRVvgkYglGLZayEVERDyjDFQFay2jH/+Sp6cvK2t152W5FnhsZbasDL18IiLiDWWgKoQillDEsnlbsGzcOy/ahV5YEgIgPyvgZYgiItKEKYFXIRR2G4AXloQqtsDDEYqiCTxXCVxERDyiBF6FUMQl7aKSUFmXeW6Wv0ILPE8JXEREPJLSBG6MOcoYs8AYs8gYc30V55xhjJlnjJlrjHk2lfHFi2+Bx4rYcrMCBMNK4CIi4r2UZSBjjB94EDgcWAF8aYx53Vo7L+6cPsCfgf2ttb8YY9qnKr7KgtEWeKwLPeAzZAf8BMOWLduCgBsTFxER8UIqW+AjgEXW2iXW2lLgeeDESudcDDxorf0FwFq7NoXxVRCORFvgxaGyRVti8743bY0mcLXARUTEI6lM4F2An+Lur4gei7cbsJsxZpoxZrox5qhED2SMucQYM9MYM3PdunX1EmyFIrbooi1Z0QS+oagUUAIXERHvVDuBG2MuNMackeD4GcaY85MUTwDoAxwMnAVMMMa0rHyStXa8tXaYtXZYu3btkvTUFYUiFavQM/y+ss1LftkaTeDqQhcREY/UpAV+HbAxwfH1QMKCtEpWAt3i7neNHou3AnjdWhu01v4ILMQl9JQLhSuOgWf6y7vQNxaV4jNucxMREREv1CSB9wAWJTi+JPqznfkS6GOM6WmMyQRGAa9XOudVXOsbY0xbXJf6khrEmDSxFri1sHlbkKyAr2zp1F+KSsnNCmCM8SI0ERGRGiXwzUDPBMd7A4U7+2VrbQi4HHgHmA+8YK2da4y5zRhzQvS0d4ANxph5wFTgj9baDTWIMWliY+Dgxrzji9g2FJVqFTYREfFUTbLQZOAeY8zx1tpVAMaYzsDdwKTqPIC1dlLlc621N8V9b4FrojdPxRZyATfm3SInoyyB/7K1VKuwiYiIp2rSAv8TkAssjlWA47rUc6M/a1RiXegAGwtLK46BF5aqgE1ERDxV7SxkrV1njBkCnAMMjR4eBzxnrd1WH8F5Kb4LvaAkRIa/fAy8oCSkKWQiIuKpGmUha20x8Fj01qjFd6EDFcbAQXPARUTEWzWZB369MWZ0guOjjTGNrws9rgUO0QTuVwIXEZGGoSZj4JcACxIcnw+MSU44DUf8GDgkaIFrDFxERDxUkwTeGbfQSmU/s/2SqGkvtpBLTJZfXegiItJw1CSBrwUGJDg+EPBkrnZ9qtwCjy9iAyVwERHxVk0S+ETg/6KV6AAYY4YC/wJeSnZgXttZEZvmgYuIiJdqkoVuBAYDs4wxsTXRWwOfADckOzCvxYrYjHHLqVYuYsvXGLiIiHioJvPAi4CDjTG/AvaKHp5lrf2gXiLzWKwLvUVOBpu2BjWNTEREGpQaZSFjTCugA+AHMoEDjDEHAFhrb0t+eN6JJfCWsQTuVxe6iIg0HNXOQsaY4cDbgAGaA+uA9sBWYBXQuBJ4tAq9RbNM2LCVzICPgM+UdamrBS4iIl6qSRHbPcDLQFtgG7A/sAvwNW6v8EYlHG2Bt2qWAUCm34cxhozoOLjGwEVExEs1SeCDgf+z1kaACJBprV2BS9531kdwXgqGy7vQgbLu86xoAlcXuoiIeKkmCTwMBKPfrwW6Rb9fj2uJNyqxLvSWzTKB8gQe+6oudBER8VJNstBsXCt8ETAduMEY4wMuJvESq2ktVsTWPKe8Cx0oGwvPCtTk2kdERCS5apLA7wDyot//FXgLmIwrZjstyXF5LhSJ4PcZmkfHuuNb4HnZAYwxXoYnIiJNXE3mgb8f9/1SYA9jTGvgF2utrfIX01QoYvH7TFlXeax4LcPvU/e5iIh4rk6ZyFq7cednpadQ2JLhM2XFarEu80y/60IXERHxkpqSVQjHWuAJutCVwEVExGtK4FUIhiNk+H0M6tqSQ3dvT/9OzQE4YVBnAn4lcBER8ZYSeBXCEUvAb2idm8ljFw4vO/7rA3p6GJWISBKUFoGNQEYu+DSjJl0pgVchGLYE9B9bRNKdtfDdyxAJQ/vdYerfYeFk97OW3eHIv0N2c/j5axg2GrLydvx40mAogVchHImoq1xE0ls4BG9dA189UX4sIxcOuBpyWsE3z8H/zin/WXAbHHx96uOUWlECr0IwWsQmIpJ2Nq+Emf+B79+Edd/DgX+AvsfAylmw+7HQoos7b5/fudZ5Zi589RRMfwj2vQyy8r2NX6pFCbwKoXCEDHWhi0hNRCKuK3rdfHe/dS/oOgL8Sfio/ekLePcv8MtSMH7oNhx6HAgZORXP2/QTfPYAhIqh+75wygQYeIb7Wde9Kp7rz4BBo9z3+Z3g0UNh1uOw3xV1j1fqnRJ4FcJqgYtIZcFiWDIVQiWwaRksfBcKfi7/+bZNsK3S8hjZLaBZm+o9fm576HM4tNm1/FjROlj4Dix6D/I7w25HueS85COY91rix+l7DBz1d2jVo/p/W9dh0HMkfHofdBoMPQ+s/u+KJ5TAqxAMWzI0Bi4iAOEg/PAevHsjbFxSfrzjAOg8FGJLKweyoOdB0HW4O/bzNy7hlxZV73k2LIYP/rb98VY9YOQfYf+ryovMIhF38WAjFc/1Z0J+xxr/iQAceSc8fzY8cRyMuASO/kf53yYNjhJ4FdQCFxGKNsDUO+C7l6B4s2sZj3rOJdScVtC8045/v1UP2OOkmj1n4VooWl9+P7MZtNxl+0Tq80GLrjV77J3pOAAu+wLeuxm+eASad4EDrkruc0jSKIFXIRiOEPBrDFykyVm3AKbe6SqyV3wBxVvcGPLux0KfIyGQWb/Pn9fe3bySkQNH3+267t+/BdruBrsf4108UiUl8CqEI7Zs+VQRaSKWz4BnowVfrXpA9/3g0L9C+36ehpVyxsBJ4+CXH2HixTD6Xeiwh9dRSSVK4FUIRizN1AIXaRpKi+CTf7nq7Rbd4LyJNSsAa4wycmDUszD+EHh2FFzyIeRWsxhPUkIZqgrhSIQMjYGLNG7Wwvw34MG9XQLf4xQY/Z6Sd0zzznDWs1CwCib/0etopBIl8CqEwipiE2nUNiyGZ06D/50LWc3hoslwyiNqZVbWZS846Dq34Mu8172ORuKoC70KoYglQ13oIuktVArLpsHqOYB1Vdu77A9fPgrTxoI/C466C4ZfnJzFVhqrA66C+a+7ZVl7HADNWnsdkaAEXqVQOKIWuDROpUVuEZCCVRAuhR8/ccVKR90FvQ5y3cqzX4BP73XTiPodB4PPcXOc60uw2CWIkgLIzHNx1GYu88/fuGlY3fdx639/fI+b/pXIwDPh8NtqP2e6KfFnwEkPwfiDYPKf4NRHvY5IUAKvUii6nahIWrIW3rgSNiyC3ofA+kWw/HO3I1XROgiXlJ/bohv4/PD0qTDsIlj2OayZAx0GuGU737zaFXftd4XbCKPNrtB5SM22oVz2Gcx5yX3tsIdb5SsQXQI0WATT7ncXEfHyO4Op9BxZedDrEOg0aPuf/fgRfPNMxWO7Hg7Dfg277Au+DFg7z53XfT/osX/14xfouCeM/BN8eCf0P8ld2ImnlMCrEApbAmqBS7qa/YJrgbboDh/c7hYd6TkSMvMhp6VbrrPd7m5N7dy2ULwJnjsbvpjgltQ84QEYfK5L0oumuFbXm1eXP35eB9jv97D3GNj4I2z4IXEc1sLs/7nWdUauW797yYduYZR4bfrAOS+7hUQK18AP78DGpds/XuFqt0lH/AVIjC/gVirreSAs/RS67e2WHY1fAKXbCHeT2jnwGvj+Dfd/YZf91JXuMWOt9TqGOhk2bJidOXNm8h/39vc4Yo+O3HnygKQ/tki9Kljtqqrb9XWFWds2ufW4dzbGa63rXk+0H3Q4CJuWu3N+/gq+edYtEZqZD6UFO37cQA6MvBb2vdxNTYqEo48VtwRoy12qPwZdWuT+xsqyW6oALRVWz4HxB7uK/VMneB1Nk2CMmWWtHVb5uFrgVQhF1AJvVDb95MZGA1muC7exru+8ZZWrrA4Vw4kPuq7x6iY1YxInb3BjoG16u+/b7goDTocFk2HuK65F22Uv91yJNO/iWvkxPj+07ln9v6myzNzyWCT1Og5w67J/+Hc3lLHPpVX/20u9UgKvgutCVxV6nYRK3bim8UGrnolbWIVrYdsvVTyAcR/0xg/fPO12ZKoskOW2VOw2wnWhtuxevr1iJOy6W2f+x419xoz8E/zqxjr/eZ4p65Z+Y/uf/fy1K9o66zlo26f+YjDGLa+pJTabpgOugRVfus1dvnnGbZvauhccepO72GsKwkFX07HwHbczXbyRf4TOg+s9BCXwKoQiERWx1UWoFB47HFZ94+633wOOutONdQJEQvDFeJjxsPu+KlnNXZXw+oVucY3MSi3Ebb+4+akx+Z3hsJtdC/TLx2D1bLcz1JHR5/72Wbdgx+7HuEKsdBAshq0bXMX48s9h1hPw03R3sZLVvOK5LbrBqGfS52+T9BTIhHNegrkT4fNxboe2799077tj7vE6uuSLhN2wTSTk9mVfOBl+eB9KNrupiG12rdirF9yWkrCUwKugIrY6+uRfLnkferMrmvrkXnjyxEonGRh6HvQ6OPFjxObwrp0PJ493G0pU7vq21v183Xx3/oyH4JUx7metesLpj7uK2djvdRsBy6fDxDFwxO2usCsjO3l/dzJsXOIuPoLbYMvPrugrFPeB0LwLnPBvN7VLvUTiFWNgz1PdDeCdG+Hzf7ud1HLbwfDRrg4j3RWsccNSq2eXH8ttB/2Ph92Odp9fVQ091bOUJnBjzFHAWMAPPGqtvauK804FXgKGW2uTX6G2E9ba6DQyfTjW2LZNbmz0k3+6ebYHXuOODxzlrtDjr0w7D3ZjaDsy+Kwd/9wY6NDf3cAl+UVT3IId7fttn/BzWrpNGv53Pjx7OuS2hyP+5mKtfG4k7L7W9/ietTDvNVccVrzFFYhhXes6u7m7yOmwhxuK6DgwOoVKF5fSwBx+m+sp+uE9KC107/eLP0ifefaxxsCi99zfETP3VXdRcsTt7j3Zvn+05sP7/JCyBG6M8QMPAocDK4AvjTGvW2vnVTovH7gSmJGq2CoLR1xlvqct8PdvcV2lux7qPtRXzoQRY+CgPzXMD+/SIvj4n+4KPFzqxsOOirs+y2zmkmt98/lhtyN2fE7vX8EfF8HST+DDu1yLfd5rcOpjLs7Vc2DSn2D5Z+781r3clXbfo6D7vskb4wsWuximjXVf/ZluvL/f8e7DYmd7TYs0JD4/nPyw+37VbPjPkW5nt92Pd7Use57q7WdX8WZ3cb9h8fY/K1oLC992syMAAnG9cnnt4cI3XNJuYFLZAh8BLLLWLgEwxjwPnAjMq3Te34C7Ac9Wzg/FErhXY+DrF7mFM9r2dd2nmbnuqu/DO2HzcldAUt9VuNa6Vaym3e/GfboNh9OfcN3P79zgxnx67O+Szpp5bkxo6wYYdBbsdZGbS9yQK1Mzst1c6N6HunH4d26Ax4+BrHw3hzinFRx4rVv8Y+VM+HICTH/QTcfa9TDoPLR8IRFfwHXN76hlHFv97Jel7vVcPt1NwwpuddOfjr0X9rqwYb9mItXVaSCcMsFtRTr1dnds2TQ45p+p/z9evBk+uANmPlZ1vU0g23WFH3AN7Hak28QlDaQygXcBfoq7vwLYO/4EY8xQoJu19i1jTJUJ3BhzCXAJQPfu3ZMeaMjrFvj7N7v/UOe/6q7+wCXUKbe55S2/ftq1BM94CvLaJf/5I2G35vGsx13Ls9UuMPO/8MhI2LLSFYNt+MF1NQFktXA9BXuPcUtYphOfD/b9HbToAq9d7t64B1wD+13uknhMSaFLuAvedouMxBfOxWS1cMU9iRRvdj0TMc27wuCz3evb44CGNw4vUlf9joM/r3Dz/T+4HabdB99NdBf9nQbB7se6C/6a/t+PhGHFTFi/oOpzNixyXflF69zFc3AbDD3fvefiL75jjK9BdInXVIMpYjPG+IB7gQt3dq61djwwHtxCLsmOJRyOJfAk/oP+stRVTHcctOP/KMs+c2NHh/ylPHmDa9kddjPsdYGbPvTBHa7K+7THov8hk3SxEdwGL//GxXDANW5aiDHQ/0R47izXYj39cdcrULLFXVhk5qX/RhD9T4R+J1T9Omblua7tfsdDJOL+9pjgVtdTsnJWxcVJKvx+vuu67zQIMK4l3xCHQkSSyecH/HD4ra4X8afp5cWpb14Fn90PB1xdPrskVOJ6wH6aXnVredsmt3LgDp834FaK676P60UbfHZKpnWlWio/dVcC3eLud24L/oUAACAASURBVI0ei8kH9gQ+NO6DrSPwujHmhFQXsgUj7kO41l3oq2a7Obix+cjz34SXR7spFnkd4YT7XTdNZdbCu3+B/E6w72WJH7tVD7cmdbd9XBHWhF+5qUPH3AN9j65dvPGePxsWT4Wj7oZ9flt+fJf94NrvXc9ALPFkt6j78zUk1U2oPp8rhovJaek+IAafXT9xiTQGg850N3CfdYs/cEv0vn5FxfOyW7i1HTJzEz9OINvNHuk6vOru+OwW7qK5kUtlAv8S6GOM6YlL3KOAsk88a+1moGy5JmPMh8AfvKhCD9W2Bb5pOUy+Hha85VrFZz7t5h1PvdPNyx02GqaPgxcvgtHvuBWN4s2d6FpxJ45zxVQ70m04XD7LjT3PeBiePweO/ScMvaD8P3U45BYYaNWzet1Da793b6pDb6qYvGNiFyQiInVhjBt2+910N20ytqS38bmCt6ayGEwdpSyBW2tDxpjLgXdw08j+Y62da4y5DZhprW0wO8WHYi3wmoyB//SFq7gMlbpq8a+ehPv2dF2q/U9yU5cyc91/2vGHwLNnus0g+p/gxl2D21zleYcBMGhU9Z4ztw0MOdc9/gvnuw0GpvzNdRVZG12VaxPsfpzb/m9nCXjeq4Bxm1iIiNQ3f0bjmCvukZQOXFprJwGTKh27qYpzD05FTImUtcCr04VuLXz7vEuezTvBuS+7aUd7nuoWMxlxsat2jsnv6Ja5fGUMvH0dTL0DRr/nlgrdtBwueKPmVZpZeXD2/9y49YLJroADoO8xbhx92lh46hQX245a9nNfcV3l+R1q9vwiIpJyaV55VD9iVej+nbXAt21yXdfLPoWuI2DUs+VV4d33hnNeSPx7nQfDZTPc9KsnT4SnTnbbJO51kRvbqQ1/BuxxsrtV1nGAK0x79VJXgJZorHft97Duezi6ES6DKCLSCKVf3XwKxLrQM3a2EtvsF1zyPvZf8Ot3aj6lq0N/l/S3bnDLYx7xt1pGvBMDTnOrJM17FT76R+Jz5r4CGNelLyIiDZ5a4AnEutB32gJfONnNiR7+m9o/WbfhMPpdV8lcn1WT+13hduT66C43nanb8PKfbV4J0x9yx9Nl2UMRkSZOLfAEYl3oGTsaAy/eAj9+4pbXrKvOg930sPpkDBz9D7db12u/c8t4ghvDf/Mqt8hIY9xFSESkkVILPIFwWRX6Dq5vFn8AkaBbSStdZDd3c9CfPgX+0dMtdmAtlBa4dcvre3lWERFJGiXwBILhaiyluvBtt9Rmt72rPqch2vVQOOVRt/NVTItubuqbiIikDSXwBMp2I6uqiK20CBa+A32OSM8lRAee7m4iIpK2NAaeQDDsutCrLGKb8jfYttFN+xIREfGAEngCsSr0hEVsyz5zS5eOuAR22TfFkYmIiDhK4AnscCGX92+Flt3gsFtSGpOIiEg8JfAEqlzIpXgLrPgSBpxR9U45IiIiKaAEnkC4qhb48s/Bhmu/3KmIiEiSKIEnEJtGllF5HviPH4M/E7qN8CAqERGRckrgCcQWcvFXLmL78WM371v7YouIiMeUwBMob4HHJfCtG2H1HOhxoEdRiYiIlFMCTyDhQi7LpgFW498iItIgKIEnkHAhl+XTwZ8FXfbyKCoREZFySuAJhBPtRrZ2HrTrC4FMj6ISEREppwSeQMKFXNbMgw57eBSRiIhIRUrgCcS60MumkW3dCIWroX1/D6MSEREppwSeQDhiMQZ8sRb4mrnuawclcBERaRiUwBMIhm3FRVzWzndf26sLXUREGgYl8ATCkUjF8e+1cyG7JeR39C4oERGROErgCQTDloA/QQGbqWJ/cBERkRQLeB1AQxSOWAKxFri1rgt90ChvgxKRJicSibBixQqKioq8DkXqSW5uLl27dsVXee+NalACTyAUiZSvwrb5JygtUAGbiKTc+vXrMcbQt2/fWn3AS8MWiURYuXIl69evp3379jX+ff2PSCAUtuXroK+Z576qgE1EUmzTpk106NBBybuR8vl8dOjQgc2bN9fu95McT6MQitjyncjWRqeQtd/du4BEpEkKh8NkZGR4HYbUo4yMDEKhUK1+Vwk8gVAkbhrZmnnQohtkt/A2KBFpkoyKZxu1uvz7KoEnEArHTSNbO08rsImISIOjBJ5AKGJdEVuoFNYvVAGbiIg0OErgCYTCETeNbMMiiITUAhcR8dDjjz9OIKBJU5UpgSfgWuDGdZ+DEriISA0ddthhXHjhhUl5rDPPPJOVK1cm5bEaE13SJBCMtcDXzAVfANru5nVIIiKNTmlpKZmZmTs9Lycnh5ycnBRElF7UAk9gW2mYZpkBtwJbmz4Q2Pl/MBERcS688EKmTJnCE088gTEGYwyPP/44xhieeeYZjjnmGHJzc/nrX/+KtZaLL76Y3r17k5OTQ69evbjhhhsoKSkpe7zKXeix+9OmTWPo0KE0a9aMvfbaiy+//NKLP9czaoEnMHrz/ey7eQ74CqHP4V6HIyICwK1vzGXez1tS/rz9Ozfn5uOrv5jV2LFjWbJkCZ06dWLs2LEAbNni4r7uuuu4++67efDBBwGw1tK+fXueffZZOnTowOzZsxkzZgwZGRnceuutVT5HJBLhz3/+M2PHjqVdu3ZcffXVnHHGGfzwww9NZry8afyVNVG8maOCUyjMaOcK2Hoe5HVEIiJppUWLFmRmZpKTk0PHjm4Xx+LiYgDGjBnDOeecU+H8O+64o+z7Hj16sHjxYsaNG7fDBG6t5b777mPo0KEA3HLLLeyzzz4sXryYvn37JvtPapCUwCtb8DaZhHi19638+swztAOZiDQYNWkFN1QjRozY7tiECRN49NFHWbp0KUVFRYRCISKRyA4fxxjDoEGDyu537twZgDVr1jSZBK4x8Ers3Ff42bZmc+tBSt4iIkmWm5tb4f6LL77IZZddxplnnsmkSZP4+uuvuemmmwgGgzt8HJ/Ph9/vL7sfW9FsZ4m/MVELPF7xFlg8hcnhQ8nLVuGaiEhtZWZmEg6Hd3rexx9/zJAhQ7jmmmvKji1durQeI2s81AKPt2AyJlzKW+G9ycvWtY2ISG317NmTWbNmsXjxYtavX19li7pv377MmTOH1157jcWLFzN27FgmTpyY4mjTkxJ4vEiIbR324mu7K3lZSuAiIrV17bXX0rZtWwYNGkS7du2YNm1awvPGjBnDeeedx0UXXcSQIUOYMWMGt9xyS2qDTVPGWut1DHUybNgwO3PmzKQ93tfLf+HkcZ/x3wuHc8juNd9gXUQkWebPn0+/fv28DkPq2c7+nY0xs6y1wyofVwu8ksISty+rutBFRKQhS2kCN8YcZYxZYIxZZIy5PsHPrzHGzDPGzDbGTDHG7JLK+ACKogk8N1MJXEREGq6UJXBjjB94EDga6A+cZYypvEvI18Awa+1A4CXgH6mKL6ag2CXwfLXARUSkAUtlC3wEsMhau8RaWwo8D5wYf4K1dqq1dmv07nSgawrjA+K60FXEJiIiDVgqE3gX4Ke4+yuix6oyGpic6AfGmEuMMTONMTPXrVuXxBDjutCVwEVEpAFrkEVsxphzgWHAPYl+bq0db60dZq0d1q5du6Q+d0FJiMyAj8xAg3xpREREgNSuxLYS6BZ3v2v0WAXGmMOAG4GDrLUllX9e34pKQuo+FxGRBi+VzcwvgT7GmJ7GmExgFPB6/AnGmCHAI8AJ1tq1KYytTGGxEriIiDR8KUvg1toQcDnwDjAfeMFaO9cYc5sx5oToafcAecCLxphvjDGvV/Fw9aawJKwELiIiDV5KB3qttZOstbtZa3tba++IHrvJWvt69PvDrLUdrLWDo7cTdvyIyVdYElQCFxHx2OOPP04gUP5Z/OGHH2KMYcWKFTv8PWMMTz/9dJ2f/8ILL+Swww6r8+PUJ1VqVVJYEtIqbCIiDcx+++3HqlWryvb9Tpann366bCvSeGPHjuXFF19M6nMlmzJVJUUlYfLa6mUREWlIMjMz6dixY8qer0WLFil7rtpSC7ySguKQ5oCLiNTBhAkTaNGiBcXFxRWO33333XTv3p1wOMzFF19M7969ycnJoVevXtxwww2UlFQ98ShRF/rUqVMZOHAg2dnZDBw4kKlTp273ezfeeCP9+vWjWbNmdOvWjd/+9rds3ry57DHPO+88wHW9G2O48MILge270K21/POf/6RXr15kZmbSu3dv7rvvvgrP1aNHD2666SauvPJKWrduTYcOHbj66qsJhUI1ewGrSZmqkqKSkJZRFZGGafL1sHpO6p+34wA4+q5qn37GGWfw+9//ntdee40zzzyz7PiTTz7JueeeizGG9u3b8+yzz9KhQwdmz57NmDFjyMjI4NZbb63Wc/z8888cd9xxnHHGGTz//POsXLmSK6+8crvzcnJyGD9+PN26dWPx4sVcdtll/P73v+eJJ55gv/3249///jeXX345q1atKjs/kXHjxvHXv/6VsWPHcsghhzBlyhSuuuoq8vPzGT16dNl5DzzwANdddx0zZszg66+/5pxzzmHPPfescE6yKFPFCYUjbAuGtZGJiEgdtGjRghNPPJEnn3yyLIHPnDmTefPmMXHiRHw+H3fccUfZ+T169GDx4sWMGzeu2gl83LhxtG3blgkTJhAIBOjfvz933nknxx9/fIXz/vKXv1R4nr///e+MGjWK//73v2RmZpZ1le+se/6uu+7iiiuu4JJLLgGgT58+LFiwgDvuuKNCcj7wwAO5/vrry87573//y/vvv68EXt+KSsKAthIVkQaqBq1gr11wwQWccMIJrF27lvbt2/Pkk08yYsQI+vbtC7hu9kcffZSlS5dSVFREKBQiEolU+/HnzZvHiBEjKlSqH3DAAdudN3HiRO677z4WLVrEli1biEQilJaWsnr16moXxG3ZsoUVK1YwcuTICscPOuggxo4dy9atW2nWrBkAgwcPrnBO586d+fHHH6v9d9WExsDjFJZGdyLTGLiISJ0cccQRtG3blmeffZZgMMjzzz/PBRdcAMCLL77IZZddxplnnsmkSZP4+uuvuemmmwgGg0mNYcaMGZx++umMHDmSV155ha+++oqHH34YgNLS0qQ+V0xmZmaF+8aYGl2Y1IQyVZzCYm1kIiKSDH6/n3POOYennnqKXr16sXnzZkaNGgXAxx9/zJAhQ7jmmmvKzl+6dGmNHr9///489dRThMNh/H4/ANOmTatwzqeffkrbtm25/fbby4699NJLFc6JJdz4x6msefPmdO3alY8//pjjjjuu7PhHH31Ez549y1rfqaYWeJyyrUTVhS4iUmfnn38+X331FTfffDPHHXccrVu3BqBv377MmTOH1157jcWLFzN27FgmTpxYo8e+9NJLWbduHZdccgnz589nypQp3HjjjRXO6du3L+vWreOxxx5jyZIlPPnkk4wbN67COT179gTg9ddfZ926dRQWFiZ8vj//+c888MADTJgwgR9++IFHHnmEhx56iBtuuKFGcSeTEngc7QUuIpI8AwcOZPDgwXzzzTecf/75ZcfHjBnDeeedx0UXXcSQIUOYMWMGt9xyS40eu0uXLrzxxht88cUXDB48mCuvvJJ77723wjnHHXccN954IzfccAMDBgzg+eef5557Km5yOXz4cK688krGjBlD+/btufzyyxM+36WXXsptt93GnXfeSf/+/bn77ru566676qU4rbqMtdazJ0+GYcOG2ZkzZyblsd6avYrLnv2Kd64aSd+O+Ul5TBGR2po/fz79+vXzOgypZzv7dzbGzLLWDqt8XC3wOEXqQhcRkTShBB6nIJbANQ9cREQaOCXwOLEWeG5W4kpEERGRhkJNzThnDu/G/ru2IeDXdY2IiDRsSuBxOjTPpkPzbK/DEBEpY61NuN2lNA51KSRXU1NEpIHy+/1JX51MGpZgMFhhOdiaUAIXEWmgWrZsyZo1a+ptKU7xViQSYc2aNbXee1xd6CIiDVTbtm1ZsWIFCxYs8DoUqSe5ubm0bdu2Vr+rBC4i0kD5fD66d+/udRjSQKkLXUREJA0pgYuIiKQhJXAREZE0pAQuIiKShpTARURE0lDabydqjFkHLEviQ7YF1ifx8aScXtv6pde3/ui1rT96bXduF2ttu8oH0z6BJ5sxZmaifVel7vTa1i+9vvVHr2390Wtbe+pCFxERSUNK4CIiImlICXx7470OoBHTa1u/9PrWH7229UevbS1pDFxERCQNqQUuIiKShpTARURE0pASuIiISBpSAhcREUlDSuAiIiJpSAlcREQkDSmBi4iIpCElcBERkTQU8DqAumrbtq3t0aOH12GIiIjUi1mzZq1PtBtZ2ifwHj16MHPmTK/DEBERqRfGmIRbZqsLXUREJA0pgYuIiKQhJXAREZE0pAQuIiKShpTARURE0pASuIiISBpSAhcREUlDSuAiIiJpSAlcRESalkgErPU6ijpL+5XYREREtlOwGtb/AFtWwuaf4JelsPZ72LQMtm6AQA606ALNu0CLru5W+fvMZl7/FTukBC4iIunNWtiwCJZ9Bss/d183VVp9NLcdtNsd+h7jvg9uhc0r3G3tPChcs/3jZreA/E6Q3xHyO7vEnpUHxg/GB75KX43ffd/rYGjeud7/bCVwERFJPxt/hB/ehaWfwPLpULTOHW/WBrrvCyMugQ79oUV319LOyNnx44VKYMvP0RZ7NLEXroGCVbBlFaz/yH1vIzuP7ZyXlMBFRETKrJ4Dc16Ehe/Auu/dsZbdofehsMu+0H0/aNsHjKn5YweyoHVPd6tKOAShYrBhl8gjEfd9JHo/9n1eh9r9fTUNOSXPIiIiUhuhUpg7EWY8DD9/Db4A7LI/DL0AdjsS2vROXSz+APjzUvd8O6EELiIiDc/WjTDzMfjiUShc7cavj/4HDDgdmrX2OroGQQlckqt4s7tKthHodUjturJEpOkqKYTpD8Fn90PJFtc9ftKD7qs+TypQApe6i0Rgyi3w/VuuEjRmyHlw7L/c2JKIyI5EIq7F/dHdriCt77FwyA3QcU+vI2uwlMCl7r56HKaNhd6/gkGjoPNQN43jk3/C+oVw5tOQ1z45zxUshtIiILoIQ9liDNGvxucuGLLyk/N8IlL/CtfBxIthyVTocSCMeg66Dfc6qgZPCVzqZvNKePcm6DkSzp1Y3sW166HuyvmVS2H8wbDP76B9P+h5kCsEqcxat9jChkWuGz4cco9VsAo2LIaNS9xt8wrKkvWOnPIoDDw9mX+piNSHtfPh6VPd4irHj3XFaeoqrxYlcKm9SARev8JNnTj+/u3fdHucDK17wUu/hndvdMdadodBZ7lK0lCJm5Kx7ns3br51Q+LnyWkFrXu7uZ1tekNOtICl8vMZ42L6/N8w63ElcJGGbtln8NwoCGTD6Heh0yCvI0orSuBSe5/dD4unuHHuquZOdhoEV8yCog2w/DOYdr8b4wLAuO7uNrtC36Oh8xBo188lbH8GREJuPmVNK06LN8PU22HTT9CyW53+RBGpJ/Neg5cvdhf1574MrXbxOqK0owQutbN8Bky5DfqfBMNG7/z83DbQ73h3Cxa7BO3z109sA05zCfy7l+CAq+vnOUSk9r55Dl69FLqNgLOe17SwWtJuZFJzWzfCy6PdusAnJOg635mM7PpL3uB6A7qOgG//1yh2HBJpVOa+Cq/9DnodBOe/puRdB0rgUjPWwmuXuZ1+Tv+vW+y/IRp4BqybD6u+9ToSEYn58WN4+TfuAnvUsztfn1x2SAlcambGw7BgEhx+G3TZy+toqjbgNLdd4Mz/eB2JiIDbyvP5c10h6tn/g8xcryNKe0rgUn0rv4J3/+q249vnUq+j2bGcVrDnqTDnJVfUJiLeKd4SrTbPgnNehJyWXkfUKCiBS/WUFMBLF7mq8BMfTI95msN/DcEimP2C15GINF3WwhtXwqblcMaTrupckiJlCdwY080YM9UYM88YM9cYc2WCcw42xmw2xnwTvd2UqvhkJ965AX5ZBqc+mj5FJ132gk6D4ctHVcwm4pWvnnC7iR1yg9vyU5ImlS3wEHCttbY/sA9wmTGmf4LzPrHWDo7ebkthfFKV7yfBV0/CAVel3xtw79+6hWJ+eM/rSESanjXzYPJ1bmOjA67xOppGJ2UJ3Fq7ylr7VfT7AmA+0CVVzy+1VLQe3vg9dBgAB9/gdTQ1N+A0aN4Vpt3ndSQiTUtpEbx4IWQ1h1PGg08jtsnmyStqjOkBDAFmJPjxvsaYb40xk40xe1Tx+5cYY2YaY2auW7euHiNt4mJjV8Wb3RswkOl1RDXnz4B9L4Nl0+CnL7yORpKtYI2rcdj2i9eRSGWT/uQ2Mzp1QvI2M5IKUp7AjTF5wMvAVdbaLZV+/BWwi7V2EPAA8Gqix7DWjrfWDrPWDmvXrl39BtyUff00fP8m/Oqv0CHRaEeaGHq+q0r/6B9eRyLJUrAG3rwa7tvT7WL10AGw9FOvo5KYb/8H3zwNI/8AvQ72OppGK6UJ3BiTgUvez1hrJ1b+ubV2i7W2MPr9JCDDGNM2lTFK1Nr5MOmPbpexfS/zOpq6ycqD/a+CRe/Bj594HY3UhbUw/SG4fwh89RQMPsdtPRnIhMePdZvrFFWxKY6kxi/L4K1roPt+cND1XkfTqKWyCt0AjwHzrbX3VnFOx+h5GGNGROPTuzHVSrdGx67y4JQJ9bvsaarsPQaad4H3blJFeroqLXI72719PfTYHy6bAcffB7sfA2M+gf2ugG+ehQeGwpePQSTsdcRNj7WuZgbglEcSbx0sSZPKFvj+wHnAr+KmiR1jjPmtMea30XNOA74zxnwL3A+Mslaftik3+U+wboEb987v6HU0yZGRA4fcCD9/Bd+97HU0UlNF6+Hx42Deq3DYLXD2C25Fr5isPDjidvjtp9BxgGsBPrS/W3c7EvEq6qbnqydgyYdwxN803zsFTLrnx2HDhtmZM2d6HUbjMfsFN6Z44B/g0L96HU1yRcIw4VdQsAou/7LhruMuFW1aDk+dDJtXwGn/dS3uHbHWJfqpf4f1C6DDnnDQn6DvsWoR1qfCdfDAXtBpIFzwRnos9pQmjDGzrLXDKh9XXb+UW78I3rjKjV0d/Gevo0k+nx+O+z8oWgcf3O51NFIda+bCY0e4f7PzXt158gaXOPY4GX73OZzyKAS3wQvnw9hBrpCxYHX9x90UvX8LBLfCsfcqeaeIErg4wWI37h3IcqutNdaWSpehMPxi+GICLJ/udTSyI+t/cIVpGLjo7ZovIuTzw8DT4bIv4MynoW0fmHoH/N8eWl432X760lWd7/s7aLeb19E0GUrg4rocJ/8R1syBkx+GFo18fZ1D/wqtdoGJl7hNFqThKVoPz5wGxg8XvVW3aYz+APQ7Hs5/Fa74yo2Rv3ezu2iVuouEYdK1kN8JRv7R62iaFCVwgc//7ZZKPfBa2O1Ir6Opf1n5rrp+809uqlya14E0OsFieO4s19V91vPQulfyHrtNbzjsVij4Gb5+KnmP25TNehxWfeuKCLPyvY6mSVECb+qWfOS2CO1/EhzyF6+jSZ1uI+Cg62D289ozvCGJRODV38KKL+DkR6Db8OQ/R8+R0H1f+ORetcLrautGmHIb9DjQbd8rKaUE3tR99oCbKnbyw01vreKRf4Q+R7jNFpZ97nU0AvDxPTD3FTj8NtjjpPp5DmNckWbBzzB9XP08R1Pxyb+gZAsc/Q8VrnmgiX1iSwWbV8Ci92HIuW6edFPj87uu9Jbd4fmzXRW+eGfJh/Dh32HgKNjv9/X7XL0Ogr7HwMf/hC0/1+9zNVabV7pi0IGj0nup5TSmBN6Uff20+zrkPG/j8FJOSzjnRTA+ePoUTTHySsFqePk30HY3OC5F05COvBMiITeEJDX38T1gI3DwdV5H0mQpgTdVkbBbS7rXwa4iuylr0xvOecFVPj9xAhSu9TqipiUccsm7tAjOeAIyc1PzvK17uj3uv3sJvn8rNc/ZWGxY7IoA97oQWvXwOpomSwm8qVr6KWxZ4XbqEuiyl2uJb/4JnjgetqzyOqLGLVgM6xbCwnfhzatg6Sdw7L+gfb/UxnHgH6DjQLcJSsGa1D53OvvwLvBluN3GxDONdLUO2am5r0BGLux2lNeRNBw99oez/+emMD12uEvoqU4ojY21sGUlrPgSVsyEn7+BjUtcAVm8YaNh8Nmpjy+Q6RYuemQkvDIGzn25cWzeU5/WzIU5L8L+VzaevRLSlBJ4UxQOwfzXoe9RkNnM62galp4j4cK33CIi4/ZxLfO+R8NuR0OHPVRpuzPBbfDz19GEHU3aBdHeDH+ma+32PsR1u8ZuLXeB/A7exdyuLxx9N7xxJUy9s/HtAZBsU+908733v9LrSJo8JfCmaOnHsHWDWy9attd5MIz52G1NuWCyWzf9g9shr4Ob79rzQPe1dS8l9NKtsGwa/PgRLJ/hknck6H7Wqgf0OAC6DIOuw6Hjnm6p3oZorwth5Sz45J+uolpzmhNbtxC+fxNG/gmatfY6miZPCbwpmvsKZObBrod5HUnD1byzG98b+QdXIf3Du/Djx/DjJ67oCaBZG5eYuo1wC4N0HgoZ2d7GnQpbN8Kcl2Dh2y55h4pd67rzELcWdrd93OuS187rSGvm6HvcVMKXL3ZLuNbXPPR09tlYCGTD3mO8jkRQAm96wiGY/6brFm6Kc79rI7+jK/Yber4b013/Ayz/HH76wq0YtvBtd54/y3W577Kv29GtdU/X1ZjV3LU807m1bi2s/Aq+fBTmTnRJu00fGPZrdyG4y37p//8pI9vNRnjmdHjp166Xavhor6NqOLb8DN/+D/a6AHLbeh2NoATe9Kz4ArZthN2P9TqS9GSM222p3W7ugwxci3T557DsM/f10/vA/qvi72W1gF4jYdfDoc/hroWfDiIR+OEdt+LWii9dz83gc1zi7rin19ElX1a+K1586dfw1jWw5js48u9No2dlZ2Y8QtVjjQAAIABJREFUAjYM+17udSQSpQTe1CyY7KZ/9D7U60gaj2at3QVR7KKotMiNp25Z5ZaZLNkCG3+ERVNg/hvunPZ7QJ/D3FKu3fYGf4Z38ScSibiW9if3wtq5brW6o++BwWc1/g0rsvLdJipTboVpY91WmadOaNozEkq3wldPwO7HuZ4laRCUwJuaBZNdYVF2c68jabwyc101e2XWwtp58MN7bgnbzx90CSKruVvas/evYJf93WpkXna3L/nQrU62eja02x1OHg97ntLwLjLqk8/v1mPfZX949VJ4+ADY+7duA5ym+N6Z8wJs+8W9BtJgKIE3JesXwYYfYMQlXkfSNBnjpqJ12MOtAFa8xVVvxxJ6rHXeolt5i777fm4/61RYtwDeuREWvQctusMpj7pq7Ka2yU283Y6E381wrfHPH3Tznw+7FQae0XTmi1vrus87DHC1DtJgKIE3JQsnu699tXhLg5DdHPod727WugVOfvwYFr4DM/8LMx6G7BauFdhpMLToAs2jtxZdkrfkaHCbG+P+9D63LsDhf3MXeRr3dfLawYn/hr0ugkl/cNudfnqvmwe9x8mpW/rVK8s+cz1HJ/w7vQsxGyFjrfU6hjoZNmyYnTlzptdhpIcnjnfrff9OW2c2eCWFsPgDN31t2TSX3CvLbgF5Hd2GLJm5sN8Vrhu+JpZ8CG9e7R5/4Cg48g5VGO9IJALzXnG1AWu+c0V9e5zkCvu67dM4eyteudTN/b52gRZ+8ogxZpa1dljl42qBNxWlW2H5dHWfp4usPOh/grsBhErcNJ6y2wr3tWA1FG92U9ueHQXnvpR4/L2y4DZ472b44hG3IM35r7mNbWTHfD43rLDHKW7GwTfPwNxX3c5+zdq417D3r6DXIa6XJN2VFMC8V2HA6UreDZASeFOx7DMIl9a8hSYNQyDLVf9WVQFctAEeP9Yl8bOe3XEyXj3HLVaybj7sfSkcdnP6z+FONWPcePAu+8FRd8OCSW6WwZKp8N3L7py2fd36+l2Hu9Xo2uyafi30ua9AcCsMOdfrSCQBJfCmYvEHbqERFaE0Trlt4PxX4amT4enT4PixMOisiglj2y/w+TiYdh/ktHIbd2g1vrrLynNFbQPPKJ9psPgDWDzVrVg38z/R85q7GQZt+7hbmz4uqee0dBdQgZyGt+DP18+4OLsO9zoSSUAJvKlY/EHjWC1LqpbfES6aDM+fA6/9zhWmDT3fLRwz73WY/hCUbHaFV8f8U2Pd9SF+psF+V7gx8/ULYWV0J7b1C2HJR/Dtc1U9QDSZZ7uvGTmQ38nVJnQalNI/hfWL4KfpcNgtDeuiQsoogTcFW1a57tLBZ3kdidS3nJauJT7vNfhiPLx/s7uBW4Tj4Ouh4wBvY2xKfD5ov7u7xXdDlxTAhkWwYbFb6CdY7LqqQ8WuPiG4rfz7ZZ/Bo4e5eekjxqSuG/6bZ8D4XHGjNEhK4E3B4g/c116HeBuHpIY/Awac5m6/LHX//l2GQaeBXkcmMVn5bvOXzkN2fm7RBtej8vb17sLs+LFuC9T6FAnDt8+7pX+bd6rf55JaS7OKCqmVhW+7bji1vJqeVj3cuuVK3ukrt41b2vWkh2DtfBi3L0y+zrXO68viqVDwMww5p/6eQ+pMCbyxC5W4FthuR2ocSyRdGQODz4YrZrlNdGY87Goc6ss3T0NOa9jt6Pp7DqkzJfDGbumnUFqoN6JIY5DbFo77P1fPMGO8WwMg2bZuhO/fclX1gczkP74kjRJ4Y7fwbTc9pddBXkciIsky8o9uRsEXE5L/2HNecmtGDFb3eUOnBN6YWQsL3naLemj6mEjj0Xmw24r28wfdpjjJ9M3Trl5GdRMNXsoSuDGmmzFmqjFmnjFmrjHmygTnGGPM/caYRcaY2caYoamKr1FaPRs2L9fmJSKN0cF/hm0bkzsWvvo7WPUtDNbKa+kglS3wEHCttbY/sA9wmTGmf6Vzjgb6RG+XAA+lML7GZ/YL4AtAvxO8jkREkq3LUBh0Nkwfl3izm9r45hnwZbi1z6XBS1kCt9austZ+Ff2+AJgPVF7t/0TgSetMB1oaYzQJsTYiYTeW1ecIaNba62hEpD4cepNLuG/9AcLBuj1WqBRm/w/6Hu2mrkmD58kYuDGmBzAEmFHpR12An+Lur2D7JC/VsfQTKFztKklFpHFq3gmO+BssngIvj65bEv/hHdi6QRuXpJGUr8RmjMkDXgaustbWqvrCGHMJroud7t27JzG6RmT2C27zhN00/i3SqA0f7ZZdfecGKC2CUx9zS+rW1NfPuP3lex+a/BilXvx/e3ceJ0dd53/89enuOTJnjjly3wkhXAHCLRpREFGIgC7xBBT5qbCe+/upu49FVpTVXRcRiSC3KKeIS1A0HAIxHIEkhpA7k4tMjjlyzX109/f3R9VMJpOZTCbT09U9834+Hv2Y7qrq6k/XVPe761tV30rqFriZZeCF9yPOuae7mGQHMK7D47H+sEM45+5xzs12zs0uLi7un2LTWUuDd/GK4y/T0ecig8E5N3hdrG5+xes3ffeq3j2/tgI2Pg+nzIOwethOF8k8Ct2A+4G1zrnbuplsAfAF/2j0s4EDzrldyapxwNjwF2ipVfO5yGBy+jVw9bNe5y73zIFXfur9mD8aK58AF1PzeZpJ5hb4ecDngQvMbIV/u8TMvmJmX/GneQ7YDJQB9wJfS2J9A8fK30P+aJj4vqArEZFkmnAufO1NmHkZvHIr3DHLuwb8kc4Vdw7+8TsYe6Z3nXJJG0lrK3HOLQaO2Bm3c84BNySnosOtfnc55Uv+yEXX/gBL12ak+j1Q9gKc/TUIhYOuRkSSLXcEfPIBOOM6+NuPYeH34eVb4cQrvCvUjT/Hu2Jdmx3LoHq91wQvaSVNU6p/tC57hI+U38ee+X9nxGfug6KpQZfUe6ufhnhUzecig92Ec+HaP3sB/da93mmly38DGbkw7kwoPcG7bXze6275hCuCrlh6ybyN3vQ1e/Zst3Tp0oTMKxaL88vbf8QXa+8mPxLDLvh3OPur6bMl6xzc7Tebf2Wxrj4mIge11EPZS7DlVdi+BKo2QKzZG3fyPLji18HWJ90ys2XOudmdh2sLvINwOMQln/sWF98xhfvyH2Hm8/8G2173mqMysoMur2dbXoWKVXDZnQpvETlUZq63b3ym3zNjLAr7tkL1Bhh3VqClybHRxUw6mV6az+knzuS65m/DxT+F9X+Gx67yfr2mujfmQ26xukEUkZ6FI95uwhmXqOe1NKUA78KJowvYWdPMvpO+CJ+4C7Ysgt9eDo37gy6te1XrvX1ZZ3w5PVoLRESkTxTgXZg5ugCAtbtqYNZn4FMPwY7l8NDHoXZ3sMV1Z/Ht3oEoZ3wp6EpERCQJFOBdOH6UF+BrdvnnTs6cC5953Lviz/0XQvXGAKvrwt4tXkcMs6+F3KKgqxERkSRQgHehKC+LkvysgwEOMPXDcM2foLXRC/HtbwVXYGev3e5dNvTcrwddiYiIJIkCvBszRxewZmen3ovGnAZfeh6GDIPfXAqruurOPcn2b/cuQnDa570rE4mIyKCgAO/GzFEFlFXW0RyNHTpi+GT40gsw6hR46lpY+G/eVnlQXv0JWAje963gahARkaRTgHfj+FEFROOOssq6w0fmFsHVf4Izr4c37oQ7TvX6G67a4HWmkixVG2DFo16XiYVjk/e6IiISOHXk0o22I9HX7KzhhNGFh08QyYRL/tu7ZOffbvH6G174fSgcBzM+5v3NLvT6Hu6vS3q+/GPIyIHzv90/8xcRkZSlAO/GxBG55GSGWb2zhiN2izLpfPjiQu8I9S2LYMNCWPrgwS4KF/8c5s6HCecktsDqjbDmGS+8deS5iMigowDvRjhknDi6kJXlR9F5ixmMmOLdZl8L0RaINnoXEXj2m/DQx7wQn/XpxBX4+h0QyYKzvpq4eYqISNrQPvAjOGlsIat31tAai/fuiZFMr/l8ygXeRUUmvg/+9yvw2i8Ss4+8dje88zjM+izkFfd9fiIiknYU4Edw8thCmqNxNlZ0cSDb0cougM/+3rtU3ws3eUetx2M9P+9IXvlP75Kh597Yt/mIiEjaUhP6EZw8digAK8v3tx/UdkwiWXDl/ZBXAm/Oh3efhGkXwfSPwOQPeiF/tN68G5Y9BOfc6J3SJiIig5IC/AgmjsghPzvCyh0HmNfXmYVCcPFPYMJ53sFn6/4EKx6BUAZMPM+7CMnxHz/yPNb9Gf76PZjxcbjwh32tSERE0pgC/AjMjJPHFvJu+YFEzfDg9XhjUdi+BDb8FdY+C098Fk67Gi66xdt/3ln5MnjqS15vcFfcC6FwYmoSEZG0pADvwcljh3Lf3zfTHI2RFUlgaIYj3pb3xPPgQzd553Qvvh1W/cG7nnfpCZA9FJprYOdyL+TzSuDTT0BmTuLqEBGRtKQA78HJYwppjTnW7arllHFD++dFwhnw4Zu9A93e/BW88xhEmw6Ozy6ESR/wptFR5yIiggK8R9NH5gNQVlnXfwHeZtTJcPndMPdXUF8JTTWQlQd5pWoyFxGRQyjAezB+eA7hkLG5ug+nkvVWKAT5I72biIhIF3QeeA8ywiHGD89hS3V90KWIiIi0U4AfhclFuWyuUoCLiEjqUIAfhcnFuWypriceT+KlQkVERI5AAX4UJhXl0RyNs2N/Y9CliIiIAArwozK5OBdA+8FFRCRlKMCPQluAb65K4pHoIiIiR6AAPwrFeVnkZ0XYrC1wERFJEQrwo2BmTCrWkegiIpI6FOBHaXJRrvaBi4hIylCAH6UTRheyY38jGypqgy5FREQkeQFuZg+YWaWZrepm/BwzO2BmK/zbTcmq7WhcefpYsjNC3LNoc9CliIiIJHUL/CHg4h6m+btzbpZ/+2ESajpqw3MzuWr2OJ5ZsYPdB5p6foKIiEg/SlqAO+cWAXuT9Xr94brzJxOLOx58fUvQpYiIyCCXavvAzzGzd8zsL2Z2QtDFdDZueA7nTS3i1fVVQZciIiKDXJ8C3MzyzOxjZjYtAbUsByY4504Bfgn87xFe93ozW2pmS6uqkhumJ44ppKyyjpZoPKmvKyIi0lGvAtzMHjWzr/v3M4AlwLPAajP7eF8Kcc7VOOfq/PvPARlmVtTNtPc452Y752YXFxf35WV7bcbIfKJxxyb1yiYiIgHq7Rb4HOA1//6lQD4wCrgZ+Pe+FGJmI83M/Ptn+rXt6cs8+8PxowoAWLe7JuBKRERkMIv0cvrhQIV//0LgaedchZk9CnzvSE80s8fwfgAUmVk58AMgA8A5dzfwSeCrZhYFGoF5zrmUu37npKJcMsMh1u3W+eAiIhKc3gZ4FTAJKMcL8O/4w3OAI+4Uds59uofxdwJ39rKepMsIh5hakse6XQpwEREJTm+b0H8PPGJmLwIFwAv+8FnAxkQWlspmjMpXE7qIiASqtwH+/4DbgVXAhc65Bn/4aODeRBaWymaMzKeippm99S1BlyIiIoNUr5rQnXNR4LYuhv8sYRWlgRkjDx7Idu6ULg+UFxER6Ve9PY3slI4drJjZJWb2ezO72cx6uz89bc0YlQ+g/eAiIhKY3jah/xo4CcDMxgJPAXnAl4EfJba01FWcl8WQjDA79zcGXYqIiAxSvQ3w44B/+PevAN52zn0U+AJwVSILS2VmRmlBFhW1zUGXIiIig1RvAzwTaLsU1xzgL/79DcDIBNWUFkoKsqmo0VXJREQkGL0N8PXAJ81sPN554C/6w0cB+xJZWKoryc+iUgEuIiIB6W2A/wdwK7AFWOycW+oPv4iDTeuDQmlBNhU1zaRgZ3EiIjII9PY0smf8re9RwMoOo14Cnk5kYamutCCLxtYYtc1RCrIzgi5HREQGmV6f+uWcqwAqzCzbzHDONTnn3uiH2lJaaUE2AJU1zQpwERFJul5fD9zMrjWzMqAOqDOzjWZ2TcIrS3El+W0Brv3gIiKSfL3aAjezbwA/Ae4CXvUHzwF+ZWb5zrlfJra81FVakAVARa0CXEREkq+3Tej/DHzDOXdPh2HPmNk64P8CgybAS/wm9IoanQsuIiLJ19sm9HF4B6x19pI/btDIy4qQmxnWueAiIhKI3gZ4OV6TeWdz/HGDSmlBNpXaAhcRkQD0tgn9LuAOM5sK/N0f9n68pvWbEllYOigpyKJS+8BFRCQAvT0P/Gdm1gh817+Bt+X9L865uxJdXKorLcjmH+/tD7oMEREZhI7lPPD5wHwzy/cfD9prapb6/aE75zCzoMsREZFBpMcAN7Pnexjfft85d1ECakobJflZNEfj1DRGKcxRZy4iIpI8R7MFvqPfq0hTbaeS7a5pUoCLiEhS9Rjgzrlrk1FIOpo5qgCAv6zaxXEj8wOuRkREBpNed6UqB00tyePCmaU8sHgLtU2tQZcjIiKDiAK8j75+wTRqmqI8/Ma2oEsREZFBRAHeRyeNLeSDxxXzwOItuja4iIgkjQI8AeYcV8Ke+haqatUrm4iIJIcCPAEmFuUCsKW6PuBKRERksFCAJ8CkEV6Ab92jABcRkeRQgCfA6KHZZISNLdUNQZciIiKDhAI8ASLhEOOH57Clui7oUkREZJBQgCfIpKJctmoLXEREkkQBniATR+SydU898bhOJRMRkf6nAE+QiUW5NEfj7K7R9cFFRKT/JS3AzewBM6s0s1XdjDczu8PMysxspZmdlqzaEmGSfyrZVp1KJiIiSZDMLfCHgIuPMP6jwDT/dj1wVxJqSpj2c8F1KpmIiCRB0gLcObcI2HuESeYCDzvPm8BQMxuVnOr6blRBNlmRkLbARUQkKVJpH/gYYHuHx+X+sMOY2fVmttTMllZVVSWluJ6EQsaEETk6F1xERJIilQL8qDnn7nHOzXbOzS4uLg66nHZjhg5h14HGoMsQEZFBIJUCfAcwrsPjsf6wtDGyMJsKHYUuIiJJkEoBvgD4gn80+tnAAefcrqCL6o3Sgmyq61poicaDLkVERAa4SLJeyMweA+YARWZWDvwAyABwzt0NPAdcApQBDcC1yaotUUYWZANQWdvE2GE5AVcjIiIDWdIC3Dn36R7GO+CGJJXTL0oLvQCvqFGAi4hI/0qlJvS017YFvvtAc8CViIjIQKcAT6D2ANeBbCIi0s8U4Ak0NCeDzEhIR6KLiEi/U4AnkJkxsiCb3QcU4CIi0r8U4Ak2siBbTegiItLvFOAJVqrOXEREJAkU4Ak2siCL3Qea8M6KExER6R8K8AQrLcimORrnQGNr0KWIiMgApgBPsJGFOpVMRET6nwI8wdrOBd+1v4lKhbiIiPQTBXiClfoB/rVHlnPWf77EhoragCsSEZGBSAGeYCMLszmuNJ/pI/NxDlaWHwi6JBERGYCSdjGTwSIjHGLht95PNBZn5k0L2agtcBER6QfaAu8nkXCIycW5akIXEZF+oQDvR9NL89lQURd0GSIiMgApwPvR9NI8duxvpL45GnQpIiIywCjA+9HUknwAyiq1FS4iIomlAO9H00vzALQfXEREEk4B3o8mjMglMxJio7bARUQkwRTg/SgcMqYU52kLXEREEk4B3s+ml+axYbcCXEREEksB3s9mTxzOzgNNrN6pHtlERCRxFOD97NKTR5EZDvGHZTuCLkVERAYQBXg/G5qTyYeOL+GZFTtojcWDLkdERAYIBXgSXHnaWPbUt/DK+qqgSxERkQFCAZ4EHziumBG5mTz7zs6gSxERkQFCAZ4EGeEQp44fxrrdNUGXIiIiA4QCPEmmlOSytbqBWNwFXYqIiAwACvAkmVKcR0ssTvm+hqBLERGRAUABniRTir1+0TdVqVtVERHpOwV4kkwpzgVgU2V9wJWIiMhAoABPkqE5mRTlZWoLXEREEkIBnkSTi/IU4CIikhBJDXAzu9jM1ptZmZl9r4vx15hZlZmt8G/XJbO+/jalJJfNVWpCFxGRvosk64XMLAzMBy4EyoG3zWyBc25Np0mfcM7dmKy6kmlKcR576rezr76FYbmZQZcjIiJpLJlb4GcCZc65zc65FuBxYG4SXz9wbUeib65WM7qIiPRNMgN8DLC9w+Nyf1hnV5rZSjN7yszGdTUjM7vezJaa2dKqqvTpX7wtwDdUKMBFRKRvUu0gtmeBic65k4EXgN90NZFz7h7n3Gzn3Ozi4uKkFtgXY4cNobQgi5fWVgJQ09RKWaXCXEREei+ZAb4D6LhFPdYf1s45t8c51+w/vA84PUm1JUUoZFx68mhe3VDJgYZWvv3EO8y7542gyxIRkTSUzAB/G5hmZpPMLBOYByzoOIGZjerw8DJgbRLrS4rLZo2mNeb47+fX8eLaCqrrWmhoiQZdloiIpJmkHYXunIua2Y3AQiAMPOCcW21mPwSWOucWAF83s8uAKLAXuCZZ9SXLSWMKmTgih9+9+V77sIqaZiYVJe1fISIiA0BS94E7555zzk13zk1xzv3YH3aTH944577vnDvBOXeKc+6Dzrl1yawvGcyMy04ZDcA5k0cAUFHTFGRJIiKShrTZF4DPnTOBuuYYc2eNZu781xTgIiLSawrwAJTkZ3PTpTOpaWoFoLKmuYdniIiIHCrVTiMbVPKzIgzJCLNbW+AiItJLCvAAmRmlBVlqQhcRkV5TgAespCBbTegiItJrCvCAjSzIpqJWW+AiItI7CvCAtTWhO+eCLkVERNKIAjxgpQXZNLXGqWlUb2wiInL0FOABKynIBmB3TRNPLy9nb31LwBWJiEg6UIAHrDQ/C4BnVuzg20++w5NLt/fwDBEREQV44Er9LfCHXt8KwOYqXV5URER6pgAPWFuAN7TEANhcVR9kOSIikiYU4AEbkhmmINvr0fbsycPZUq0AFxGRnqkv9BQwuTiPYTkZnDNlBG9u3suBhlYKczKCLktERFKYAjwFPHjNGWRGQry+aQ8Am6vrOHX8sICrEhGRVKYm9BQwLDeT3KwIk4tzAe0HFxGRninAU8i4YTmEQ6b94CIi0iMFeArJjIQYPzyHzdU6lUxERI5MAZ5iJhflqgldRER6pABPMZOKctm6p554XBc3ERGR7inAU8zk4jyaWuOc8eMX+fLDS4MuR0REUpROI0sxHzmhlLLKOtZX1PDCmgq2721g3PCcoMsSEZEUoy3wFDMiL4ubLp3Jjz5xEgB/W1cZcEUiIpKKFOApalJRLpOKcnl5vQJcREQOpwBPYXOOK+aNTXto9C90IiLSxjlHc1TfDYOZAjyFXTCjhOZonNc3VQddioikmIWrd3P6LS9yoKE16FIkIArwFHbmpOHkZIZ5ca2a0UXkUG9t2Uddc5S1u2uCLkUCogBPYVmRMBfMKGHh6t20xuJBlyMiKWRjZa3/Vz03DlYK8BQ3d9YY9ta3sHhjNc453i0/wNPLy3lz856gSxORAG3yg7usojbgSiQoOg88xX1gejGFQzJ4ZsUOlm3bx50vlwFgBrdfNYu5s8YEXKGIJFttUys7DzQB2gIfzLQFnuIyIyEuOWkkz727mztfLuOK08aw8Jvv56xJw/n2k+/w61c3sX1vQ9BlppzVOw9w5V2vs7e+JehSRBJuk3+9hOG5mZQpwActBXgauOyUMbTE4pwytpBbLz+J40bmc9/VZ3DGxGH851/Wcf5/vcxTy8r79BqtsfiAOiXl8be2s2zbPv6yalfQpYgk3Ea/2fyimaVU1jbrSPRBSgGeBs6ePJyfX3UK9149m+yMMAB5WREev/4cXv6XOZw5cTj/sWA1uw40tj/Hud5dDOWrv1vOhbctYt8xbrHu2N+YMl8i8bhj4erdAPx11e6AqxFJvLLKOjLDIS6YUeI9rtJ+8MEoqQFuZheb2XozKzOz73UxPsvMnvDHLzGzicmsL1WZGZefOpaS/OzDxk0qyuVnnzqFaNzx3T+8S2NLjHW7a/jQba/y7SdXEIs7NlTUcsdLGw/pEGZjRS03Prqc1TsPsGTzHl5cW8F7exv4xhPec3pjY0UtF/zsFU695Xn+6e432LYn2Muh/mP7fiprm5lUlMsbm/awv0HN6J01tcZ0ZkMa2X2giTU7D54uVlZZx+TiXI4fVQDAxgo1ow9GSQtwMwsD84GPAjOBT5vZzE6TfQnY55ybCvwc+Gmy6ktn40fk8K+XzGDRhirO++nfuOJXr1NV08zTy3fw5YeXcvn817jthQ187v4lbKyo5dEl7zF3/mv8aeUurnnwbW758xpK8rO46eMzWbShimsefItX1lceVZC3RON868kV5GZF+NqcqayvqOXLDy+lvjl62LTxuGNl+X5eWFPRr73LLVy9m4ywccvcE4nGXWDn0UdjcZa/t4+m1sS/16raZlZs339YS0tbKNc0tfL1x/7Bd59aedjrb6mu50P/8yqX/nIx1XXNCa/tWO1vaOl1y1GbeNx1uc51Z/veBm57YcMRjx9ZsnkP9y/e0i//v97Ysb+RT8x/jU/Mf43Xy7xOnTZW1jG1JI8xQ4eQnRHSgWyDVDKPQj8TKHPObQYws8eBucCaDtPMBW727z8F3Glm5o71Uz2IfP6cicwYVcDdr2yisTXGz6+axUOvb+WuVzZx0phCrjpjHD98dg0X/nwRAKeOH8p3LjyOrz2yjFU7arhl7gl87uwJRONx7lm0hWsefJsRuZmcP62IorwshmSGMQAzzPtDU2ucleX7WbWjhrs/dzoXnziSsyeP4AsPLOFTd7+BA8IhmDVuKHvqWli2bR+VtV5g5GaGOWdKEVNKclmzs4aV5Qc4bfxQThk3lH31LVTWNreHS05mhElFuYwdNgTnYO2uGt7aupfRhUM4dfxQCoZkEA4Z9c1RzIwFK3Zy7pQizps6gtGF2Tz0+hYqapoIh4yQQcgMMwMO3dXQdtfh3dm2p4E3Nu9h6JAM3je1iCGZEVpjcX8+RiRk+LM57LlNrXGeWlbOe3sbKMnP4lOzx9IaczS3xoiEQ2SEQ2SEjUgoREbEyAiFMPObJRgLAAANQklEQVTmE3eOmHM45/1AqmuOUtcUpa4lytAhGZjBU8vKaWqNc+KYAi44roSapihvbt7D+opazpw4nOq6ZrbuaSDuHKt3HWBKcR5llXUcV5rP4rJqWmNx9ta3cNWv32DeGeMPW5/a3kd3vLWA9vffPtwfYEA0HmdPXQutMceIvEwywkY07ojGHHHniISMcMjbhnhxbQXLtu1jWkkel54ymqxIqH3aWDxOzDnCoRCZYSMjHKKxNUZFTRPxOLTG4/x9YzV76pq5YEYJZ08eQahzYR3sb2jhvsVbaGiJcc+iTVw1exwZ4RDhkLWvS+t31/LHf+wA4JEl2/jMmeOPOM/uJOKL69El26hvjjJ2+BCu/+0ybrxgKtv3NXDlaWMJhYwpxXm8sr6yfZ3Ky4oQjTtaonEiIfPXt4O1O3fw/+scROOOFdv38872/YwfnkNJQRbrd9eSEQ5x/rQiRhYOOebam6MxNlXWc6CxlWmleRTnZRF3B1/b4fy/3nrf/jlyB4e3fSac/yTX6bnxtvfjvPUvPztCdka4wzycN03H13OHv27b/ONd1EV7LY4hGWHyszMIhQ5dHzo+unBmaVKuImnJykYz+yRwsXPuOv/x54GznHM3dphmlT9Nuf94kz9Ndad5XQ9cDzB+/PjTt23blpT3kG6ccyzZspdZ44aSnRFmZfl+Vmzfz8ljh3LSmELCIWPZtn08+85Ovn/JDLIi3v71lmicl9ZW8Od3d7F06z5qmlppbI3ReVWJhIzRQ4dw+alj+NaF09uHP7JkG/cs2szEEbm0xuK8s30/w/MyOXXcMD4wvZiSgiz+vHIXS7ftY0t1PRNH5DBr3DDe3rqX9/Y2UJAdoTg/i6K8LEJm1Da3srmqngZ/q314biZnTRrOzgNNrN5xgKjfUtAWgAC/mOedYnf7ixu4/cWNx7T8cjPDnDlpOHsbWllZvv+w99+Tk8YUMu/McTz7zk7e3LyXrEiI7Iww0Vic1rijNRY/qnnmZUXIy4qQkxVmb30LNY2tfGLWGE4dP5T7F29h654GcjLDnDSmkBPHFPLyukoaWrwfcXXNUb7z5AqyM8JML81n3e4asjPCPHjNGeytb+HLDy+lpunot1x7KzMSIiNk1PfQ4jKpKJdLThrJ4rI9vLN9/2HjIyFr/z+3GZGbSThkxJ3jrEkjGFWYzTPv7KSqtudWhfOnFfHND0/n3kWbeXFtBZmRELG4oznqtWBkhI3rzp/M7AnD+MGC1ZTva+xhjv0nNzPM/decwYQROcy750227WnADH77xbN437QifvjsGh54bUv7D6Rj+UofM3QIsycOY8e+RqrqmplWkk9DS5SlW/fR0sddLSX5WRQOyWBLdf1h/8NjYYa/EeH9IDe8AYYXtq2xY3+NjvM2vB/7bfNu+wHXFD38u7CzB689gw8eV3LMdRxely1zzs0+bHg6BnhHs2fPdkuXLu3f4uUQrsMv5c6/QnsrFneEQwe3hlti8fYfEh3F447a5ighg9zMSPvrehd0iBP3fxk7By2xePvBfuA1K8firv2XfMw5XJz2n8xtH1rv/sEtyKxIiEjY20Jsa/LPCBtx59UdjR/6xdbxuWZey0GbptYYWZFQ+zQd339rLE5rLE7ccUgrQdtWfudlHI3F2+tq+190nKbtM932WrG4877ozA4b1xKNd/kFfaT/ats3Rtu8Dj4+dIJQyPvxYWY0tcaIxr2t7ojfghGNO+/9x+Pk+9OBd45zyIyI30LRcf1ojXnLKxK2LteTWNxR19MPEoPCIRldjmqOxojHIeJv6R/1PI/wWn3V9sMPvP99fXOMsL+lDQc/A1mREM5BY2vMqz8UIua8VozWDutqx4ACb10dkhE+bN0Eb3k0tR57gEdCRq5fZ0s07n2OrOugtEM+j9Y+Tdu4rurrSlNrjObWOBY6POjNDp1/qEOL4tHOPx531LdEOeS3SKcYzckKt68/idBdgCezCX0HMK7D47H+sK6mKTezCFAIqMuxFGMdPmx9Fe4QPGZdfymDF1Bdfema2SFhbQbZoUPn4TUt9q3OIZldzeDoZ5rdTQHhkBEOhbsd35VIhy+Grv4Xnb+IOi/jjjIjITIj/X8oTFfvL9Ova0in5Zif3XW4mhmZETtiveGQUZjT9fOPRlfrX1/nmUiRcIjCnEPff8fPgBntgQkQwsgIH76Mj1ZWJNztZ7K3krmu9ebz1FuhkHW7jiZbMo9CfxuYZmaTzCwTmAcs6DTNAuBq//4ngb9p/7eIiMjhkrYF7pyLmtmNwEK8TZcHnHOrzeyHwFLn3ALgfuC3ZlYG7MULeREREekkqX2hO+eeA57rNOymDvebgE8lsyYREZF0pJ7YRERE0pACXEREJA0pwEVERNKQAlxERCQNKcBFRETSkAJcREQkDSWtK9X+YmZVQCI7Qy8Cuu26VfpEy7Z/afn2Hy3b/qNl27MJzrnizgPTPsATzcyWdtXnrPSdlm3/0vLtP1q2/UfL9tipCV1ERCQNKcBFRETSkAL8cPcEXcAApmXbv7R8+4+Wbf/Rsj1G2gcuIiKShrQFLiIikoYU4CIiImlIAd6BmV1sZuvNrMzMvhd0PenOzLaa2btmtsLMlvrDhpvZC2a20f87LOg604GZPWBmlWa2qsOwLpelee7w1+OVZnZacJWnh26W781mtsNff1eY2SUdxn3fX77rzewjwVSdHsxsnJm9bGZrzGy1mX3DH671t48U4D4zCwPzgY8CM4FPm9nMYKsaED7onJvV4TzP7wEvOeemAS/5j6VnDwEXdxrW3bL8KDDNv10P3JWkGtPZQxy+fAF+7q+/s5xzzwH43wvzgBP85/zK//6QrkWB7zjnZgJnAzf4y1Drbx8pwA86Eyhzzm12zrUAjwNzA65pIJoL/Ma//xvgEwHWkjacc4uAvZ0Gd7cs5wIPO8+bwFAzG5WcStNTN8u3O3OBx51zzc65LUAZ3veHdME5t8s5t9y/XwusBcag9bfPFOAHjQG2d3hc7g+TY+eA581smZld7w8rdc7t8u/vBkqDKW1A6G5Zal1OnBv9ZtwHOuzu0fI9RmY2ETgVWILW3z5TgEt/ep9z7jS8JrEbzOz9HUc67xxGnceYAFqW/eIuYAowC9gF/E+w5aQ3M8sD/gB80zlX03Gc1t9jowA/aAcwrsPjsf4wOUbOuR3+30rgj3jNjBVtzWH+38rgKkx73S1LrcsJ4JyrcM7FnHNx4F4ONpNr+faSmWXghfcjzrmn/cFaf/tIAX7Q28A0M5tkZpl4B6ksCLimtGVmuWaW33YfuAhYhbdMr/Ynuxp4JpgKB4TuluUC4Av+0bxnAwc6NFXKUeq03/VyvPUXvOU7z8yyzGwS3sFWbyW7vnRhZgbcD6x1zt3WYZTW3z6KBF1AqnDORc3sRmAhEAYecM6tDrisdFYK/NH77BIBHnXO/dXM3gaeNLMv4V0G9p8CrDFtmNljwBygyMzKgR8AP6HrZfkccAnewVUNwLVJLzjNdLN855jZLLym3a3A/wFwzq02syeBNXhHWN/gnIsFUXeaOA/4PPCuma3wh/0rWn/7TF2pioiIpCE1oYuIiKQhBbiIiEgaUoCLiIikIQW4iIhIGlKAi4iIpCEFuIj0OzObY2bOzMYGXYvIQKEAFxERSUMKcBERkTSkABcZBMzsn81snZk1mdlGM/s3M4v447aa2Y/N7D4zqzGzajO71cxCHZ6fb2a/NrMqM2s2s6VmdlGn1ygxswfNrMJ/nfVm9sVOpRxvZovMrMHM1pjZR5Pw9kUGJHWlKjLAmdnNeN1RfhNYARwP3A1kA//uT/bPwO3AGXgX7bgbqAB+4Y9/wB/3OeA94CvAn8zsZOfcOjMbArwKNAKfBTYDU4Hhncr5GfBdYBNed5pPmNkE59y+xL5rkYFPXamKDGBmlgNUA1c45/7aYfgXgDucc0PNbCuw3Tl3fofxtwKfd86NM7OpwEbgY8655zpMsxxY4Zz7ot+f9XxgqnOuvIs65gAvA1e2XY3KzErxrgN9sXNuYaLfu8hApy1wkYHtBGAI8Acz6/hrPQxkm1mx//iNTs97Dfi+mRUAM/1hizpNswg4x79/OrCmq/DupO1iFjjnKswshnfhGxHpJQW4yMDWth/7U8CGLsbvTWItAC1dDNOxOCLHQB8ckYFtNdAETHbOlXVxa7sM5tmdnncusMM5V+PPA+D9naZ5Pwevkb0MmKnzvEWSRwEuMoA55+qAW4FbzewGMzvOzE4ws3lm9tMOk84ys5vNbLqZfQb4BvA//jw2Ab8HfmVmHzGzGWb2C+BE4L/95z+Gd03nBWb2YTObZGYfMrOrkvVeRQYbNaGLDHDOuVvMbBdwI14oN+I1pz/UYbJfAhOApUArcCcHj0AHuA4vrH8HFADvAh93zq3zX6PBzD4A/BfwOJAHbAV+0l/vS2Sw01HoIoOcfxT6fc65HwVdi4gcPTWhi4iIpCEFuIiISBpSE7qIiEga0ha4iIhIGlKAi4iIpCEFuIiISBpSgIuIiKQhBbiIiEga+v9DAjXROrCcngAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "sg.utils.plot_history(direct_history)" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1/1 [==============================] - 0s 946us/step - loss: 2.0196 - acc: 0.4559\n", + "{'loss': 2.0196211338043213, 'acc': 0.4559091031551361}\n" + ] + } + ], + "source": [ + "direct_test_metrics = dict(\n", + " zip(direct_model.metrics_names, direct_model.evaluate(test_gen))\n", + ")\n", + "print(direct_test_metrics)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Comparison of model performance\n", + "\n", + "The following table shows the performance of the two models, for comparison." + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
lossacc
with DGI pre-training1.590.563
without pre-training2.020.456
\n", + "
" + ], + "text/plain": [ + " loss acc\n", + "with DGI pre-training 1.59 0.563\n", + "without pre-training 2.02 0.456" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pd.DataFrame(\n", + " [pretrained_test_metrics, direct_test_metrics],\n", + " index=[\"with DGI pre-training\", \"without pre-training\"],\n", + ").round(3)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "In this demo, we performed semi-supervised node classification on the Cora dataset. This example had extreme data scarcity: only 8 labelled training examples, with one or two from each of the 7 classes. We used Deep Graph Infomax to train a GCN model on the whole Cora graph, without labels. We then further trained this GCN model in the normal manner, to fine-tuned its weights on the small set of labelled data. The GCN model pre-trained with Deep Graph Infomax outperforms a GCN model without any such pre-training." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "nbsphinx": "hidden", + "tags": [ + "CloudRunner" + ] + }, + "source": [ + "
Run the latest release of this notebook:
" + ] + } + ], + "metadata": { + "celltoolbar": "Tags", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/demos/node-classification/gcn-node-classification.ipynb b/demos/node-classification/gcn-node-classification.ipynb index 654ddac7b..b34291b04 100644 --- a/demos/node-classification/gcn-node-classification.ipynb +++ b/demos/node-classification/gcn-node-classification.ipynb @@ -1333,6 +1333,8 @@ "2. built a TensorFlow Keras model and data generator with [the StellarGraph library](https://github.com/stellargraph/stellargraph) \n", "3. trained and evaluated it using TensorFlow and other libraries\n", "\n", + "For problems with only small amounts of labelled data, model performance can be improved by semi-supervised training. See [the GCN + Deep Graph Infomax fine-tuning demo](gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb) for more details on how to do this.\n", + "\n", "StellarGraph includes [other algorithms for node classification](README.md) and [algorithms and demos for other tasks](../README.md). Most can be applied with the same basic structure as this GCN demo." ] }, diff --git a/docs/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.nblink b/docs/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.nblink new file mode 100644 index 000000000..c89920100 --- /dev/null +++ b/docs/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.nblink @@ -0,0 +1,3 @@ +{ + "path": "../../../demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb" +} diff --git a/docs/demos/node-classification/index.rst b/docs/demos/node-classification/index.rst index c342b5cbf..8b55b76f1 100644 --- a/docs/demos/node-classification/index.rst +++ b/docs/demos/node-classification/index.rst @@ -119,6 +119,14 @@ This table lists all node classification demos, including the algorithms trained - yes - - yes + * - :doc:`GCN, Deep Graph Infomax and fine-tuning ` + - GCN, DeepGraphInfomax, semi-supervised training + - yes + - + - + - + - + - yes See :doc:`the root README <../../README>` or each algorithm's documentation for the relevant citation(s). See :doc:`the demo index <../index>` for more tasks, and a summary of each algorithm.