diff --git a/.buildkite/pipeline.yml b/.buildkite/pipeline.yml
index 001d983bc..5b46c2f0f 100644
--- a/.buildkite/pipeline.yml
+++ b/.buildkite/pipeline.yml
@@ -115,7 +115,7 @@ steps:
<<: *timeout
key: "test-notebooks"
depends_on: "runner-3_6"
- parallelism: 51
+ parallelism: 52
command: ".buildkite/steps/test-demo-notebooks.sh"
plugins:
<<: *plugins
diff --git a/demos/embeddings/deep-graph-infomax-embeddings.ipynb b/demos/embeddings/deep-graph-infomax-embeddings.ipynb
index 3395a0044..352d874ca 100644
--- a/demos/embeddings/deep-graph-infomax-embeddings.ipynb
+++ b/demos/embeddings/deep-graph-infomax-embeddings.ipynb
@@ -25,7 +25,9 @@
"source": [
"This demo demonstrates how to perform unsupervised training of a GCN, GAT, APPNP, or GraphSAGE model using the Deep Graph Infomax algorithm (https://arxiv.org/pdf/1809.10341.pdf) on the CORA dataset. \n",
"\n",
- "As with all StellarGraph workflows: first we load the dataset, next we create our data generators, and then we train our model. We then take the embeddings created through unsupervised training and predict the node classes using logistic regression."
+ "As with all StellarGraph workflows: first we load the dataset, next we create our data generators, and then we train our model. We then take the embeddings created through unsupervised training and predict the node classes using logistic regression.\n",
+ "\n",
+ "> See [the GCN + Deep Graph Infomax fine-tuning demo](../node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb) for semi-supervised training using Deep Graph Infomax, by fine-tuning the base model for node classification using labelled data."
]
},
{
@@ -603,6 +605,17 @@
")"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Conclusion\n",
+ "\n",
+ "This notebook demonstrated how to use the Deep Graph Infomax algorithm to train other algorithms to yield useful embedding vectors for nodes, without supervision. To validate the quality of these vectors, it used logistic regression to perform a supervised node classification task.\n",
+ "\n",
+ "See [the GCN + Deep Graph Infomax fine-tuning demo](../node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb) for semi-supervised training using Deep Graph Infomax, by fine-tuning the base model for node classification using labelled data."
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {
diff --git a/demos/node-classification/README.md b/demos/node-classification/README.md
index 3d5826d7f..00d954341 100644
--- a/demos/node-classification/README.md
+++ b/demos/node-classification/README.md
@@ -11,6 +11,7 @@ These demos are displayed with detailed descriptions in the documentation: https
| [Node classification with Cluster-GCN](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/cluster-gcn-node-classification.html) | [source](cluster-gcn-node-classification.ipynb) |
| [Node classification with directed GraphSAGE](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/directed-graphsage-node-classification.html) | [source](directed-graphsage-node-classification.ipynb) |
| [Node classification with Graph ATtention Network (GAT)](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gat-node-classification.html) | [source](gat-node-classification.ipynb) |
+| [Semi-supervised node classification via GCN, Deep Graph Infomax and fine-tuning](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.html) | [source](gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb) |
| [Node classification with Graph Convolutional Network (GCN)](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn-node-classification.html) | [source](gcn-node-classification.ipynb) |
| [(Moved) Node classification with Graph Convolutional Network (GCN)](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn/gcn-cora-node-classification-example.html) | [source](gcn/gcn-cora-node-classification-example.ipynb) |
| [Inductive node classification and representation learning using GraphSAGE](https://stellargraph.readthedocs.io/en/stable/demos/node-classification/graphsage-inductive-node-classification.html) | [source](graphsage-inductive-node-classification.ipynb) |
diff --git a/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb b/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb
new file mode 100644
index 000000000..46181ea56
--- /dev/null
+++ b/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb
@@ -0,0 +1,858 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Semi-supervised node classification via GCN, Deep Graph Infomax and fine-tuning"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "nbsphinx": "hidden",
+ "tags": [
+ "CloudRunner"
+ ]
+ },
+ "source": [
+ "
Run the latest release of this notebook:
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This demo demonstrates how to perform semi-supervised node classification, using [the Deep Graph Infomax algorithm](https://arxiv.org/pdf/1809.10341.pdf) and GCN on the Cora dataset. It uses very few labelled training examples, demonstrating the benefits of pre-training a model with Deep Graph Infomax for data scarce environments.\n",
+ "\n",
+ "> Other related demos:\n",
+ "> - [the GCN node classification demo](gcn-node-classification.ipynb) describes the node classification task in more detail, in a supervised context\n",
+ "> - [the Deep Graph Infomax embeddings demo](../embeddings/deep-graph-infomax-embeddings.ipynb) describes using Deep Graph Infomax in more detail, including applying to algorithms beyond GCN.\n",
+ "\n",
+ "This follows the usual StellarGraph workflow:\n",
+ "\n",
+ "1. load the dataset\n",
+ "2. create our data generators\n",
+ "3. train our model\n",
+ "\n",
+ "We do step 3 three times:\n",
+ "\n",
+ "1. Pre-train a GCN model using Deep Graph Infomax, without any labelled data\n",
+ "2. Fine-tune that GCN model using the small training set\n",
+ "3. Train a fresh GCN model from scratch with the training set (no pre-training)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "nbsphinx": "hidden",
+ "tags": [
+ "CloudRunner"
+ ]
+ },
+ "outputs": [],
+ "source": [
+ "# install StellarGraph if running on Google Colab\n",
+ "import sys\n",
+ "if 'google.colab' in sys.modules:\n",
+ " %pip install -q stellargraph[demos]==1.1.0b"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "nbsphinx": "hidden",
+ "tags": [
+ "VersionCheck"
+ ]
+ },
+ "outputs": [],
+ "source": [
+ "# verify that we're using the correct version of StellarGraph for this notebook\n",
+ "import stellargraph as sg\n",
+ "\n",
+ "try:\n",
+ " sg.utils.validate_notebook_version(\"1.1.0b\")\n",
+ "except AttributeError:\n",
+ " raise ValueError(\n",
+ " f\"This notebook requires StellarGraph version 1.1.0b, but a different version {sg.__version__} is installed. Please see .\"\n",
+ " ) from None"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import stellargraph as sg\n",
+ "from stellargraph.mapper import CorruptedGenerator, FullBatchNodeGenerator\n",
+ "from stellargraph.layer import GCN, DeepGraphInfomax\n",
+ "\n",
+ "import pandas as pd\n",
+ "from sklearn import model_selection, preprocessing\n",
+ "from IPython.display import display, HTML\n",
+ "\n",
+ "import tensorflow as tf\n",
+ "from tensorflow.keras import Model, layers, optimizers, callbacks"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Loading the graph"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "tags": [
+ "DataLoadingLinks"
+ ]
+ },
+ "source": [
+ "(See [the \"Loading from Pandas\" demo](../basics/loading-pandas.ipynb) for details on how data can be loaded.)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "tags": [
+ "DataLoading"
+ ]
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "The Cora dataset consists of 2708 scientific publications classified into one of seven classes. The citation network consists of 5429 links. Each publication in the dataset is described by a 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 1433 unique words."
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "dataset = sg.datasets.Cora()\n",
+ "display(HTML(dataset.description))\n",
+ "G, node_classes = dataset.load()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "StellarGraph: Undirected multigraph\n",
+ " Nodes: 2708, Edges: 5429\n",
+ "\n",
+ " Node types:\n",
+ " paper: [2708]\n",
+ " Features: float32 vector, length 1433\n",
+ " Edge types: paper-cites->paper\n",
+ "\n",
+ " Edge types:\n",
+ " paper-cites->paper: [5429]\n",
+ " Weights: all 1 (default)\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(G.info())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Data Generators\n",
+ "\n",
+ "Now we create the data generators using `CorruptedGenerator` ([docs](https://stellargraph.readthedocs.io/en/stable/api.html#stellargraph.mapper.CorruptedGenerator)). `CorruptedGenerator` returns shuffled node features along with the regular node features and we train our model to discriminate between the two. \n",
+ "\n",
+ "Note that:\n",
+ "\n",
+ "- We typically pass all nodes to `corrupted_generator.flow` because this is an unsupervised task\n",
+ "- We don't pass `targets` to `corrupted_generator.flow` because these are binary labels (true nodes, false nodes) that are created by `CorruptedGenerator`"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Using GCN (local pooling) filters...\n"
+ ]
+ }
+ ],
+ "source": [
+ "fullbatch_generator = FullBatchNodeGenerator(G)\n",
+ "\n",
+ "corrupted_generator = CorruptedGenerator(fullbatch_generator)\n",
+ "gen = corrupted_generator.flow(G.nodes())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Model pre-training with Deep Graph Infomax\n",
+ "\n",
+ "We create and train our `GCN` ([docs](https://stellargraph.readthedocs.io/en/stable/api.html#stellargraph.layer.GCN)) and `DeepGraphInfomax` ([docs](https://stellargraph.readthedocs.io/en/stable/api.html#stellargraph.layer.DeepGraphInfomax)) models. Note that the loss used here must always be `tf.nn.sigmoid_cross_entropy_with_logits`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def make_gcn_model():\n",
+ " # function because we want to create a second one with the same parameters later\n",
+ " return GCN(\n",
+ " layer_sizes=[16, 16],\n",
+ " activations=[\"relu\", \"relu\"],\n",
+ " generator=fullbatch_generator,\n",
+ " dropout=0.4,\n",
+ " )\n",
+ "\n",
+ "\n",
+ "pretrained_gcn_model = make_gcn_model()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "infomax = DeepGraphInfomax(pretrained_gcn_model, corrupted_generator)\n",
+ "x_in, x_out = infomax.in_out_tensors()\n",
+ "\n",
+ "dgi_model = Model(inputs=x_in, outputs=x_out)\n",
+ "dgi_model.compile(\n",
+ " loss=tf.nn.sigmoid_cross_entropy_with_logits, optimizer=optimizers.Adam(lr=1e-3)\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "tags": [
+ "parameters"
+ ]
+ },
+ "outputs": [],
+ "source": [
+ "epochs = 500"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "dgi_es = callbacks.EarlyStopping(monitor=\"loss\", patience=50, restore_best_weights=True)\n",
+ "dgi_history = dgi_model.fit(gen, epochs=epochs, verbose=0, callbacks=[dgi_es])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAEYCAYAAACju6QJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3ib1dn48e/R9t4zTuIMZzrbZLBHKAlQRim7UCgt9AVa1o+3QIEChU5KoW/TAS0FWiirtIyGTSAQskkgO3ESJ7YT721Ztmyf3x969FjySOLEkSL7/lxXLqRHj+U7KvTWOec+91Faa4QQQggRWSzhDkAIIYQQ/ScJXAghhIhAksCFEEKICCQJXAghhIhAksCFEEKICGQLdwBHKjU1Vefm5oY7DCGEEOKoWLt2bZXWOq379YhP4Lm5uaxZsybcYQghhBBHhVJqT2/XZQpdCCGEiECSwIUQQogIJAlcCCGEiECSwIUQQogIFNIErpRaoJTappQqVErd1cvrv1VKrTf+bFdK1YUyPiGEECJShKwKXSllBRYBZwIlwGql1Bta683+e7TWtwXc/wNgRqjiE0IIISJJKLeRzQYKtda7AJRSLwLnA5v7uP9y4Cchik0IIcQh6OzspKqqirq6Ojo6OsIdTsRzuVzk5ORgt9v7/bOhTODDgOKA5yXAnN5uVEqNBEYBH/Xx+vXA9QAjRowY2CiFEEL0qaSkBKUUubm52O12lFLhDiliaa2prq6mpKSEUaNG9fvnj9VGLpcBr2qte/16p7V+EngSoKCgYMAONP94WwUvrNxLi7eDi2bmcNbkTKIc1oF6eyGEiHjNzc2MHz8ei0VqoI+UUoqUlBQqKysP6+dDmcBLgeEBz3OMa725DLjpqEfUzSfbK1lfXEes08atL60HIDvBRYzThtWiOGdKFk1t7RSMTCYtzsnG0npinTbsVgtzRicT47BJwhdCDHqSvAfOkcxghDKBrwbylFKj8CXuy4Arut+klJoAJAHLQxgbALefOY4fnz0Ri1J8vL2CjaUN7K5qptHjpc7t5Tfvbwfgz+zq8z1Gp8Uwd3QK4zPiqGluI85lI85l47QJ6aTHuahtbiPGacNhk/8AhBBCHL6QJXCtdbtS6mbgXcAKPK213qSUeghYo7V+w7j1MuBFrfWATY0fqjhXVxHB6RMyOH1CRtDrFQ0eXA4rq3bVUNHYygljU2jv1DS3tvPBlgosCr4qqefN9ft4obW9x/snRNlp9HiZOzqFP101C7vFIiN2IYQQh0WFIU8OqIKCAn2sHWbS0ampdftG3+7WDsoaPCzdXsneGjc7yptYVVQDQJTdyuOXTae+xYvDamH+pAxincdqWYIQQsCWLVuYOHFiuMM4JjzzzDN897vfpb2954CtPw72mSql1mqtC7pfl2xxFFgtitRYJwBOm5WkGAcTs+LN1z/ZXsnG0nr+tmw3N/x9rXndZbfw47Mn0t6puey4ETI6F0KIATZ//nxycnJ45plnjvi9Lr30UhYuXHjkQR0mSeBhcMq4NE4Zl8aC/Eze3rCfU8en09rewUNvbua+1zcB8NoXpVx7Qi7jM+OYnJ0Q5oiFEGLoaGtrw+FwHPS+qKgooqKiQhBR76SSKozGpMVy8+l55A9LYNbIZH5/xUwWTM7k3nMmsruqmdtf/pKr/rqKvdXucIcqhBAR75prruHDDz/k2WefRSmFUopnnnkGpRTPP/88Z599NjExMdx3331orfne977HmDFjiIqKYvTo0dxzzz20traa7/fMM89gs9l6PF+2bBkzZ84kOjqaWbNmsXr16qPy95ER+DFkeHI0f7pqFgBnT8nio60V/OSNTZz86yVcPW8k3zlhFLmpMWGOUgghgj345iY272sI+e+dlB3PT74++ZDvf+KJJ9i1axdZWVk88cQTADQ0+OL+0Y9+xC9/+UsWLVoE+JqspKen88ILL5CRkcFXX33FDTfcgN1u58EHH+zzd3R2dnL33XfzxBNPkJaWxm233cYll1zCjh07gpL9QJAEfozKToziW3NHMnNEEv9YuYfnlu/hHyv28Pfr5nDC2NRwhyeEEBEnISEBh8NBVFQUmZmZAHg8HgBuuOEGrrzyyqD7H3nkEfNxbm4uO3fu5A9/+MMBE7jWmscff5yZM2cC8MADDzB37lx27tzJ+PHjB/TvIwn8GDcpO55HLsjn6nkjuen5L7j1pfW8fctJZpGcEEKEW39Gwceq2bNn97j21FNP8Ze//IWioiKam5tpb2+ns7PzgO+jlGLatGnm8+zsbADKy8sHPIHLGngEUEoxITOe318xk/oWL3e8/CWdnZG9/U8IIY4lMTHBy5OvvPIKN910E5deeimLFy9m3bp13H///Xi93gO+j8ViwWrt2kHk77R2sMR/OCSBR5CJWfHcd+4kPtleyWVPruCNL/eFOyQhhIgoDofjkE5RW7p0KTNmzOD2229n1qxZ5OXlUVRUdPQD7AeZQo8w35ozgn11LSzesJ8f/nMdn2yrRGvNLy6aKu1ZhRDiIEaNGsWSJUvYuXMnCQkJfY6ox48fz1//+ldef/118vPzeeutt3jttddCHO2Byf/jRxilFD9aMIEPbz+FMyak868vSnhtXSmfbD+802yEEGIoueOOO0hNTWXatGmkpaWxbNmyXu+74YYbuOqqq7j22muZMWMGK1eu5IEHHghtsAchrVQjWJ27jedX7uWJD3fgtFp48uoC5o1JCXdYQohBTFqpDrzDbaUqI/AIlhjt4KbTxvK9k0bR2NrO9/+xluZeDlERQggx+EgCHwTuPGsCL10/l/oWL49/sJ1In1URQghxcJLAB4k5o1O4eFYOT326m7tf2yDbzIQQYpCTKvRB5FffnEp6vJNFS3ZywthUvj4tO9whCSGEOEpkBD6IKKW448zxpMc5+cXbW3n03W20tQ988wAhxNAmy3QD50g+S0ngg4zFopg/KYPSuhZ+v6SQB9/cFO6QhBCDiN1up6WlJdxhDBper/ewDzmRBD4IXXN8LvNGp3DmpAxeXlNMo+fArf+EEOJQpaenU1paitvtlpH4Eers7KS8vJyEhITD+nlZAx+ExmXE8c/r57JyVzXvby5nWWEVC/Kzwh2WEGIQiI+PB2Dfvn0H7QsuDi4mJobU1MM7YVIS+CA2c2QScS4bH2ypkAQuhBgw8fHxZiIX4SNT6IOY3WphYX4mb321j6qm1nCHI4QQYgBJAh/kbjhlDK3tnTz7eVG4QxFCCDGAJIEPcmPSYjkpL43X1+9jd1Uz97++UbaWCSHEICAJfAg4d0oWe2vcnPboxzy3fA9fltSFOyQhhBBHSBL4EPC1yRmkxznN51vLGsMYjRBCiIEgCXwISIx28Mmdp/HhHacQ77KxdX9DuEMSQghxhCSBDxFRDitj0mKZkBUvI3AhhBgEJIEPMRMz49i6vwGPtyPcoQghhDgCksCHmLMmZ9Lc1sG/15WGOxQhhBBHQBL4EDNvTApThiXw1NJdeDs6pZexEEJEKEngQ4xSihtOGc2uqmbyfvw2f166K9whCSGEOAySwIegBZMzyU2JBuAvn0oCF0KISCQJfAiyWS3856YTuGvhBKqa2iisaAp3SEIIIfoppAlcKbVAKbVNKVWolLqrj3suUUptVkptUkq9EMr4hpLEaAcXTB+GRcGLq/aGOxwhhBD9FLIErpSyAouAhcAk4HKl1KRu9+QBdwMnaK0nA7eGKr6hKDPBxQUzhvGPlXsornGHOxwhhBD9EMoR+GygUGu9S2vdBrwInN/tnu8Bi7TWtQBa64oQxjck3XJGHg6rhcufWkFTa3u4wxFCCHGIQpnAhwHFAc9LjGuBxgHjlFLLlFIrlFILensjpdT1Sqk1Sqk1lZWVRyncoWFkSgxPXl1ASW0LL68uPvgPCCGEOCYca0VsNiAPOBW4HHhKKZXY/Sat9ZNa6wKtdUFaWlqIQxx85o5O4bjcJJ5dXhTuUIQQQhyiUCbwUmB4wPMc41qgEuANrbVXa70b2I4voYuj7KzJmeypdlPT3BbuUIQQQhyCUCbw1UCeUmqUUsoBXAa80e2e/+AbfaOUSsU3pS4blUMgLyMOgB3lctCJEEJEgpAlcK11O3Az8C6wBXhZa71JKfWQUuo847Z3gWql1GZgCXCn1ro6VDEOZXnpsQBslz3hQggREWyh/GVa68XA4m7X7g94rIHbjT8ihLISXMQ6bSz6qJAZwxPJH5YQ7pCEEEIcwLFWxCbCRCmFy26lrMHDjc9/Ee5whBBCHIQkcGF6+ILJKAV7a9zUt3jDHY4QQogDkAQuTAvys3jp+nkAfF5YFeZohBBCHIgkcBFkxohE0uKc/Pq9bTR6ZBQuhBDHKkngIojdauGJS6ezq7KZf6/rvk1fCCHEsUISuOhh3pgUMuKdrCmqpbW9g1pp7iKEEMccSeCiB6UUBSOTWbunlsueXMGMn74f7pCEEEJ0Iwlc9GrWyCRK61pYt7cOAHebnFQmhBDHEkngolcL8jNx2bv+9SitbQljNEIIIbqTBC56lZ0YxZp7z+ThC/IBKK2TBC6EEMcSSeCiT7FOG2dMTAd8CbzB4+WGv6+huMYd5siEEEJIAhcHlB7nwmZRlNa28PyKvby7qZxnPy8Kd1hCCDHkhfQwExF5rBZFZoKLfXUtaHzT6PFR9jBHJYQQQhK4OKiRKdGs2l1De6cGoEb2hQshRNjJFLo4qNvPHE9di5c6t6+1amVja5gjEkIIISNwcVCzRibxwe2n4LRZuPH5LySBCyHEMUBG4OKQZCdGkRLrJC3OSVWTJHAhhAg3SeCiX9LinDICF0KIY4AkcNEvqbFOGlvbaWnrCHcoQggxpEkCF/2SFucEYNO++jBHIoQQQ5skcNEvc0YlE+eycd2za/B4ZRQuhBDhIglc9MvIlBjuP3cS9S1eyuo94Q5HCCGGLEngot+yE6MA2F/vodNo7iKEECK0JIGLfstMcAGwdk8N0x58j0VLCsMckRBCDD2SwEW/ZRkJ/NH3ttPY2s57m8vDHJEQQgw9ksBFv0U7bETZreZziwpjMEIIMURJAheHpcWoQI9xWKlokMYuQggRapLAxRFZOCWLikYPWksxmxBChJIkcHFYvnviKFJjnUzOjsfboak1TioTQggRGpLAxWG599xJrLl3PulxvoK2ikbZEy6EEKEkCVwckYx4X2vVzwurZRpdCCFCSBK4OCIZ8b4R+ENvbebdTbKdTAghQkUSuDgiGfEuJmTGAfD7JTt4bnmRjMSFECIEQprAlVILlFLblFKFSqm7enn9GqVUpVJqvfHnu6GMT/Sfw2bhnVtPZsHkTDaWNnD/65tYsasm3GEJIcSgF7IErpSyAouAhcAk4HKl1KRebn1Jaz3d+POXUMUnjswlx+WYj//0yc4wRiKEEENDKEfgs4FCrfUurXUb8CJwfgh/vziKTp+QwfaHF/I/p47hs8IqGjyyrUwIIY6mUCbwYUBxwPMS41p3FymlvlJKvaqUGt7bGymlrldKrVFKramsrDwasYrD4LBZOG18Oh2dms8Lq8IdjhBCDGrHWhHbm0Cu1noq8D7wbG83aa2f1FoXaK0L0tLSQhqgOLAZIxKJc9r4ZLt8sRJCiKMplAm8FAgcUecY10xa62qttb+x9l+AWSGKTQwQu9XCzJFJrC+uD3coQggxqIUyga8G8pRSo5RSDuAy4I3AG5RSWQFPzwO2hDA+MUAmZMVRWNGIt6Mz3KEIIcSgFbIErrVuB24G3sWXmF/WWm9SSj2klDrPuO2HSqlNSqkvgR8C14QqPjFwJmTG4e3Q7K5qDncoQggxaNlC+cu01ouBxd2u3R/w+G7g7lDGJAbehMx4ALaWNTIuIy7M0QghxOB0rBWxiUFgTFosVovih/9cx6c7pJhNCCGOBkngYsA5bBYeu2QaI1Oiufc/G2lrl7VwIYQYaJLAxVFx/vRhPHjeZPZUu/nJG5vCHY4QQgw6ksDFUXPq+HSuPSGXf67ay766lnCHI4QQg4okcHFUnZzna7RT1uAJcyRCCDG4SAIXR1VanBOAqsZWtNac8ZuPeX7lnjBHJYQQkU8SuDiqUmN9CbyyqZXq5jZ2Vjazbm9dmKMSQojId0QJXCkVq5Q6RymVN1ABicElJdYBQFVjG6W1vnVwWQ8XQogj168ErpR6QSn1Q+OxHViJ7wCSTUqpc49CfCLC2a0WkqLtVDZ5KK2TBC6EEAOlvyPwU4FlxuOvA3FAFvAAcN+ARSUGldRYJ5WNrV0j8HoPWuswRyWEEJGtvwk8GSg3Hp8JvKa1LgdeACYOZGBi8EiLc7J0exX/+qIEgLb2Tqqb28IclRBCRLb+JvBKYJTx+ExgifE4GpB2W6JXLruVFm8HW8sazWsyjS6EEEemvwn8FeB5pdQHQDzwvnF9OrBjIAMTg0e0w9rjmn86XQghxOHp72lk/wuUACOAO7TWbuN6NvDUQAYmBo+ffH0y35o7EotSOG0WLv7TctbuqWXhlKyD/7AQQohe9SuBG2d6P9bL9UcHLCIx6KTFOc2GLgBzx6Twt8+LyExwUZCbzPThiWGMTgghIlN/t5FNU0pNDnh+tlLqFaXUA0qpkJ4tLiLXGRPS6ejUPPzfLXz76VVSkS6EEIehv2vgfwamACilcoBXgVjge8DDAxuaGKy+MXMYPzwjj8tnj6C+xUtDS3u4QxJCiIjT3wQ+HlhnPP4GsFprvRC4Grh0IAMTg1ecy87tZ47j5LxUAErq3Af5CSGEEN31N4E7AP+xUqcCbxuPtwOZAxSTGCKyE6MAqUgXQojD0d8Evg34plJqBL594B8Y17OA2oEMTAx+w5J8CVz2hAshRP/1N4E/CPwM2A18prVeY1z/Gl1T60IckpQYBy67hQff2szH2yrCHY4QQkSUfiVwrfXr+PaAzwLOCXjpQ+DOAYxLDAFKKTzeTrSGG5//ItzhCCFEROn3caJa63Kt9XrAoZRyGdeWa603D3h0YtCblpMAgNWiwhyJEEJEln4ncKXUtUqpQqAJaFJK7VBKXTPgkYkh4bnvzOHGU8fQ6Gmnqqk13OEIIUTE6G8jl1uAPwBvABcZf94C/qCU+sHAhycGu4RoOyeM9W0n27SvIczRCCFE5OjvCPwHwC1a69u11q8bf24DbgNuGfjwxFAwITMOgG8/vYql2yvDHI0QQkSG/ibw4fgK1rr70HhNiH5LiXXyk69PAuDVtSVhjkYIISJDf/uXl+Br4LKz2/VTjdeEOCzXnjCKbWWNvPnlPjzeDlz2nkeQCiGE6NLfEfgfgd8ppX5uHGRytlLqF8AT+NbGhThsp01Ip7mtg837ZS1cCCEOpr/HiT6qlGoBfmT8Ad/I+/9prf840MGJoSUrwQVARUMrLW0dRDlkFC6EEH3p9xGgWutFwCKlVJzxvHHAoxJDUmqs78zw7/9jLQCr7jmD+Ci7TKcLIUQvDprAlVLvHeR187HW+msDEJMYopJjHEHPZ//sQy4tGM4vvzm1x71Lt1cya2QSMU45hl4IMTQdyhp4aT/+CHHYXHYrca7ghPzSmmI6OjXtHZ3moSeVja1c/fQq/vWF1E0KIYaugw5ftNbXDtQvU0otwFfwZgX+orX+RR/3XQS8ChwXcGCKGALSYp00etpZmJ/JzBFJPLJ4C2uKavi/jwr5rLCKb88byb5634m25Q2eg7ybEEIMXiGbf1RKWYFF+I4hLQFWK6Xe6N5D3VhbvwVYGarYxLHDbvVNCs0amcSlxw3n/z7awbXPrMbd1gHAy2tKaPH6Hlc3tYUtTiGECLd+90I/ArOBQq31Lq11G/AicH4v9/0U+CUgw6shqLXdl5yHJUYR57Lz8vfnkZ+dYL7uT94A1c2SwIUQQ1coE/gwoDjgeYlxzaSUmgkM11r/90BvpJS6Xim1Rim1prJSWm8OJq3tnQBkJ0YBMCEznpe/P49/33h8j3ur5fATIcQQFsoEfkBKKQvwGHDHwe7VWj+ptS7QWhekpaUd/eBEyPi3kmUae8L90uKcPe6tkRG4EGIIC+UenFKC+6XnEFy5HgfkAx8bW9MygTeUUudJIdvQ8aerZvHhlnIy4oMTuD+xB5IpdCHEUBbKEfhqIE8pNUop5QAuw3csKQBa63qtdarWOldrnQusACR5DzHDEqO4el5uj+suu5X4blvMGj3t5pq5EEIMNSFL4FrrduBm4F1gC/Cy1nqTUuohpdR5oYpDRK7eptFrm71hiEQIIcIvpG2stNaLgcXdrt3fx72nhiImETnS4pwU17RgtSjiXDYqGlupamrtsV4uhBBDwTFTxCbEweQkRZOTFMWfr5rFwxfkA/Dwfzfj7egMc2RCCBF60khaRIwfLZhAo8fL6LRYAG4/cxyPvb+dHeVNTMqOD3N0QggRWjICFxEjLc5pJm+AU8b5thCW1Lr5xdtbeX7lnnCFJoQQIScjcBGxhiX5mr0UVjbxp092AnDlnJHhDEkIIUJGRuAiYqXEOHDZLfxnXVc7gf+sK6W4xs2ra0vQWocxOiGEOLpkBC4illKKYYlRbC9vMq/d+tJ68/HMEYnmlHt9i5dFSwq542vjcNqsIY9VCCEGmozARUTrbW+439ayRvPxpzsqeXLpLr4srgfg2c+LWL6z+qjHJ4QQR4skcBHR/P3Q545O7vHa1v0N5uNa475adxsNHi8/eWMTlz+1IjRBCiHEUSBT6CKiPXR+Pu9sLCMvI5YVu2qCXtsSMAKvMTq21Ta38XlhVUhjFEKIo0ESuIhoc0enMHd0Cm9v2B903aJg876AEbjbPwL3sr64DoDUWEfoAhVCiAEmU+hiUEiMDk7GC/IzKa1robDCV+Dmn2qvaPTw3uZywHeamXRxE0JEKkngYlBIjvEl8Kk5Cbx0/VzuO3cSAO9s9I3M/SPwN7/cR01zG/MnpqM1lDd4whOwEEIcIUngYlBIirYDkB7nYs7oFLISopgxIpG3N5YBXSPwqqY24l02Lp89AoB9dR72Vrv5+/Ii2TcuhIgoksDFoOCfQk+P79pWtjA/k037Gthb7Tar0AGm5iQyMiUagP31Lfxs8Rbue30TywplW5kQInJIAheDgsNm4ZypWWZ/dICF+VkAvL1xPzXurgQ+MSuOrARfG9bSuhbz+h8+Luz1vbeWNbCnuvlohC2EEIdNqtDFoLHoiplBz4cnR5M/LJ5/ryvF4+0qVpuUHU+M00ZClJ39dR4qm1oB2Baw7SzQgsc/BaDoF+ccpciFEKL/ZAQuBrWF+VlBHdkA8tLjAMhKcLG/voWSWjfga7da3uChwePlqaW7KKv30Nkp6+JCiGOTjMDFoLYwP5Nfv7sNgN9dPoN3N5UxIdOXwLMToyiqdlPR2EqU3UqLt4M5P/vQ/NlHFm/h4lk55nOtNV/srWXmiCSUUqH9iwghRDcyAheD2ui0WH79zam8e+vJnDctm0VXzMRm9f1rn53oorCiCa1hcnZ8j5+Nd9l4ZW2J+XzV7hou+uNy1u6pDVn8QgjRF0ngYtC7uGA4441RdyB/IRv41sUDDUuM4m/XHhd0bUOp7yCUvTVu81pFowePt2MgwxVCiEMiCVwMWdmJLvNxQW7wYSjjM+OYnJ0QdG3Lft9aekltC/UtXjo6NSf9cgkT7nuHCmkII4QIMUngYsjKiPMl8HEZsYxOjQl6bUJmHC578LnhW4zTzR57fztf++0n1Ld4aW33Vbd/tLUiBBF32V3VzNVPr6LR4w3p7xVCHDskgYsha+bIJK6aO5JnvzObRKOTG8DJ49K4cMYwAP52zXHceOoYwLcf3K+8oZWNxpQ6QEMfibSzU+Nuazefd3Rq/r68yJx2b25tp6Wt7yn4PdXNvXaIu/u1r1i6vVLW44UYwiSBiyHLZbfy0wvyyUqIIiGqK4H/7rLp5GX41sxPm5DOD07PA6D7jrLlu7o6tzW0tNObh/+7hUn3v2semvLupjLue30Tj3+wA4Dv/2MtP/7Phl5/dn99C6f8+mN+tnhLj9d2lPsOabFapBpeiKFKErgQQKzThtWisFoU8S570GtRDqt5WEqg5Tu7Enh9S+8j8KeX7Qag0eNL8P7BtP+UtOIaNzuNx91VNvoazDz16e6g62v31FBttIZtbu39i4MQYvCTBC4EoJQi3mUjOcaBpZdR7ZRhCT2u+c8VV8qXbP1r5L3xr1W3d/pG4jXNvuTc1NpuJuru6txdXwqKqnytXJtb27noj8sD3lcSuBBDlSRwIQyJ0Q5SehlpA0wfnhj0PC891nw8MjmadzaVsfCJT9kX0Fs9cHuZP9H6/1nR2Mr++haaWtupamrrdZ27LmBUv7/eV+Xu792eGe8rwGvqxwi8o9t6vBAiskkCF8KQEe9kWGJUr6/595EfPyaF//7wRE6fmG6+Niyp62fW7fWNyrXWPL9yr3m9ewIvqW1h3s8/wuPtpK2js9c19PqAA1j8XwZKa30J/LeXTge6ptB//e5W3jGOTu3L35bt5vRHP5H2sEIMEtJKVQjDY5dM77Mo7PgxKcS5bNx46lgmZyewaV/XdHliVNeofc2eGnJTo6lze/npW5vN6/6Rcl/bviqbPCREB6+9B06hu41K9RJjBD4qNQaHzUKj8b6LluwEYPfPz+6zzWtxjZuyBg/ljZ6gJjZCiMgkCVwIQ3Yfo2/wTa9veOAs8/mYtK594/FRXf8ZrSmq5b1N5eb+8EcuzOfH/97IXf/6ig82Z+Cw9T7pVdHYytj04G5xgVPoLQEjcLtVkR7nJNZpo7m13axwB1i7p7ZHUxq/ZuNLwO6qZkngQgwCMoUuxGEYndq1Bh4fsAVtR0Uj5Q0eqowjSvONbm7VzW28tKaY/fUeRiRH89dvFwS9n7+QrcHjZUe5r+NbfYvXTPgtxtp1aV0LWQlRWCyKWKeNJk87DQGJfnVR3/vC/fvNi6rcfd4jhIgcksCFOAxJAcVusY6uEbjH20l7wBrz6LTgDm8fbCknzmULahwDvgRe3uBh6gPvceZvl9LZqalze8lK8BWrdY3A3eQYa+4xThtNre1BW9j6aigD0Gx8Cdhd1fu2NSFEZJEELsRh+uVFU/j7dbPNKewRydFBrydF24nrtqccIM5lIyEquNq9sqmVV00+m1QAACAASURBVANOPqtsaqW+pY0Mo9rcvwZe3tBqVqDHGQk8cKq9oY/96IHvsVtG4EIMCiFN4EqpBUqpbUqpQqXUXb28/n2l1Aal1Hql1GdKqUmhjE+I/rj0uBGclJdmJsaRKcEJ3J98u4tz2Xsdga/b2zX9XVzjps7tJSXGgdNmMUfgjR4vcS7fiD/W1XMEfqB94eYUenXzof4VAV872Jue/4LVRTX9+jkhxNEVsgSulLICi4CFwCTg8l4S9Ata6yla6+nAr4DHQhWfEIfrzEkZAFxSMDzoelqcM+i5f4TuG4F3JfD0OCeVja2sL65jxgjffvPiWjd1LV4So+1EOay0tHWgtaa5rYMYpy+BxzhtNLd2UG9Uq8c5bYc0hb632k1HP7aS1bjb+O+G/Xz32TWH/DNCiKMvlCPw2UCh1nqX1roNeBE4P/AGrXVgK6sYQDasimPenNEpFP3iHE7OSwu63n0EPm90CuAbCdutXf/p5abGsL64jqqmNs6dmg3Anmo39W4vCVEOou2+BN7a3klHpzYTeKzTRqOnawQ+PDn6gFPoLW0d2CyKto7OoIYzvakL2IPeZlTUB1a7+1U0ePjmHz+npFam5YUItVAm8GFAccDzEuNaEKXUTUqpnfhG4D8MUWxCHLH4KFvQNrGM+OAR+JzRvu1dFd1ap45KiTGnvmeNTCIj3slnO6po6+hkdFoMLocVt7fD3EseayZwK02tXnO/eE5SFA3dptDvfm0Db2/Yzwsr99LQ4mWs0UFul9GadVdlEw++uSloRP7B5nKmP/S+OWXun74PTOBFVc2U1LpZuqOKNXtq+TygL7wQIjSOuX3gWutFwCKl1BXAvcC3u9+jlLoeuB5gxIgRoQ1QiD4o5duf7e3o5KKZOeZo2m/miCSgZ2IPfD46LYbhSdGsMY4JnZQVT7TDiqetw+y61jUCt+PxdlLT3EqMceCKvz87QE1zG/9ctZd/rurqCDc5O4GtZY0UVTVzyrg07nz1K9buqeUbM3KYkuPb8rbN2Mb236/2c1xusrl27u3oSvJ3vvolCVF2cpJ8ywI7K6WyXYhQC2UCLwUCFwlzjGt9eRH4Y28vaK2fBJ4EKCgokGl2ccxIj3PS2t7J/y6Y0OO13NQYnr6mgFkjgxutBK6Vx7vsTB+eaCbwcRlxRNmtuNsCR+BWALOYrbi2hcRoB/FRdnMkv3ZPLS8EtHI1Y0iJJsZhZbcxArcaXdtW7q7GYvEleP8If0dFI4UVjWw3EnqgqqY2Ojo1NcapaDsr+lcYJ4Q4cqFM4KuBPKXUKHyJ+zLgisAblFJ5WusdxtNzgB0IEUFumT+O9m5rxZ/ceSrlDb5p89MnZJjXE6Pt1Lm9ZgL3J86LC4bzl898R4g6bBaiHDaWbq/koTd9rVn9I3D/EadF1c3ER9mJd9lo8XbwjxV7eOitzebadaAoh5WRKTFmJXqs8SXg4f/6zhwv+sU55jr6ssJq5j+2NOjnL3tyOU9eXUCjpx2rRZlr37uqmthR3kiDx9vjC4oQ4ugIWQLXWrcrpW4G3gWswNNa601KqYeANVrrN4CblVLzAS9QSy/T50Icy04Zl9bj2siUGEamxPS4/smdp+HxdphJ0J/Ax2fGcfaUTCYbXdyi7L519ZW7fWvS/gTu34pWXOOmYGSy2RHu3v9s7DO+aIeNEcnR7Kjwjaod1p5lMP7+6lF2q7n+7bdiVw0bS+tp9HhpaPHS1tFJcoyDvdVufvnOVnZXNfPhHaf2+fuFEAMnpGvgWuvFwOJu1+4PeHxLKOMRIpwSouwkBEx7jw04ovQPV84yH0c7gv8z9Sf6pGjfCNzboUmLc5pT6oFunZ9HQ0s7Ty/zjehjnFaGJ0fx0bYKOjt1r9vOGlp8swKrfzyf219az2vrgle6dlY2m73ewXdW+ifbK9lQWm+ul/vi6mRPdXOPHu/90d7Ryeb9DUzNSTz4zUIMMdKJTYgwG5MWw4PnTeaJy6b3+rrLbg16HtMtgYNvHT2+W9e3m04bw63zx3H/1yfhNKrjo+xWRiRH09beSWVTK7Xu4ATe0alp9LQTH9AsprvN++qDnucaDWzKG1pp8HQdrvLnT3Yy/7GlbN7XEHT/5zuruPhPn/d5MlugdzaVcf6iZQfd9ibEUCQJXIgwU0rx7eNzSYl1HvxmunqvJ8YEN4Pp3pwlcB96tMNq/NPGcKOhzB0vf9mjQK2hxUuDx2u2gO1tVL+xNDghD+/WQrbWKGzbVu6rTH99ffAI/sVVxawuquWvxjr/gVQ2tqJ1z613QghJ4EIc82qag5NXjL8K3WnDZpxfnhbn5MS8VL45K4coY8SeHheYwH2JONppNTvCfVZYRUen5puzcvifU8cAviNMGzztXe1anT17uW8rC0763RN4VZMvgTcZI+zX1+8Let0fX+D2tr40GcsLgY1lDuaBNzZx9hOfHvL9QkQqSeBCHOP8CdHPZhSeKaXMQra0OCfRDhuPXjyNacN9xW+B+8v9I3CnzcKwpOCzwHNTojku17dH/dnPi9i6v8EsiOttCr2tW5X98KTgBO7fWran2lecV9bgoaLRg8coiKs0jlotb2iltT24SK47f0Hde5vLe4zke+Nua+eZz4vYvL+h1yr8I9Hg8fLhlvLD+tkL/7CMZz8vGtB4hJAELsQxbkJm30VgicY6eOBoOzvRl6ADp9BHpfqq4BUKp83KH66cSZxZze4wT0d75vMiWts7zTXw+F4SeHfDkqIwtpMDUN3cSntHJ3tr3OZxqKf9+mN++pZvG1xFo8e8t6LhwFPj/gK/F1bu5Z7XNvTazjXQW1/t73pv4/dorc1+8f2lteb0Rz/mhZV7eXVNCdc9u8Y86/1QdXZqviyuY2Np/cFvFqIfJIELcYy779xJ/PeHJ/b6WlLACNxvVEoMLruF1IA19V9fPI2ff2MKE7N8XwbOnpLFLGPU3f1wFcAsiPNXvANcPW8kv7poatB9Svmm8gN/vqa5jU+2V9LeqZk3xtf/vbmtgw1GAqtoaDUTe1mDh75sLWsIWj5obutg3V5fpzmte+/ftHR7pfl4f73vvZcVVjPtofd4f3P/R88t3g52VTWzaV+9uQ5f3XTo0/ngm0Xo1AQd+yrEQJAELsQxzmW3mnvCu0uKdmCzKBIDEuh3ThzFGzefGNSXPSHKzuWzR6AChsojjbVrj7ejx/GmsQEHpvg9dH4+00ck9rjPYlFBFfGri2q4zji57LTx6eb13VXNdHRqqppamTLM9/fxJ9nuKho8LHj8U97dFJx0P91RSWFFI6PuXsyKXcH917XWrNhVQ/6weACzcn3zft8Xhztf/ZLOfpzCBphV+tVNbVQbI2//EsGh8jfGqZcELgaYJHAhIsRFM3M4Z0pW0LXc1BhGp8VgsXQl5hinjXEZB997ffuZ47lyzgjOnZrdYwTuX3vuvgY+olvBmn8aPvDnF28oA+DZ78zmrMmZXe/paaewoolODdOG+74IlAck8LJ6D/8x9pwX93G62dIdVWYV/M8Xbwl6rbCiiaqmVi6ckWO+H2Ae9lLn9lJ9CMlX66798f6K+urmVvNn+5vA/Yn7cKfxheiLJHAhIsRvLpnGoitnBl27/cxxvHzDvMN6v4RoO49cOIUYpy3oeFMAu9X3haD73nKX3RpUHOffbuafyo8xiuVOn5DOKePSgmYBAFbt9o2ax6TFEGW3Bk2hP/7Bdm59aT01zW0U1/Tc950R7+SrkjozkX5ZUh9UBOc/UGXOqGTinDZ+/vZW3t1UZraxha6kfiBPLyti6gPvUd7gMZN/0Ai8HxXxEJDA+xiB765q5qRffcT+etnrLvpHErgQEcxlt5qFbAPlsUumcfNpeUDwFLrfyOQYs2jNv93MP4X+1NUF3DZ/HHct7HmYC8Anxhp1eryL1DgHf/1sN3/5dBdVTa28Z6xRby9vZG9NzxH4BTOGoTW8u7HMvOY/lAW6prtTYh3mDMINf19LaZ0bqzFDcaA1dz9/tfjOyiZqjWRd1dRq7gao7TYC93gPXEnv/xLQVwL/62e7KK5pMWcuhDhUksCFEAC8+v15LPl/p/KNmTlEGSPp3raRHTcqienGFLg/gfu/RMwelcwt8/OCpvDvOHMcV8wZgdNm4YMtFaTEOMjPTmBCpm+t+uH/buF7z60xp6Z39JHAvzYpA5fdwirjnHKAoqqu+/zJNinawUUzc8zrK3bVMM04KrXsEEa5/g5xxTVuc/95g6fdTP5LtlWwyuhL/8Hmcibc9w6bArrTNbW2B621+xN3i7cjaMags1Pz6Lvb+GxHFdA1e9GbfXUt0o1O9CAJXAgBQEFusrndzK/71DrAnWdN4F/fPx6AWGMK/cIZw7jzrPHmHvVAPzgjj59dOIXrTx4NwFn5mThsFh6/dDqrfnwGM0Yksm5vHS67hVinje3lTeyt7pnAsxKigrbLAewxTlV7fX0pG0vrcdktuOxWHr14KoWPLDSn+ydlx2OzKHZXuQ84Ym5t7zATblG1O6jVrL/T3bq9ddz92leArxkOwOeFvqWB+hYv8372YVD/+MCRd+Dj51ft5fdLCiky/q7+42J7c9drG/jRv77q83UxNEkCF0IcVPcqdYtFEeOwmvvEp+QkcNNpYw/4HjedNpYfnpHHrfN90/MxThvpcS5ONyrVTxmXxriMWN76ah9byxp6/HxqrNM8QjUz3kVyjIOiajf1LV5ueXE9izeUmVP5SilsVgtzR/u2saXFusiId/H0st184w+fA75itN++vz2oBe2Ocl+RHfi+HNT1UXi2u6oZj7fD/Pv7R/8rd1XT2Noe1KI2MGk3BDz+66e7gt7zQAV25fUeKkPUTtbj7TDrCcSxTRK4EOKA3r31ZN6/7ZQe1x++MJ+r5o085Pdx2a3cfua4HqPo0yb4Evi5U7OZMzqFWrcXi0Vx6njf0axXzBnB09cU4LBZSI31Jej4KBsjU6LZU93Mlv1dyb57PcA04xSz1vYOmtt8I9zN+xvQWvP2xv088eGOoOlv/znpwxKjKKpy99nCtVP71ur9p7IVVjRRXOPm852+kXhZvYfOTo3H20F9S9d7+L8QtLR1sKfGHVTk131tPVB9i/eAI/SBdNe/vuKM33xCc4h+nzh8ksCFEAc0PjMuqFGM34Uzcsx17CORPyyBj+44hXOnZvG/Z43nvdtO5qM7TqVgpK/RTLzLzukTMgDMEXi8y05uSgzbyhp5J6CoLanbTMEVc0Zw3Ymj+M6Jo3AHHHW6r95jrikv31nNF3trAczq9xPGprCnuplad5v5O7vbur/RHHm/t7mck361hOVGAi9v8PDLd7cy4b53glrh+kfjOyub0BrOzu/aZnegEXh9i+/89dtfXm/GerSs2OVb39/TyzLGQGpr7+SSPy1nmbEMIfovpOeBCyFEb0andZ2F7i+AyzP+Gbj33H9iW5zLxrlTs1i8YT/PBPQYT+o2AnfZrdx37iQAXvjuHF5aXcwra0vYVFrPvjpfUdrP394K+IrktpQ1kBRtJy89jua2EvbWuJmcHc8tZ+QxJi2WVUU1fLK9kn9/Ucrm/Q3UNAdPsW8zps4rGlt5Zpkvrg0l9aTGOqhqaqOktoV/rS3BYgydrj1hFLVuL1+V1PU5Am9t76DF20GLF177opQ3v9zHjkfONl//z7pSJmTFDciXKfDt6S9r8LCnuplJ2QPznr3ZW+NmVVENH22t4ISxqUft9wxmMgIXQhyTzpqcyUvXz+Xy2cPNaykxXWvcZ0zM4LUbjw/6md62vfkV5Cbz4PmTUQqW7qiktDa4qvu9zeUU17SQkxRNptHqdXdVMykxDgpyk0mKcXDW5Ex+duEURqZEU1LrNkfggWaOSKS8wWPGUtbgYWJWPBbl2+t+xytf8sLKvdgsiolZ8Tz7ndkcPza1zwYx3befeTu61uy11tz60noWPD5wp6/5m/IUHeURuL9Zj6y3Hz5J4EKIY9ac0SlB7V9TjDVw/6Emk7MTeO47s7nm+FwAPAc53SzaYeP8adn8Y8XeoO1o95w9gUsLfF8UshNdZgLv1DAiJabH+2QmuChvaKXW3cY5U7PY/vBC4lw2Yp025k/KwN3WQUdAv/YTx6aSPyzBrGpfXVTL2PRYcw08JcbRo0GM1pqiquagwrfuApcFDtfH2yqCTlnznzbnr/A/EHdbO5/uqKS6qbXf29xKjK2CuyoP/nt89zXJunw3ksCFEBEjOcY3hR54VOjJ49I4LjcZOLSE9tgl0xmZEtwSdnJ2AgXG4S7utg4yA05y694+FnxV8GUNHmqb20iKtuOwWTh3ahZfn5bNMOM0uMAK9uNGJZsHu/j5z2AH39R/ndtLe0cn72ws44E3NnH/65s49dGP+XRHzzVif6vXwINVOrr1eT/QtLyf1pp7/7OR+/6z0Twgxj+rENgkpy/Pr9jLVX9dxayHP+Dr//cZHm8Ht7+0nh3ljb3ev2RrBRf/6XPztDrwjcQDt/Z5vB3c95+NFFZ0jcy11pz+m0/49tOrDhqTn7ejk8fe327u6x9ohRVNXPrn5WaHvnCQBC6EiBhRdl+zk+7Hivr3r/sbthyIxaLMRjR+E7PimWkUzeUPSwg6irV7sgdfJ7nKxlZq3V6SjXX3n39jKj//xpSgn/XLz07glDxfVf0PTh/LzaeN5bxp2ebr/iLBsgYPd732Fc98XsTfV+wBCKqy9/Mnt6qA09oCR8wdnZpL/7yCP32y80AfBTsrmyipbWFfvYeiajdaazPpF/fSTOe372/niqdWmM8bA0bE1c1tbN7fwGvrSvna40t7PTFuWWEVq4tqKaltMQsGtQ7+svCs8Xf/62e7zWv+WYE1e4IL+LTW3Pj8WrMZTqD1xXX87sMdfLytssdr3XV0an77/nYqDqFTn98ra4pZubuGpz7dffCbjxIpYhNCRIycJN/o9tSAU87A16jlnVtPIi/94Ie4AHxzVg6vr9/HfedOItphJTnGQXKMg3duPYnRqbHmlrWqpjbz1LZAgSP0pG5V6hMDiskuO24404cn4rBZmDcmhTduPoEpwxKClgUA8wvF6+v39dh73ttIeHdlM1kJLj7d3pW4Tv/NJ7x584m8sGovW8saaPF2mNvi+rJka1dye3HVXv68tGtvenVzG1rroFif+HAH4OsMl50YRb0xWj8uN4l1e+vMLxFaw9ayRiZmBRfB7TcS5O7qZvbWuBmdFsOeaje3vbSef994Ao8s3szzK/cCBI2cW/qYWalubmPxhjKyEqI4MS+4EM6/b763OoXuVu2u4YkPd7C9vJE/fmvWQe8H8H89Wbq9ss/WwUebjMCFEBEjOzGKVfecwc29NI2ZkBlv9jw/mJPy0lh1zxlcd+IoLp89Iug9/OvSGfEuXHZLr1voMhO6rnXfZpYQbTdbzJ43PZvLjPdXSjE1J7FH8gaYlBVPQpSdp41R523zx5mv7ajoWeRVXOvmF29v5bcfbAe62rC+vKaYf67aa56bXlLbc126vMHD+HvfZsWuaraVN5IZ7yI9zslzy/eY92QnuGht7zSXJD7YXM5xj3xgvu4/W73G7SU3JZoF+Vm0d2q+KunaUx/4u59fuYfiGjf7jXXyoqpmimvcnDg2lUcvnsrWskae+byIf6zYy9enZjMtJyFoCr2vpRH/4TS99biv7Mf57ZXGNHj3ZYgD8RdBbt7fELI9+t1JAhdCRJT0eFfQ8alH8j4HMio1hnEZcb0m3MBp8jmjUnq8/udvzSI3JZrJWQef0gfftP7sUcnmXvDvnjSK3146jWiHNWhEnhzjIDPeRXFNC9vLuxLc2vvOpGBkkjnt7lfaS2HZR1sraG3v5JllRdS520iKcTA6LYaWgHXoMem+bX01xij8u8+tCeoE5y9cq2luJTnGYTbY+WJPLf7/afz3F9e4+fG/N3LZkyvMhPtlcR2Nre0MT4rmjIkZKAW/fGcrcU4bP70gn7mjU9hlnB8PwQk8cL3c/369TX0faATuP9r1V+9s5abnvzAL8GKctqDGPp8XVvXYHdDa3kFHp6Yk4Mjb8n5MvQ8kSeBCCNGLn56fz1NXF/T6WmAC91esBzp+bCof33kaCd0ayxzItSfkAr6tcDFOGxfOyDEL6PzfIRKj7AxPjmJvTTNFAVPrLruV2aN8hXz+OgHwFdIFjg5b2zvYWOpLUE67hTq3l6RoOyOTgyvtRxs1BdXNbUFfFAAsyneU67xffMSywmqSYxykGMWFX5bUUzDSF4c/ga40Dn4prfOttQNmYd7w5GjiXXYyjO58F84cRkKUnTHpsbS1d1Jc4+anb23me8+tMX///oAjYf1T8gccgXdLwB9uKWfaQ++xuqiG9zeXs2RbhflZ/ntdKef9fhk1zW14Ozq5+ulVPLe8KOjzO+u3S7n/9Y2U1rWQa9RH+BN4e0cna/fU9mskfyRkDVwIIXrRfW07UEqMg3OmZplbzwbC8WNSee+2k+kMKP5Kj3extayR8RlxbC1rJDHazvDkaF77orTHz1934iiiHVbOmZrNaY9+bF4vrW1hfGYcnZ2aCxd9zmajKK6o2k2Tx8v4zDhGdCvU889O1DT3rLA+KS/NPBYWfBX0/u19AGMzYtlR0ciiJYW8vKa4xyyAUl1JdXiyr6Yh1mWDBsxT5CZk+moZNpTWBxWzAfx9+R7uO3ciSinKjWRe3tDaY73ePy1e020K3b/Ov7WskZ2Vvt73/i8Z4JtG9yVkF+2dOqiS/+U1JRRVuymq9q3TnzwujaJqt5nA1xXXcfGflvOHK2dy9pSsHp/dQJMRuBBC9JNSikVXzOTkcWkD+r7jMoI7qvnXtk+fkI7DZiEx2sHwpJ5FdeDrUnfz6XnkpkQT47Bit/qSWWmdb6p3+a5qM3lbLYpdFU3Uub2+9zRG+pOy4rlt/jgz+VQ3tZmtYEen+Ubl3bfDJcc6zAY74Eu+aXFO2jo6zeR9wtiunzlhTFexmf/3PnHZdG6bP46pxi6CSVnxxDptLN9V3ePv+fSy3dzz7438v1e+NEfjbe2dPYr/uk+h/23Zbv731S/No2A376s3D67pXihY09xmFtGtL67jlF8voazew6trioNO7PPPNpQ3+H7XJ9sqsVpUyDrLyQhcCCGOUf5uaMePSeXlNcUkRtnJTe1K4N+clWOOVv2UUhTkJhPnsrF4w35WF9Vy+oQMnl+5h6RoO8vvPoN/rtrLg29uhlbftLy/0n5iVjy3zM8zp91rmtvMJPfKDfNo8LSzrSx4j3dytCNotmLOqBTe3VTG9vImLpiezTUnjCIz3sW/vijh1+9u48o5I8xjWOON42gnZycwOburXsBmtTB7VDJvrN8X9LsWXTGTm174gn+u8o2A89K7WvCWNXiC4ug+hf67D3dQ6/YS47DS3NZh9q3vzYuri0mP61oWAHhpdTGb9zfwnRNHcdHMHD7ZVskFM7L52eIt5gj84+0VzByRaHazO9okgQshxDHq3nMm8qt3tlKQm8RPz89nZEoMY9Nj+bK4nqqmVn510dReC/qeufY4wDcyfWl1MVfMHsF7m8q59oRcXHYrYwJ6zydFO8y1dv82vRiHFYfNQk1zGx2dmii7lZRYJymxzh7d0GKctqBz4/PSY3HafDMH4zPjzS1yN546hvOndzW6OZjjx6Tw0daKoGv5w+KZMiyBDcY6/o6KJjLinZQ3tLK3xk1uSgyNrV7sFgtVxhR6bXMbDR4vtW4vd541nhtPHcO4e9+mqNqNy27B4/XtMf/6tGze/NL3hcH/z0DLdlbh7dBMz0lkXEac2bM/Pd5JRUMrG0vr2VjawJ1njT+kv99AkAQuhBDHqBPGpvL6zScCsDBgTfWB8yYf8Of8a8FXzRvJe5vLufmf62jv1OaWOf90OPi2vSXFOFh0xUyOG5Vk/rzLZjHXi/2JvftjICihZyf4dgj4K7cDZweUUuQY0/9PXV2AzXrgnQTnTc/m4f9uCboW5bByXG6ymcABfnhGHo9/sIN7XttAdXMbGfFO0uKcdGjNSXmpfLqjigsWLQNgvLGrIN5lp7q5jbQ4J7NzU4hz2cyz3fvin3qf2q0JUEaci/IGD/f8ewPpcU6+NefQj9g9UpLAhRBikDp+TCopMQ6+LK5j3ugU89S37IQoc/TpP8HtnKnBRVcNnq7EHLjGnRBlJycpigWTM9lb4+Zio5Dvy/u/ht3mS8ojkqNZX1wXNNIPdOakjIPGnh7nYu7oZPN4U/D1sr989nDaOzvZVtbI5v0NXH7cCOaOTuGuf31FdXMb5Q2tVDa28j+njCEvI5ZPd1SZ/dbHG18oEqJ8CTw11slvLpkGwKIlhQeNKSPeSXa3XQdZiS7e2ViGu62De8+Z2K+dB0dKitiEEGKQsloUX5vsO3P8yrldDWssFsWoVF9y7X6Gut+fvjUTp/+wldiuxjVKKT64/RTuPnsiT15dYDaySYi2E+3wjQkfvjCfZ649rkd1e3899505/OIbU8znUXYreRlxPHR+Ps9dN5tV98zHYlGMSYvlle8fz8+Nezu1r0DuzEmZPHJhvvnz/un7OGONOjXg7xXYX78v154wqkdfgFPGpZn71E8Z4KLGg5ERuBBCDGLXnTgKu1XxtUmZQdfHpMWwZX8DiX0k8AX5Wdx+ppufv721R4c7V8Be897Eu+w92t0eDofNYjaVAYLi8K+zB8oOWF/PSnAR67Rx5ZyRxDhs7K5qNusFEnpJ4HEHmEJXypecr57Xc3p8/sQMXHYLCVF2xqb3PuNwtEgCF0KIQWxseiwPnZ/f47p/Oj0xuu/97v4p56ownrh1oMTaXeD0dmAyv2DGsKD7Yp2+5J8WsH/96nm5OGwW7n99U4/3XZifyR+u7L1HeozTxi1njCPaYe21a9/RFNIpdKXUAqXUNqVUoVLqrl5ev10ptVkp9ZVS6kOlVOiqAYQQYgi5fPZwHjxvctAotLs8o9K6+1a1UIpzHfqacmBXpgAb0gAACPRJREFUvKxeOuT5+TulBfa5d9gsXD0vN6iTnV/3fvfd/c+pY/i2cSZ9KIUsgSulrMAiYCEwCbhcKTWp223rgAKt9VTgVeBXoYpPCCGGkqyEqIMmnWGJUbxx8wncf+6Bq96Ppv6MwONcduKcNt+fAyR+/3p3b19e3vzBidxzdvDpYv5z6I81oRyBzwYKtda7tNZtwIvA+YE3aK2XaK39HeJXADkhjE8IIUQ3U3MSiXIceM37aIp19G+lNyvRRVbigQ+q8Z8vHu3s+d5j02O5am5u0LXkEFaW90co18CHAcUBz0uAOQe4/zrg7aMakRBCiGNaf0+e+/rU7IPeMyI5hmVUB22PCxT4hcVhtTD2EM+ZD7VjsohNKfUtoAA4pY/XrweuBxgxYkRvtwghhBiCfnBG3kHvuf/cSZw2Po38YQc/7nXNffPNlq/HmlAm8FIg8OieHONaEKXUfODHwCla615LH7XWTwJPAhQUFITm3DYhhBBhMTw5ivzsQztb/VBEOazm/vi+vPr9eSTHOI7Z5A2hTeCrgTyl1Ch8ifsy4IrAG5RSM4A/Awu01hU930IIIcRQ8+n/nh7y31mQmxzy39lfISti01q3AzcD7wJbgJe11puUUg8ppc4zbvs1EAu8opRar5R6I1TxCSGEEJEkpGvgWuvFwOJu1+4PeDw/lPEIIYQQkUp6oQshhBARSBK4EEIIEYEkgQshhBARSBK4EEIIEYEkgQshhBARSBK4EEIIEYEkgQshhBARSGkd2Z1IlVKVwJ4BfMtUoGoA30/IZ3q0yOc68OQzHXjymR65kVrrtO4XIz6BDzSl1BqtdUG44xhM5DM9OuRzHXjymQ48+UyPHplCF0IIISKQJHAhhBAiAkkC7+nJcAcwCMlnenTI5zrw5DMdePKZHiWyBi6EEEJEIBmBCyGEEBFIErgQQggRgSSBB1BKLVBKbVNKFSql7gp3PJFCKfW0UqpCKbUx4FqyUup9pdQO459JxnWllPqd8Rl/pZSaGb7Ij11KqeFKqSVKqc1KqU1KqVuM6/K5HiallEsptUop9aXxmT5oXB+llFppfHYvKaUcxnWn8bzQeD03nPEfy5RSVqXUOqXUW8Zz+UxDQBK4QSllBRYBC4FJwOVKqUnhjSpiPAMs6HbtLuBDrXUe8KHxHHyfb57x53rgjyGKMdK0A3dorScBc4GbjH8f5XM9fK3A6VrracB0YIFSai7wS+C3WuuxQC1wnXH/dUCtcf23xn2id7cAWwKey2caApLAu8wGCrXWu7TWbcCLwPlhjikiaK2XAjXdLp8PPGs8fha4IOD6c9pnBZColMoKTaSRQ2u9X2v9hfG4Ed//OQ5DPtfDZnw2TcZTu/FHA6cDrxrXu3+m/s/6VeAMpZQKUbgRQymVA5wD/MV4rpDPNCQkgXcZBhQHPC8xronDk6G13m88LgMyjMfyOfeTMc04A1iJfK5HxJjqXQ9UAO8DO4E6rXW7cUvg52Z+psbr9UBKaCOOCI8D/wt0Gs9TkM80JCSBi6NO+/Yqyn7Fw6CUigX+BdyqtW4IfE0+1/7TWndoracDOfhm3SaEOaSIppQ6F6jQWq8NdyxDkSTwLqXA8IDnOcY1cXjK/VO4xj8rjOvyOR8ipf5/e/cWomURx3H8+0sLjTIJzJvKEjuoIUEEdrKl80G6KCtJU7QugpK6CEIjksqlM1gWBmZBhWUUKRF1o2REF1kJllmWSAdCMisJD4X8unhm8/FlIZH1XZ93fx9YeHZmnveZGdj97zMzO6MjqYL3a7bfLsnp1z5g+w9gNXAe1XTD4JJV77f/+rTkHwf81uaqHu4uAK6TtIVq2vESYCHp07ZIAN/nU+C0snryKGAqsLKf69RkK4GZ5XomsKKWPqOsmp4I/FkbEo6izAu+CHxt++laVvr1IEkaIWl4uR4KXE61tmA1MKUUa+3Tnr6eAqxydr7aj+25tk+0fQrV78xVtqeRPm2L7MRWI+kaqvmcQcBS2wv6uUqNIGkZ0EV1bOBW4EHgHWA5cDLVca832d5eAtMiqlXrO4FZttf2R70PZ5IuBD4C1rNvbnEe1Tx4+vUgSJpAtYBqENXLy3LbD0kaTfX2eDzwBTDd9h5JQ4BXqNYfbAem2t7cP7U//EnqAu61PTl92h4J4BEREQ2UIfSIiIgGSgCPiIhooATwiIiIBkoAj4iIaKAE8IiIiAZKAI+IQ05SlySXfbMjog8kgEdERDRQAnhEREQDJYBHDACS5kjaKGm3pE2S7u/Zq1rSFkkLJC2RtEPSNkndko6o3X+spBck/Sppj6S1kq5oecYJkl6StLU85xtJs1uqMlbSGkk7JW2QdHUbmh/RkQb/f5GIaDJJ84FZwD3AOmAssBgYAjxQis2h2kb4XKpTuhZTbYu7sOQvLXnTgR+AO4B3JU2wvbHsLf4hsAuYBmwGxlBtpVn3JHAf1TGe84A3JI2y/Xvftjqi82Ur1YgOJuloYBtwve33a+kzgGdsDy8nSf1o+6Jafjdwq+2TJI0BNgHX2n6vVuZzYJ3t2ZJuA54Dxtj+qZd6dFEdcHFDz8lqkkZSnWl+le0P+rrtEZ0ub+ARnW08MBR4S1L9r/VBwBBJI8r3n7Tc9zEwV9IwYFxJW9NSZg3VcZwA5wAbegveLdb1XNjeKmkvMPKAWhIR+0kAj+hsPfPYNwLf9pK/vY11Afi7l7SsxYk4CPnBiehsXwG7gdG2v+vla28pN7HlvvOBn23vKJ8BMKmlzCTgy3L9GTAu/+cd0T4J4BEdzPZfQDfQLelOSWdIGi9pqqTHakXPljRf0umSbgHuBp4qn/E98CbwvKQrJZ0paSFwFvBEuX8Z1fnkKyVdJulUSZdKurldbY0YaDKEHtHhbD8s6RfgLqqgvItqOP3lWrFngVHAWuAfYBH7VqAD3E4VrF8FhgHrgcm2N5Zn7JR0MfA48DpwDLAFePRQtStioMsq9IgBrqxCX2L7kf6uS0QcuAyhR0RENFACeERERANlCD0iIqKB8gYeERHRQAngERERDZQAHhER0UAJ4BEREQ2UAB4REdFA/wLmvUELiS1nawAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "sg.utils.plot_history(dgi_history)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Node classification\n",
+ "\n",
+ "We've now initialised the weights of the model to capture useful properties of the graph structure and node structure. We can now further train the model to perform a node classification prediction task. To emphasise the value of the unsupervised weights, we will use a very small amount of labelled data for training.\n",
+ "\n",
+ "> See [the GCN node classification demo](gcn-node-classification.ipynb) for more details on this task.\n",
+ "\n",
+ "### Data preparation\n",
+ "\n",
+ "The Cora dataset labels academic papers into one of 7 subjects:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
subject
\n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
Neural_Networks
\n",
+ "
818
\n",
+ "
\n",
+ "
\n",
+ "
Probabilistic_Methods
\n",
+ "
426
\n",
+ "
\n",
+ "
\n",
+ "
Genetic_Algorithms
\n",
+ "
418
\n",
+ "
\n",
+ "
\n",
+ "
Theory
\n",
+ "
351
\n",
+ "
\n",
+ "
\n",
+ "
Case_Based
\n",
+ "
298
\n",
+ "
\n",
+ "
\n",
+ "
Reinforcement_Learning
\n",
+ "
217
\n",
+ "
\n",
+ "
\n",
+ "
Rule_Learning
\n",
+ "
180
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " subject\n",
+ "Neural_Networks 818\n",
+ "Probabilistic_Methods 426\n",
+ "Genetic_Algorithms 418\n",
+ "Theory 351\n",
+ "Case_Based 298\n",
+ "Reinforcement_Learning 217\n",
+ "Rule_Learning 180"
+ ]
+ },
+ "execution_count": 12,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "node_classes.value_counts().to_frame()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To simulate a data-poor environment, we will split the data into a train set of size 8, along with test and validation sets."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "train_classes, test_classes = model_selection.train_test_split(\n",
+ " node_classes, train_size=8, stratify=node_classes, random_state=1\n",
+ ")\n",
+ "val_classes, test_classes = model_selection.train_test_split(\n",
+ " test_classes, train_size=500, stratify=test_classes\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The train set has only one or two observations of each class."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "
"
+ ],
+ "text/plain": [
+ " loss acc\n",
+ "with DGI pre-training 1.59 0.563\n",
+ "without pre-training 2.02 0.456"
+ ]
+ },
+ "execution_count": 28,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "pd.DataFrame(\n",
+ " [pretrained_test_metrics, direct_test_metrics],\n",
+ " index=[\"with DGI pre-training\", \"without pre-training\"],\n",
+ ").round(3)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Conclusion\n",
+ "\n",
+ "In this demo, we performed semi-supervised node classification on the Cora dataset. This example had extreme data scarcity: only 8 labelled training examples, with one or two from each of the 7 classes. We used Deep Graph Infomax to train a GCN model on the whole Cora graph, without labels. We then further trained this GCN model in the normal manner, to fine-tuned its weights on the small set of labelled data. The GCN model pre-trained with Deep Graph Infomax outperforms a GCN model without any such pre-training."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "nbsphinx": "hidden",
+ "tags": [
+ "CloudRunner"
+ ]
+ },
+ "source": [
+ "
Run the latest release of this notebook:
"
+ ]
+ }
+ ],
+ "metadata": {
+ "celltoolbar": "Tags",
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/demos/node-classification/gcn-node-classification.ipynb b/demos/node-classification/gcn-node-classification.ipynb
index 654ddac7b..b34291b04 100644
--- a/demos/node-classification/gcn-node-classification.ipynb
+++ b/demos/node-classification/gcn-node-classification.ipynb
@@ -1333,6 +1333,8 @@
"2. built a TensorFlow Keras model and data generator with [the StellarGraph library](https://github.com/stellargraph/stellargraph) \n",
"3. trained and evaluated it using TensorFlow and other libraries\n",
"\n",
+ "For problems with only small amounts of labelled data, model performance can be improved by semi-supervised training. See [the GCN + Deep Graph Infomax fine-tuning demo](gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb) for more details on how to do this.\n",
+ "\n",
"StellarGraph includes [other algorithms for node classification](README.md) and [algorithms and demos for other tasks](../README.md). Most can be applied with the same basic structure as this GCN demo."
]
},
diff --git a/docs/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.nblink b/docs/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.nblink
new file mode 100644
index 000000000..c89920100
--- /dev/null
+++ b/docs/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.nblink
@@ -0,0 +1,3 @@
+{
+ "path": "../../../demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb"
+}
diff --git a/docs/demos/node-classification/index.rst b/docs/demos/node-classification/index.rst
index c342b5cbf..8b55b76f1 100644
--- a/docs/demos/node-classification/index.rst
+++ b/docs/demos/node-classification/index.rst
@@ -119,6 +119,14 @@ This table lists all node classification demos, including the algorithms trained
- yes
-
- yes
+ * - :doc:`GCN, Deep Graph Infomax and fine-tuning `
+ - GCN, DeepGraphInfomax, semi-supervised training
+ - yes
+ -
+ -
+ -
+ -
+ - yes
See :doc:`the root README <../../README>` or each algorithm's documentation for the relevant citation(s). See :doc:`the demo index <../index>` for more tasks, and a summary of each algorithm.