One_Shot_Learning_for_Language_Modelling.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "One-Shot Learning for Language Modelling.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [
        "x1xdz45548Ns",
        "EViEeud46BMl",
        "m-hQd0zI7hjV",
        "I9oiQuQU-VuM",
        "N94079HsAE9N"
      ],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/adriangonz/statistical-nlp-17/blob/master/One_Shot_Learning_for_Language_Modelling.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "FbJ6RF543_8W",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# One-Shot Learning for Language Modelling\n",
        "\n",
        "This notebooks allows for quick and easy experimentation with the work done by Group 17 of the Statistical Natural Language Processing module at UCL, formed by:\n",
        "\n",
        "- Talip Ucar (talip.ucar.16@ucl.ac.uk)\n",
        "- Adrian Daniel Szwarc (adrian.szwarc.18@ucl.ac.uk)\n",
        "- Matthew Lee (matthew.lee.16@ucl.ac.uk)\n",
        "- Adrian Gonzalez-Martin (adrian.martin.18@ucl.ac.uk)\n",
        "\n",
        "Our work implements the Matching Networks architecture ([Vinyals et al.,\n",
        "2016](http://arxiv.org/abs/1606.04080)) in `pytorch` and applies it to a\n",
        "Language Modelling task. We then experiment with using different distance metrics and episodes sizes.\n",
        "\n",
        "More details can be found in the associated paper or in the repository https://github.com/adriangonz/statistical-nlp-17.\n"
      ]
    },
    {
      "metadata": {
        "id": "x1xdz45548Ns",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Setup\n",
        "\n",
        "We will first setup Colab's environment by:\n",
        "\n",
        "* Installing `pipenv`, the dependencies management tool we use.\n",
        "* Clone the repository (found at [adriangonz/statistical-nlp-17](https://github.com/adriangonz/statistical-nlp-17)).\n",
        "* Install its dependencies."
      ]
    },
    {
      "metadata": {
        "id": "tdfkgKfa5mCw",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 258
        },
        "outputId": "c8bccd27-d3ae-4b38-e5e6-bc71cacf32f6"
      },
      "cell_type": "code",
      "source": [
        "!pip install pipenv"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting pipenv\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/13/b4/3ffa55f77161cff9a5220f162670f7c5eb00df52e00939e203f601b0f579/pipenv-2018.11.26-py3-none-any.whl (5.2MB)\n",
            "\u001b[K    100% |████████████████████████████████| 5.2MB 5.2MB/s \n",
            "\u001b[?25hCollecting virtualenv (from pipenv)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/33/5d/314c760d4204f64e4a968275182b7751bd5c3249094757b39ba987dcfb5a/virtualenv-16.4.3-py2.py3-none-any.whl (2.0MB)\n",
            "\u001b[K    100% |████████████████████████████████| 2.0MB 17.3MB/s \n",
            "\u001b[?25hCollecting virtualenv-clone>=0.2.5 (from pipenv)\n",
            "  Downloading https://files.pythonhosted.org/packages/e3/d9/d9c56deb483c4d3289a00b12046e41428be64e8236fa210111a1f57cc42d/virtualenv_clone-0.5.1-py2.py3-none-any.whl\n",
            "Requirement already satisfied: pip>=9.0.1 in /usr/local/lib/python3.6/dist-packages (from pipenv) (19.0.3)\n",
            "Requirement already satisfied: setuptools>=36.2.1 in /usr/local/lib/python3.6/dist-packages (from pipenv) (40.8.0)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.6/dist-packages (from pipenv) (2019.3.9)\n",
            "Installing collected packages: virtualenv, virtualenv-clone, pipenv\n",
            "Successfully installed pipenv-2018.11.26 virtualenv-16.4.3 virtualenv-clone-0.5.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "sniRfqM83xCn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        },
        "outputId": "e7f2a2bd-f404-4e2d-b814-724c5096f3be"
      },
      "cell_type": "code",
      "source": [
        "!git clone https://github.com/adriangonz/statistical-nlp-17"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'statistical-nlp-17'...\n",
            "remote: Enumerating objects: 114, done.\u001b[K\n",
            "remote: Counting objects: 100% (114/114), done.\u001b[K\n",
            "remote: Compressing objects: 100% (71/71), done.\u001b[K\n",
            "remote: Total 805 (delta 68), reused 80 (delta 43), pack-reused 691\u001b[K\n",
            "Receiving objects: 100% (805/805), 349.89 MiB | 36.08 MiB/s, done.\n",
            "Resolving deltas: 100% (446/446), done.\n",
            "Checking out files: 100% (78/78), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "lyFMWR3B5Dt6",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "e5bd18f0-9288-4b10-f0a8-f837c7bb53d3"
      },
      "cell_type": "code",
      "source": [
        "%cd statistical-nlp-17"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/statistical-nlp-17\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "uZ0HEtBJ5Id3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 306
        },
        "outputId": "d613856a-e07b-492e-fb36-5cf695a0985a"
      },
      "cell_type": "code",
      "source": [
        "!pipenv install"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[39m\u001b[1mCreating a virtualenv for this project…\u001b[39m\u001b[22m\n",
            "Pipfile: \u001b[31m\u001b[1m/content/statistical-nlp-17/Pipfile\u001b[39m\u001b[22m\n",
            "\u001b[39m\u001b[1mUsing\u001b[39m\u001b[22m \u001b[31m\u001b[1m/usr/local/bin/python\u001b[39m\u001b[22m \u001b[32m\u001b[22m(3.6.7)\u001b[39m\u001b[22m \u001b[39m\u001b[1mto create virtualenv…\u001b[39m\u001b[22m\n",
            "⠋\u001b[0m Creating virtual environment...\u001b[K\u001b[34m\u001b[22mUsing base prefix '/usr'\n",
            "New python executable in /root/.local/share/virtualenvs/statistical-nlp-17-bg53uuH_/bin/python3\n",
            "Also creating executable in /root/.local/share/virtualenvs/statistical-nlp-17-bg53uuH_/bin/python\n",
            "Installing setuptools, pip, wheel...\n",
            "done.\n",
            "Running virtualenv with interpreter /usr/local/bin/python\n",
            "\u001b[39m\u001b[22m\n",
            "\u001b[K\u001b[?25h\u001b[32m\u001b[22m✔ Successfully created virtual environment!\u001b[39m\u001b[22m\u001b[0m \n",
            "Virtualenv location: \u001b[32m\u001b[22m/root/.local/share/virtualenvs/statistical-nlp-17-bg53uuH_\u001b[39m\u001b[22m\n",
            "\u001b[39m\u001b[1mInstalling dependencies from Pipfile.lock (84d074)…\u001b[39m\u001b[22m\n",
            "\u001b[33m\u001b[22mIgnoring appnope: markers 'sys_platform == \"darwin\"' don't match your environment\u001b[39m\u001b[22m\n",
            "  🐍   \u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m\u001b[32m\u001b[1m▉\u001b[39m\u001b[22m 84/84 — \u001b[30m\u001b[22m00:03:35\u001b[39m\u001b[22m\n",
            "To activate this project's virtualenv, run \u001b[31m\u001b[22mpipenv shell\u001b[39m\u001b[22m.\n",
            "Alternatively, run a command inside the virtualenv with \u001b[31m\u001b[22mpipenv run\u001b[39m\u001b[22m.\n",
            "\u001b[0m"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "EViEeud46BMl",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Loading model\n",
        "\n",
        "We can now load one of the pre-trained models. In particular, we will choose a model trained with the following parameters:\n",
        "\n",
        "* Poincaré as distance metric.\n",
        "* $N = 5$\n",
        "* $k = 3$\n",
        "\n",
        "The `state_dict` for this model can be found in [models/poincare_vanilla_N=5_k=3_model_34.pth](https://github.com/adriangonz/statistical-nlp-17/blob/master/models/poincare_vanilla_N%3D5_k%3D3_model_34.pth)."
      ]
    },
    {
      "metadata": {
        "id": "wvgpQk3e62NI",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import os\n",
        "import csv\n",
        "\n",
        "import torch\n",
        "\n",
        "from src.matching_network import MatchingNetwork\n",
        "from src.utils import extract_model_parameters, get_model_name\n",
        "from src.data import read_vocab, read_data_set, reverse_tensor\n",
        "from src.datasets import EpisodesDataset\n",
        "from src.evaluation import _episode_to_text"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Z8iM6CW95Jx-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def load_model(model_path):\n",
        "    model_file_name = os.path.basename(model_path)\n",
        "    distance, embeddings, N, k = extract_model_parameters(model_file_name)\n",
        "    model_name = get_model_name(distance, embeddings, N, k)\n",
        "    model = MatchingNetwork(model_name, distance_metric=distance)\n",
        "    model_state_dict = torch.load(model_path)\n",
        "    model.load_state_dict(model_state_dict)\n",
        "\n",
        "    return model, N, k"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4yXNVD496x4t",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "5959a6de-00d6-48be-9727-9bab7fadb1ce"
      },
      "cell_type": "code",
      "source": [
        "model, N, k = load_model(\"./models/poincare_vanilla_N=5_k=3_model_34.pth\")\n",
        "model.eval()"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "MatchingNetwork(\n",
              "  (encode): EncodingLayer(\n",
              "    (encoding_layer): Embedding(27443, 64, padding_idx=1)\n",
              "  )\n",
              "  (g): GLayer(\n",
              "    (fce_layer): LSTM(64, 64, batch_first=True, bidirectional=True)\n",
              "  )\n",
              "  (f): FLayer(\n",
              "    (lstm_cell): LSTMCell(64, 64)\n",
              "  )\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "metadata": {
        "id": "m-hQd0zI7hjV",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Defining test set\n",
        "\n",
        "We will now define a small test set with some labels and sentences pairs. From this set, an episode will be sampled and the output will be predicted following the meta-testing framework described in the paper."
      ]
    },
    {
      "metadata": {
        "id": "uZGtpSFN7Swx",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "test_text = {\n",
        "    \"particularly\": [\n",
        "        \"their work which used <unk> paints to create designs representing body painting and ground sculptures rapidly spread across indigenous communities of central australia <blank_token> after the introduction of a government sanctioned art program in central australia in N\",\n",
        "        \"the center of education since the colonial period manila <blank_token> <unk> is home to several philippine universities and colleges as well as its oldest ones\",\n",
        "        \"over next two months however fluctuations in sea surface temperatures <blank_token> those in the central pacific caused the group to revise their predictions downward and indicated the probability for a slightly below average typhoon season in their june forecast\",\n",
        "        \"a breech <unk> could be <unk> without moving the gun a lengthy process <blank_token> if the gun then needed to be re aimed\"\n",
        "    ],\n",
        "    \"score\": [\n",
        "        \"in N the magazine chose the <blank_token> as one of N essential soundtracks it believed spoke to the complex and innovative relationships between music and screen storytelling\",\n",
        "        \"the music is used like a visual cue so that lester and the <blank_token> are staring at angela\",\n",
        "        \"currently the film holds an N <blank_token> on rotten tomatoes based on N reviews with an average rating of N N N the critical consensus reads <unk> cast and <unk> with dark acid wit american beauty is a smart provocative high point of late 90s mainstream hollywood film\",\n",
        "        \"instead <unk> was drawn to the emotion and darkness he began to use the <blank_token> and shots he had intended to <unk> to craft the film along these lines\"\n",
        "    ],\n",
        "    \"managed\": [\n",
        "        \"following another <unk> by <unk> <unk> <blank_token> to secure a <unk> and force a submission at N N of the first round and <unk> lost in his <unk> debut\",\n",
        "        \"at summerslam <unk> defeated <unk> to become the wwe world heavyweight champion and during the match he delivered sixteen <unk> most of which were german <unk> and two f <unk> to <unk> who barely <blank_token> any offense\",\n",
        "        \"all the schools under the national education system are <blank_token> by the <unk> district education office\",\n",
        "        \"he <blank_token> to win a seat later when a special election was held after <unk> opened several seats\"\n",
        "    ],\n",
        "    \"director\": [\n",
        "        \"a music video for the single was shot with <blank_token> tony <unk> and was released on june N N online through yahoo\",\n",
        "        \"emmanuel <unk> jewish holocaust survivor and <blank_token> of the search party to find hitler after <unk> out of a death pit in <unk> he never took the time to <unk> and embarked on a life consuming obsession to bring those responsible for the <unk> to justice\",\n",
        "        \"by the end of the year reports of a critical backlash suggested american beauty was the underdog in the race for best picture however at the golden globe awards in january N american beauty won best film best <blank_token> and best screenplay\",\n",
        "        \"<unk> was named best <blank_token> by the new york film critics circle awards and <unk> and carroll shared the writers guild of america award for best written drama\"\n",
        "    ],\n",
        "    \"music\": [\n",
        "        \"the <blank_token> video received N N million views in a N hour period and positive commentary from reviewers who appreciated its <unk> <unk> nature\",\n",
        "        \"a <blank_token> video for the single was shot with director <unk> brown in los angeles the video received a premier on mtv 's <unk> ball on june N N\",\n",
        "        \"in addition to <unk> the <unk> also introduces a redesigned <unk> revised title sequence and theme <blank_token> and sees changes to the doctor 's costume\",\n",
        "        \"the album the first independent release by <unk> after he was signed by sony <blank_token> in N and warner music in N was issued by his own label <unk>\"  \n",
        "    ]\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "AB1cYAA2-ECi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "For ease of re-using the current implementation, we will write these to a CSV, to then read it back as numericalised tensors using a pre-computed vocabulary."
      ]
    },
    {
      "metadata": {
        "id": "5Kcf4F5h9nxE",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "with open(\"data/test-colab.csv\", 'w') as output_file:\n",
        "    writer = csv.writer(output_file)\n",
        "    writer.writerow([\"label\", \"sentence\"])\n",
        "\n",
        "    for label, sentences in test_text.items():\n",
        "        for sentence in sentences:\n",
        "            writer.writerow([label, sentence])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "V0v1AkFY9MPN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "vocab = read_vocab(\"data/vocab.json\")\n",
        "X_test, y_test = read_data_set(\"data/test-colab.csv\", vocab)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "I9oiQuQU-VuM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Generating episode\n",
        "\n",
        "We can now load our data into a `DataSet`, and generate a meta-testing episode, composed of $N = 5$ labels and $k = 3$ examples.\n",
        "\n",
        "By default, the target and examples will be chosen randomly out of the test set."
      ]
    },
    {
      "metadata": {
        "id": "9NGZ8AFJ9_HL",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "test_set = EpisodesDataset(X_test, y_test, k=k)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "m6XhOlw6-gyQ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "episode = test_set[(0, 1, 2, 3, 4)]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "l4OYFGmx_tbs",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We can visualise the content of the chosen examples and target query."
      ]
    },
    {
      "metadata": {
        "id": "c1HCiFDu-wHJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "support_set_text, targets_text, support_labels, _ = _episode_to_text(*episode, vocab)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ah1KcNDS_SZM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "outputId": "77e134c7-fd5b-4f20-cf15-8aad37425e88"
      },
      "cell_type": "code",
      "source": [
        "support_set_text"
      ],
      "execution_count": 110,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([['over next two months however fluctuations in sea surface temperatures <blank_token> those in the central pacific caused the group to revise their predictions downward and indicated the probability for a slightly below average typhoon season in their june forecast',\n",
              "        'the center of education since the colonial period manila <blank_token> <unk> is home to several philippine universities and colleges as well as its oldest ones',\n",
              "        'a breech <unk> could be <unk> without moving the gun a lengthy process <blank_token> if the gun then needed to be re aimed'],\n",
              "       ['in N the magazine chose the <blank_token> as one of N essential soundtracks it believed spoke to the complex and innovative relationships between music and screen storytelling',\n",
              "        'instead <unk> was drawn to the emotion and darkness he began to use the <blank_token> and shots he had intended to <unk> to craft the film along these lines',\n",
              "        'currently the film holds an N <blank_token> on rotten tomatoes based on N reviews with an average rating of N N N the critical consensus reads <unk> cast and <unk> with dark acid wit american beauty is a smart provocative high point of late 90s mainstream hollywood film'],\n",
              "       ['he <blank_token> to win a seat later when a special election was held after <unk> opened several seats',\n",
              "        'following another <unk> by <unk> <unk> <blank_token> to secure a <unk> and force a submission at N N of the first round and <unk> lost in his <unk> debut',\n",
              "        'at summerslam <unk> defeated <unk> to become the wwe world heavyweight champion and during the match he delivered sixteen <unk> most of which were german <unk> and two f <unk> to <unk> who barely <blank_token> any offense'],\n",
              "       ['a music video for the single was shot with <blank_token> tony <unk> and was released on june N N online through yahoo',\n",
              "        'by the end of the year reports of a critical backlash suggested american beauty was the underdog in the race for best picture however at the golden globe awards in january N american beauty won best film best <blank_token> and best screenplay',\n",
              "        '<unk> was named best <blank_token> by the new york film critics circle awards and <unk> and carroll shared the writers guild of america award for best written drama'],\n",
              "       ['the album the first independent release by <unk> after he was signed by sony <blank_token> in N and warner music in N was issued by his own label <unk>',\n",
              "        'the <blank_token> video received N N million views in a N hour period and positive commentary from reviewers who appreciated its <unk> <unk> nature',\n",
              "        \"in addition to <unk> the <unk> also introduces a redesigned <unk> revised title sequence and theme <blank_token> and sees changes to the doctor 's costume\"]],\n",
              "      dtype='<U270')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 110
        }
      ]
    },
    {
      "metadata": {
        "id": "qY-EQrEIABV0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "7a32de5f-6881-43b5-b11f-ebc89f136dfc"
      },
      "cell_type": "code",
      "source": [
        "support_labels"
      ],
      "execution_count": 111,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['particularly', 'score', 'managed', 'director', 'music'],\n",
              "      dtype='<U12')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 111
        }
      ]
    },
    {
      "metadata": {
        "id": "YpuMxF9q_cql",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "outputId": "113300cb-c461-4722-ce9c-7c4c26408907"
      },
      "cell_type": "code",
      "source": [
        "targets_text"
      ],
      "execution_count": 112,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['their work which used <unk> paints to create designs representing body painting and ground sculptures rapidly spread across indigenous communities of central australia <blank_token> after the introduction of a government sanctioned art program in central australia in N'],\n",
              "      dtype='<U269')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 112
        }
      ]
    },
    {
      "metadata": {
        "id": "N94079HsAE9N",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Making prediction\n",
        "\n",
        "Finally, we can predict the label of the target query."
      ]
    },
    {
      "metadata": {
        "id": "bFuRY-AsAYxa",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Shape as batch\n",
        "support_set, targets, labels, _ = episode\n",
        "batch = (\n",
        "    support_set.unsqueeze(0),\n",
        "    targets.unsqueeze(0),\n",
        "    labels.unsqueeze(0))\n",
        "predictions = model(batch)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "R1Pb3WYw__Qi",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "predicted_label = predictions.squeeze().argmax()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "u8iflDebA3bQ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "90dbe39b-e6a4-4d08-b2bc-9bdfb8b1d219"
      },
      "cell_type": "code",
      "source": [
        "reverse_tensor(predicted_label.unsqueeze(0), vocab)[0]"
      ],
      "execution_count": 116,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'particularly'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 116
        }
      ]
    }
  ]
}