lectures/cap13_1_embeddings_principles.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/viniciusrpb/cic0269_natural_language_processing/blob/main/lectures/cap13_1_embeddings_principles.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Capítulo 13 - Word Embeddings\n",
        "\n",
        "Essa aula se baseia:\n",
        "\n",
        "\n",
        "\n",
        "*   No artigo \"*Distributed Representations of Words and Phrases\n",
        "and their Compositionality*\", de Mikolov et al. (2013). Clique [aqui](https://proceedings.neurips.cc/paper/2013/file/9aa42b31882ec039965f3c4923ce901b-Paper.pdf) para acesso.\n",
        "*   Na documentação do word2vec da biblioteca [Gensim](https://radimrehurek.com/gensim/models/word2vec.html).\n",
        "\n",
        "\n",
        "\n",
        "## 13.1. Princípios de Word Embeddings"
      ],
      "metadata": {
        "id": "hebpaQlpKSyS"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lBhH-LrUBPQC"
      },
      "outputs": [],
      "source": [
        "!pip install keras\n",
        "!pip install tensorflow\n",
        "!pip install -U gensim"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "KdvNPSjGbgpV"
      },
      "outputs": [],
      "source": [
        "from keras.models import Sequential\n",
        "from keras.layers import Dense,Embedding,Activation,Dropout,SimpleRNN,BatchNormalization,RNN,Flatten,Input,LSTM,Bidirectional\n",
        "from keras.utils.np_utils import to_categorical\n",
        "import tensorflow as tf\n",
        "import tensorflow_datasets as tfds\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras.optimizers import SGD\n",
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import gensim\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.model_selection import train_test_split"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Treinando um modelo Word2vec"
      ],
      "metadata": {
        "id": "kfJVldaf84XJ"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "nJ_EgSxZrl5a"
      },
      "outputs": [],
      "source": [
        "from gensim.models import Word2Vec\n",
        "corpus = [[\"hello\", \"world\",\"hi\",\"earth\",\"sunshine\",\"law\"],[\"cat\", \"say\", \"meow\"], [\"dog\", \"say\", \"woof\"]]\n",
        "model = Word2Vec(sentences=corpus, vector_size=5, window=5, min_count=1, workers=4)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "U9RW4p9fsEQD"
      },
      "outputs": [],
      "source": [
        "word_vectors = model.wv"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "hzlmbdvNsJL2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "48ea812c-6dff-44ea-a352-527a0dab2b59"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "5"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "len(word_vectors[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "RVGruelZs0Ol",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e54bf0c7-f62d-4938-83e4-27adef324f66"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 0.15723471, -0.24314074,  0.2750364 ,  0.08315043,  0.01469049],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "word_vectors['world'] - word_vectors['hello']"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Podemos retreinar o modelo com novas palavras"
      ],
      "metadata": {
        "id": "hj337Hog60gY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.train([[\"dear\", \"bear\", \"cream\"]], total_examples=3, epochs=1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wHFzJLzI60ZY",
        "outputId": "8ff953c8-cb06-4b2d-d3be-1517be240544"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:gensim.models.word2vec:Effective 'alpha' higher than previous training cycles\n",
            "WARNING:gensim.models.word2vec:EPOCH 0: supplied example count (1) did not equal expected count (3)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(0, 3)"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Carregando Modelos Treinados em Outros Corpora"
      ],
      "metadata": {
        "id": "rUy5GSUF8bSK"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "W4SlpvQp25rs",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e5bf40a7-8667-4f2d-f79d-61aceb0a9d0d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['fasttext-wiki-news-subwords-300', 'conceptnet-numberbatch-17-06-300', 'word2vec-ruscorpora-300', 'word2vec-google-news-300', 'glove-wiki-gigaword-50', 'glove-wiki-gigaword-100', 'glove-wiki-gigaword-200', 'glove-wiki-gigaword-300', 'glove-twitter-25', 'glove-twitter-50', 'glove-twitter-100', 'glove-twitter-200', '__testing_word2vec-matrix-synopsis']\n"
          ]
        }
      ],
      "source": [
        "import gensim.downloader\n",
        "\n",
        "print(list(gensim.downloader.info()['models'].keys()))"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "word_vectors = gensim.downloader.load('word2vec-google-news-300')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UdCL6QHG8GKk",
        "outputId": "007e6db4-bb0a-4b20-e913-9b340ad000f5"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[==================================================] 100.0% 1662.8/1662.8MB downloaded\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "word_vectors.most_similar('car')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Yna_FcaF8KTj",
        "outputId": "f969e8d8-8449-4a00-986f-b1ee94ef7f92"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('vehicle', 0.7821096181869507),\n",
              " ('cars', 0.7423831224441528),\n",
              " ('SUV', 0.7160962224006653),\n",
              " ('minivan', 0.6907036900520325),\n",
              " ('truck', 0.6735789775848389),\n",
              " ('Car', 0.6677608489990234),\n",
              " ('Ford_Focus', 0.667320191860199),\n",
              " ('Honda_Civic', 0.6626849174499512),\n",
              " ('Jeep', 0.651133120059967),\n",
              " ('pickup_truck', 0.6441438794136047)]"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "word_vectors['airplane']-word_vectors['flight']"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wxgsUPW_ZcLk",
        "outputId": "d80e553f-2c2d-4ee9-e2fa-6b07d7e582c3"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 2.53417969e-01, -2.23144531e-01,  3.80859375e-02,  2.54394531e-01,\n",
              "       -1.92871094e-01, -1.28906250e-01, -3.61328125e-02,  2.16796875e-01,\n",
              "        2.08007812e-01,  1.16699219e-01,  1.44042969e-01, -1.10351562e-01,\n",
              "        9.17968750e-02, -2.11425781e-01, -6.05468750e-02,  1.19140625e-01,\n",
              "       -1.39404297e-01,  1.58508301e-01,  2.34375000e-02, -4.05273438e-02,\n",
              "        5.56640625e-02,  5.85021973e-02,  1.89697266e-01, -2.05078125e-02,\n",
              "        4.58526611e-03, -7.69042969e-02,  6.83593750e-02, -4.39453125e-03,\n",
              "       -7.51953125e-02, -1.25976562e-01, -2.45117188e-01, -1.58203125e-01,\n",
              "       -1.97753906e-01,  2.93212891e-01,  2.24243164e-01,  1.37695312e-01,\n",
              "        2.87658691e-01, -9.76562500e-02, -3.07617188e-02,  1.98242188e-01,\n",
              "        7.30438232e-02,  5.85937500e-03, -1.48437500e-01,  7.95288086e-02,\n",
              "       -5.29785156e-02,  1.97265625e-01, -1.09863281e-01, -2.58789062e-02,\n",
              "        1.17187500e-02, -1.85546875e-01, -1.45019531e-01,  2.28515625e-01,\n",
              "       -1.41601562e-02, -9.98535156e-02, -1.67724609e-01,  2.03125000e-01,\n",
              "        9.13085938e-02, -2.61230469e-02,  2.07031250e-01,  1.75781250e-02,\n",
              "        1.39648438e-01, -1.73828125e-01, -2.76184082e-01,  7.51953125e-02,\n",
              "        2.00683594e-01,  8.53271484e-02,  2.92968750e-03, -9.76562500e-03,\n",
              "       -2.62451172e-01,  6.56738281e-02, -4.34570312e-02, -2.16552734e-01,\n",
              "       -1.29638672e-01,  7.36083984e-02, -1.01562500e-01, -5.37109375e-03,\n",
              "       -2.14843750e-02, -2.11425781e-01,  1.34277344e-01, -6.15234375e-02,\n",
              "       -1.34277344e-02,  1.56250000e-02, -1.03637695e-01, -2.73437500e-02,\n",
              "       -1.20849609e-01, -4.49218750e-02,  1.76330566e-01,  1.81579590e-01,\n",
              "       -4.78515625e-02,  9.86328125e-02, -1.34033203e-01,  2.93823242e-01,\n",
              "        9.66796875e-02,  9.03320312e-02, -9.13085938e-02, -1.03515625e-01,\n",
              "        8.71582031e-02, -2.04833984e-01,  1.93847656e-01, -1.44531250e-01,\n",
              "       -1.75781250e-01, -2.55859375e-01,  1.42822266e-01,  2.44140625e-02,\n",
              "        1.67968750e-01,  1.46484375e-01, -1.06445312e-01, -8.44726562e-02,\n",
              "       -1.68945312e-01,  3.54003906e-03, -3.16406250e-01, -2.11914062e-01,\n",
              "       -6.73828125e-02, -1.05834961e-01,  2.36328125e-01, -4.95605469e-02,\n",
              "        1.13769531e-01,  6.54296875e-02, -6.00585938e-02,  4.74853516e-02,\n",
              "        1.48925781e-01, -9.57031250e-02,  3.22265625e-02, -3.43017578e-01,\n",
              "        3.70483398e-02,  9.75341797e-02,  6.54296875e-02, -8.83789062e-02,\n",
              "       -6.83593750e-03,  1.87500000e-01,  8.00781250e-02,  3.90625000e-02,\n",
              "       -1.22070312e-01,  3.85742188e-01,  1.62109375e-01,  2.34375000e-01,\n",
              "       -7.29980469e-02, -3.03710938e-01,  1.66503906e-01, -2.49267578e-01,\n",
              "       -6.84814453e-02,  1.04003906e-01, -1.38671875e-01,  2.44140625e-03,\n",
              "        1.80541992e-01, -2.34863281e-01,  8.00781250e-02, -1.07421875e-02,\n",
              "       -1.97753906e-02, -3.61328125e-02,  9.76562500e-03, -1.95800781e-01,\n",
              "       -1.48925781e-01, -1.61743164e-02,  2.73437500e-02,  5.04882812e-01,\n",
              "        8.78906250e-02,  1.20117188e-01, -1.45996094e-01,  1.81396484e-01,\n",
              "        1.85058594e-01,  1.46484375e-02, -1.32385254e-01, -1.41601562e-01,\n",
              "        2.20703125e-01,  1.08764648e-01,  4.88281250e-02, -6.68945312e-02,\n",
              "       -9.32617188e-02,  8.88671875e-02, -7.32421875e-02, -1.14501953e-01,\n",
              "       -1.86523438e-01, -1.39648438e-01, -3.39355469e-02, -1.99462891e-01,\n",
              "        1.95556641e-01, -1.27929688e-01, -7.61718750e-02, -4.62646484e-02,\n",
              "       -3.71093750e-02, -2.59765625e-01, -1.58935547e-01, -9.47265625e-02,\n",
              "        1.75781250e-02, -2.04589844e-01,  5.76171875e-02,  8.76464844e-02,\n",
              "       -1.56250000e-02,  1.55273438e-01,  1.21582031e-01,  9.66796875e-02,\n",
              "       -1.70410156e-01,  6.64062500e-02,  9.76562500e-04, -1.92382812e-01,\n",
              "        2.06298828e-01, -1.87011719e-01,  1.26953125e-02,  2.48046875e-01,\n",
              "       -8.00781250e-02, -1.65893555e-01, -2.44140625e-01, -1.33789062e-01,\n",
              "       -1.99279785e-01, -8.64257812e-02,  6.66503906e-02, -2.44140625e-04,\n",
              "       -1.01562500e-01,  2.05810547e-01,  1.00585938e-01, -2.05383301e-01,\n",
              "        1.38916016e-01,  4.23812866e-02,  6.44531250e-02,  5.22460938e-02,\n",
              "       -2.51464844e-01,  2.59765625e-01,  5.55877686e-02, -4.09240723e-02,\n",
              "        2.02636719e-01,  6.54296875e-02,  2.14843750e-02, -1.92871094e-01,\n",
              "        3.25195312e-01,  5.76171875e-02,  1.60156250e-01,  1.65039062e-01,\n",
              "       -1.46484375e-01, -6.62841797e-02, -1.80664062e-01,  1.41113281e-01,\n",
              "       -5.66406250e-02, -1.71875000e-01, -2.78808594e-01, -3.05175781e-02,\n",
              "       -8.78906250e-03,  5.65795898e-02,  7.95898438e-02,  2.35595703e-01,\n",
              "        1.85058594e-01, -1.04980469e-02, -1.95800781e-01,  2.65625000e-01,\n",
              "        2.26684570e-01, -1.32812500e-01, -2.53906250e-02,  1.38671875e-01,\n",
              "        1.33789062e-01, -3.32031250e-01, -9.93652344e-02, -8.00781250e-02,\n",
              "        1.96777344e-01, -1.51855469e-01,  2.70996094e-02, -6.39648438e-02,\n",
              "        1.12304688e-02, -1.68945312e-01,  2.79541016e-02, -1.98730469e-01,\n",
              "        9.52148438e-03, -1.62109375e-01,  4.42504883e-02, -7.81250000e-02,\n",
              "       -7.71484375e-02,  1.07421875e-01, -1.97631836e-01,  3.32031250e-02,\n",
              "       -3.12500000e-02,  1.65100098e-01,  6.83593750e-03,  8.78906250e-03,\n",
              "        3.05664062e-01, -2.05444336e-01, -2.72949219e-01, -9.66796875e-02,\n",
              "        1.91375732e-01, -7.34863281e-02,  2.92968750e-02, -2.08251953e-01,\n",
              "       -9.37500000e-02, -1.91162109e-01,  1.14257812e-01,  7.66601562e-02,\n",
              "       -1.72950745e-01, -2.79296875e-01,  4.19921875e-02, -1.67480469e-01,\n",
              "        1.38671875e-01, -2.63916016e-01, -3.86962891e-01, -9.93652344e-02,\n",
              "        6.90917969e-02, -1.40136719e-01, -1.16210938e-01,  2.97851562e-02,\n",
              "       -1.07421875e-01,  4.19921875e-02, -2.07641602e-01, -1.46484375e-02],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "result = (word_vectors['airplane']+word_vectors['flight'])-word_vectors['ship']\n",
        "print(result)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-7o2_0i0FuTa",
        "outputId": "f8536699-b098-47fc-862a-0b7330a9f543"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[-0.19482422 -0.46826172 -0.14648438  0.12548828 -0.33544922 -0.52368164\n",
            " -0.171875   -0.4296875   0.19042969 -0.03369141  0.33666992 -0.21191406\n",
            "  0.4609375  -0.12548828 -0.05224609  0.22460938  0.22924805  0.25360107\n",
            "  0.19677734  0.24560547 -0.29492188 -0.12802124 -0.31689453 -0.31933594\n",
            " -0.32746124  0.02954102 -0.39038086 -0.22509766  0.10644531  0.4794922\n",
            " -0.28808594 -0.3334961  -0.05810547 -0.08178711 -0.22473145 -0.01464844\n",
            " -0.07659912  0.2529297   0.17871094  0.5859375   0.05122375 -0.00195312\n",
            "  0.6191406   0.16119385 -0.04638672 -0.56640625  0.4267578  -0.1743164\n",
            "  0.78808594 -0.04882812 -0.5805664  -0.07519531 -0.32470703  0.23022461\n",
            " -0.07110596 -0.45117188  0.19335938 -0.04370117  0.19580078  0.2446289\n",
            "  0.12402344 -0.46661377  0.05938721 -0.05102539 -0.00793457 -0.06005859\n",
            " -0.10253906  0.19628906  0.47387695 -0.01782227 -0.171875   -0.31030273\n",
            " -0.17602539  0.09411621 -0.25        0.10791016  0.21543121 -0.2866211\n",
            "  0.09283447  0.0625     -0.11743164  0.10601807  0.20202637 -0.05664062\n",
            " -0.00301361 -0.39941406 -0.1359253   0.22662354  0.03222656  0.40527344\n",
            " -0.16674805 -0.45031738 -0.47705078 -0.17919922 -0.02099609 -0.24804688\n",
            "  0.19763184 -0.42993164 -0.12451172 -0.69140625  0.3125     -0.125\n",
            "  0.00622559  0.10223389  0.64941406  0.18823242  0.6484375   0.27392578\n",
            "  0.34179688 -0.17596436 -0.3203125   0.07714844 -0.3717041  -0.03967285\n",
            " -0.8088379  -0.19213867 -0.10021973  0.42138672 -0.3413086  -0.0736084\n",
            "  0.03173828 -0.16601562  0.04589844 -0.5031738  -0.06622314  0.17150879\n",
            " -0.11523438 -0.2980957   0.63964844  0.5383301   0.02050781 -0.46679688\n",
            " -0.05664062 -0.13647461  0.12548828 -0.6513672  -0.23266602 -0.02441406\n",
            "  0.06991577  0.20678711  0.03137207 -0.32202148 -0.07324219  0.03308105\n",
            "  0.18151855  0.10693359 -0.45056152 -0.4423828  -0.01977539 -0.21240234\n",
            " -0.15820312 -0.01733398  0.44677734 -0.17010498 -0.05273438 -0.8895569\n",
            "  0.22705078 -0.13427734 -0.1694336   0.10018921 -0.46826172  0.11328125\n",
            "  0.10284424  0.34570312 -0.37304688  0.36975098  0.18359375 -0.05810547\n",
            "  0.21972656 -0.25683594 -0.02929688 -0.15979004 -0.14929199 -0.05273438\n",
            "  0.05737305  0.15258789 -0.10229492 -0.2298584  -0.05566406  0.12658691\n",
            " -0.08691406 -0.17578125  0.07299805 -0.5605469   0.31054688  0.10058594\n",
            " -0.3046875   0.23754883 -0.35498047  0.16894531  0.24780273  0.40112305\n",
            "  0.01611328  0.65600586 -0.09960938  0.37402344 -0.34106445 -0.43017578\n",
            " -0.32647705 -0.33789062 -0.21679688  0.14733887 -0.43066406 -0.35351562\n",
            "  0.2855835   0.04736328  0.10693359  0.14501953 -0.40527344 -0.06762695\n",
            " -0.32757568 -0.0413208  -0.15185547 -0.34316254 -0.11328125  0.2475586\n",
            "  0.20166016  1.0751953  -0.11116028 -0.11386108 -0.37646484 -0.5234375\n",
            "  0.05273438  0.24072266  0.75390625  0.01513672  0.05761719  0.04003906\n",
            " -0.7910156  -0.17932129  0.40185547 -0.26576614 -0.10351562 -0.57177734\n",
            " -0.31298828  0.00952148 -0.4326172   0.05974579  0.2130127  -0.3293457\n",
            "  0.10888672 -0.08563232 -0.24633789 -0.64746094 -0.15478516  0.01367188\n",
            " -0.5498047  -0.6142578   0.02270508  0.15234375 -0.13769531  0.11279297\n",
            " -0.01904297 -0.0090332   0.18139648  0.07470703  0.37060547  0.64941406\n",
            "  0.08349609 -0.16845703 -0.15795898  0.72265625  0.32305908  0.33642578\n",
            " -0.09716797  0.45703125  0.30822754 -0.2421875  -0.74560547  0.17340088\n",
            " -0.30419922  0.21923828 -0.02050781 -0.01403809 -0.35498047 -0.2685547\n",
            " -0.10598755  0.1496582  -0.22167969 -0.22875977  0.23828125  0.05834961\n",
            " -0.3095703  -0.2793579  -0.13584137  0.08312988 -0.11230469  0.38623047\n",
            " -0.66796875 -0.0949707  -0.38549805 -0.11791992  0.06469727  0.15670776\n",
            "  0.04785156  0.         -0.31201172  0.03027344 -0.03771973 -0.37121582]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Utilizando o método ```most_similar```, podemos retornar as palavras mais similares por meio de uma operação aritmética:\n",
        "somam-se os vetores positivos e subtrai o vetor negativo. A partir do resultado, podemos obter as palavras mais similares comparando-se esse vetor resultante com os vetores das demais palavras do word embedding com base na similaridade cosseno. Abaixo as top 10 palavras mais similares são mostradas:"
      ],
      "metadata": {
        "id": "GpeN2Ir77k3m"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "ecwIN1jW3g4U",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d45dd47f-5255-4ae8-c762-460e83dd82ba"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('plane', 0.6277297735214233),\n",
              " ('jet', 0.5784463882446289),\n",
              " ('flights', 0.5631440877914429),\n",
              " ('airliner', 0.5585241913795471),\n",
              " ('aircraft', 0.5546182990074158),\n",
              " ('jetliner', 0.550014853477478),\n",
              " ('NOTE_Expedia_Expedia.com', 0.5478827357292175),\n",
              " ('airplanes', 0.5451778173446655),\n",
              " ('Flight', 0.5407993197441101),\n",
              " ('airline', 0.5332231521606445)]"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "word_vectors.most_similar(positive=['airplane','flight'],negative=['ship'],topn=10) "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "xCb3VPHU38E7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d4fd0aa4-d797-499c-a22c-e67c1bdf9b76"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.39203462"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "word_vectors.similarity('orange','apple')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "QZhH0szG4CVx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2b91a55b-15ac-476a-8c2a-283bd1401d74"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.6421891"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "word_vectors.similarity('orange','blue')"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "provenance": [],
      "authorship_tag": "ABX9TyPmtz3zjr/EKMGO6hKj/vsV",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}