SimpleRNN_IntegerEncoding_Embedding.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "authorship_tag": "ABX9TyPAZOeRpEAZdfra1DLM1Z+Z",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/singhsourav0/Deep-Learning-Odyssey/blob/main/21_SimpleRNN_IntegerEncoding_Embedding.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<p><font color=\\\"blueviolet\\\">This notebook contains a simple explanation of why we use Recurrent Neural Networks (RNNs) instead of Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs), and how we can use RNNs with tokenization, including two approaches: integer encoding and embedding.</font></p>\n",
        "\n",
        "**Why RNNs?**\n",
        "- **RNNs for Sequential Data**: <font color='lime'>RNNs are specifically designed to handle sequential data, making them suitable for tasks like text analysis, time series prediction, and speech recognition.</font>\n",
        "\n",
        "- **Capturing Temporal Dependencies**: <font color='lime'>Unlike ANNs and CNNs, RNNs can capture temporal dependencies in data because they maintain a memory of previous inputs, making them effective for tasks where past context is important.</font>\n",
        "\n",
        "**Using RNNs with Tokenization:**\n",
        "1. **Integer Encoding**:\n",
        "   - **Explanation**: <font color='lime'>Integer encoding involves converting each word in a text into a unique integer. This creates a numerical representation of the text, which can be fed into the RNN.</font>\n",
        "   - **How to Use**:<font color='lime'> Use a tokenizer to tokenize the text and assign a unique integer to each word. Then, feed the encoded sequences into the RNN.</font>\n",
        "\n",
        "2. **Embedding**:\n",
        "   - **Explanation**: <font color='lime'>Embedding involves representing words as dense vectors in a high-dimensional space. Each word is mapped to a vector of real numbers, which captures semantic relationships between words.</font>\n",
        "   - **How to Use**: <font color='lime'>Use a tokenizer to tokenize the text and generate word embeddings. These embeddings can then be fed directly into the RNN.</font>\n",
        "\n",
        "**Applying RNNs on IMDB Dataset for Sentiment Analysis:**\n",
        "\n",
        "- <font color='lime'>We demonstrate the application of RNNs for sentiment analysis using the IMDb dataset. We utilize both integer encoding and embedding approaches to preprocess the text data before training a simple RNN model.</font>\n",
        "\n",
        "<font color='teal'>By understanding these concepts, we can leverage RNNs with tokenization to effectively process and analyze sequential data like text in our Deep learning projects.</font>"
      ],
      "metadata": {
        "id": "SMRv9K4Kpd-X"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h1><font color='teal'><b>Integer Encoding"
      ],
      "metadata": {
        "id": "oQYXG8viv0vc"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {
        "id": "TSqrlCn6IjR-"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "docs = ['go india',\n",
        "\t\t'india india',\n",
        "\t\t'hip hip hurray',\n",
        "\t\t'jeetega bhai jeetega india jeetega',\n",
        "\t\t'bharat mata ki jai',\n",
        "\t\t'kohli kohli',\n",
        "\t\t'sachin sachin',\n",
        "\t\t'dhoni dhoni',\n",
        "\t\t'modi ji ki jai',\n",
        "\t\t'inquilab zindabad']"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.preprocessing.text import Tokenizer\n",
        "tokenizer = Tokenizer(oov_token='<nothing>')"
      ],
      "metadata": {
        "id": "T5BqeaTxJuCz"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer.fit_on_texts(docs)"
      ],
      "metadata": {
        "id": "Kym2KDpZKQIU"
      },
      "execution_count": 63,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer.word_index"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-5_44GvTKbka",
        "outputId": "d18501f6-fc5a-4acb-f7bf-6f5ce293b3b8"
      },
      "execution_count": 64,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'<nothing>': 1,\n",
              " 'india': 2,\n",
              " 'jeetega': 3,\n",
              " 'hip': 4,\n",
              " 'ki': 5,\n",
              " 'jai': 6,\n",
              " 'kohli': 7,\n",
              " 'sachin': 8,\n",
              " 'dhoni': 9,\n",
              " 'go': 10,\n",
              " 'hurray': 11,\n",
              " 'bhai': 12,\n",
              " 'bharat': 13,\n",
              " 'mata': 14,\n",
              " 'modi': 15,\n",
              " 'ji': 16,\n",
              " 'inquilab': 17,\n",
              " 'zindabad': 18}"
            ]
          },
          "metadata": {},
          "execution_count": 64
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer.word_counts"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qk-U9NQQKiUp",
        "outputId": "7350d749-2a65-4183-c3a8-653e8c51d0c8"
      },
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "OrderedDict([('go', 1),\n",
              "             ('india', 4),\n",
              "             ('hip', 2),\n",
              "             ('hurray', 1),\n",
              "             ('jeetega', 3),\n",
              "             ('bhai', 1),\n",
              "             ('bharat', 1),\n",
              "             ('mata', 1),\n",
              "             ('ki', 2),\n",
              "             ('jai', 2),\n",
              "             ('kohli', 2),\n",
              "             ('sachin', 2),\n",
              "             ('dhoni', 2),\n",
              "             ('modi', 1),\n",
              "             ('ji', 1),\n",
              "             ('inquilab', 1),\n",
              "             ('zindabad', 1)])"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer.document_count"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pZ-UumWxKqHJ",
        "outputId": "a09f741b-91ab-4a60-9727-57cbeb4fdb8b"
      },
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "10"
            ]
          },
          "metadata": {},
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "sequences = tokenizer.texts_to_sequences(docs)\n",
        "sequences"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Kgfr8vKXKymi",
        "outputId": "15054d4e-d7a1-4364-c44a-b51266be47af"
      },
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[10, 2],\n",
              " [2, 2],\n",
              " [4, 4, 11],\n",
              " [3, 12, 3, 2, 3],\n",
              " [13, 14, 5, 6],\n",
              " [7, 7],\n",
              " [8, 8],\n",
              " [9, 9],\n",
              " [15, 16, 5, 6],\n",
              " [17, 18]]"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.utils import pad_sequences"
      ],
      "metadata": {
        "id": "Vw5ExaPBK21i"
      },
      "execution_count": 68,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "sequences = pad_sequences(sequences, padding = 'post')"
      ],
      "metadata": {
        "id": "7kPxFE0ULNw4"
      },
      "execution_count": 69,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "sequences"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w4NvceQsLbOC",
        "outputId": "fcf93297-99de-40ff-bee5-d26316cd0ce3"
      },
      "execution_count": 70,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[10,  2,  0,  0,  0],\n",
              "       [ 2,  2,  0,  0,  0],\n",
              "       [ 4,  4, 11,  0,  0],\n",
              "       [ 3, 12,  3,  2,  3],\n",
              "       [13, 14,  5,  6,  0],\n",
              "       [ 7,  7,  0,  0,  0],\n",
              "       [ 8,  8,  0,  0,  0],\n",
              "       [ 9,  9,  0,  0,  0],\n",
              "       [15, 16,  5,  6,  0],\n",
              "       [17, 18,  0,  0,  0]], dtype=int32)"
            ]
          },
          "metadata": {},
          "execution_count": 70
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from keras.datasets import imdb\n",
        "from tensorflow.keras.models import Sequential\n",
        "from keras.layers import Dense,SimpleRNN, Embedding, Flatten"
      ],
      "metadata": {
        "id": "4stadJ-aLeOK"
      },
      "execution_count": 71,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "(x_train, y_train), (x_test, y_test) = imdb.load_data()"
      ],
      "metadata": {
        "id": "ElKqvkwOMAcS"
      },
      "execution_count": 72,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x_train[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N6OmmvwvMM9T",
        "outputId": "7f4f74fd-2bb0-43a5-8b68-3a753abebe43"
      },
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1,\n",
              " 14,\n",
              " 22,\n",
              " 16,\n",
              " 43,\n",
              " 530,\n",
              " 973,\n",
              " 1622,\n",
              " 1385,\n",
              " 65,\n",
              " 458,\n",
              " 4468,\n",
              " 66,\n",
              " 3941,\n",
              " 4,\n",
              " 173,\n",
              " 36,\n",
              " 256,\n",
              " 5,\n",
              " 25,\n",
              " 100,\n",
              " 43,\n",
              " 838,\n",
              " 112,\n",
              " 50,\n",
              " 670,\n",
              " 22665,\n",
              " 9,\n",
              " 35,\n",
              " 480,\n",
              " 284,\n",
              " 5,\n",
              " 150,\n",
              " 4,\n",
              " 172,\n",
              " 112,\n",
              " 167,\n",
              " 21631,\n",
              " 336,\n",
              " 385,\n",
              " 39,\n",
              " 4,\n",
              " 172,\n",
              " 4536,\n",
              " 1111,\n",
              " 17,\n",
              " 546,\n",
              " 38,\n",
              " 13,\n",
              " 447,\n",
              " 4,\n",
              " 192,\n",
              " 50,\n",
              " 16,\n",
              " 6,\n",
              " 147,\n",
              " 2025,\n",
              " 19,\n",
              " 14,\n",
              " 22,\n",
              " 4,\n",
              " 1920,\n",
              " 4613,\n",
              " 469,\n",
              " 4,\n",
              " 22,\n",
              " 71,\n",
              " 87,\n",
              " 12,\n",
              " 16,\n",
              " 43,\n",
              " 530,\n",
              " 38,\n",
              " 76,\n",
              " 15,\n",
              " 13,\n",
              " 1247,\n",
              " 4,\n",
              " 22,\n",
              " 17,\n",
              " 515,\n",
              " 17,\n",
              " 12,\n",
              " 16,\n",
              " 626,\n",
              " 18,\n",
              " 19193,\n",
              " 5,\n",
              " 62,\n",
              " 386,\n",
              " 12,\n",
              " 8,\n",
              " 316,\n",
              " 8,\n",
              " 106,\n",
              " 5,\n",
              " 4,\n",
              " 2223,\n",
              " 5244,\n",
              " 16,\n",
              " 480,\n",
              " 66,\n",
              " 3785,\n",
              " 33,\n",
              " 4,\n",
              " 130,\n",
              " 12,\n",
              " 16,\n",
              " 38,\n",
              " 619,\n",
              " 5,\n",
              " 25,\n",
              " 124,\n",
              " 51,\n",
              " 36,\n",
              " 135,\n",
              " 48,\n",
              " 25,\n",
              " 1415,\n",
              " 33,\n",
              " 6,\n",
              " 22,\n",
              " 12,\n",
              " 215,\n",
              " 28,\n",
              " 77,\n",
              " 52,\n",
              " 5,\n",
              " 14,\n",
              " 407,\n",
              " 16,\n",
              " 82,\n",
              " 10311,\n",
              " 8,\n",
              " 4,\n",
              " 107,\n",
              " 117,\n",
              " 5952,\n",
              " 15,\n",
              " 256,\n",
              " 4,\n",
              " 31050,\n",
              " 7,\n",
              " 3766,\n",
              " 5,\n",
              " 723,\n",
              " 36,\n",
              " 71,\n",
              " 43,\n",
              " 530,\n",
              " 476,\n",
              " 26,\n",
              " 400,\n",
              " 317,\n",
              " 46,\n",
              " 7,\n",
              " 4,\n",
              " 12118,\n",
              " 1029,\n",
              " 13,\n",
              " 104,\n",
              " 88,\n",
              " 4,\n",
              " 381,\n",
              " 15,\n",
              " 297,\n",
              " 98,\n",
              " 32,\n",
              " 2071,\n",
              " 56,\n",
              " 26,\n",
              " 141,\n",
              " 6,\n",
              " 194,\n",
              " 7486,\n",
              " 18,\n",
              " 4,\n",
              " 226,\n",
              " 22,\n",
              " 21,\n",
              " 134,\n",
              " 476,\n",
              " 26,\n",
              " 480,\n",
              " 5,\n",
              " 144,\n",
              " 30,\n",
              " 5535,\n",
              " 18,\n",
              " 51,\n",
              " 36,\n",
              " 28,\n",
              " 224,\n",
              " 92,\n",
              " 25,\n",
              " 104,\n",
              " 4,\n",
              " 226,\n",
              " 65,\n",
              " 16,\n",
              " 38,\n",
              " 1334,\n",
              " 88,\n",
              " 12,\n",
              " 16,\n",
              " 283,\n",
              " 5,\n",
              " 16,\n",
              " 4472,\n",
              " 113,\n",
              " 103,\n",
              " 32,\n",
              " 15,\n",
              " 16,\n",
              " 5345,\n",
              " 19,\n",
              " 178,\n",
              " 32]"
            ]
          },
          "metadata": {},
          "execution_count": 73
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(x_train[0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fbPdVpwPMRmi",
        "outputId": "4ca8f552-85fa-421f-9b6a-01708c68c1ab"
      },
      "execution_count": 74,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "218"
            ]
          },
          "metadata": {},
          "execution_count": 74
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(x_train[2])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "doirPtl7MYH2",
        "outputId": "cb425c11-0949-4bb3-a163-0ba4954d8199"
      },
      "execution_count": 75,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "141"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x_train = pad_sequences(x_train, padding = 'post', maxlen =50)\n",
        "x_test = pad_sequences(x_test, padding = 'post', maxlen = 50)"
      ],
      "metadata": {
        "id": "0mDc5pWpMazn"
      },
      "execution_count": 76,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x_train[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "01VqYSU1M6tS",
        "outputId": "708ac8ed-0ba3-435c-d44f-03287e72fd1f"
      },
      "execution_count": 77,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([2071,   56,   26,  141,    6,  194, 7486,   18,    4,  226,   22,\n",
              "         21,  134,  476,   26,  480,    5,  144,   30, 5535,   18,   51,\n",
              "         36,   28,  224,   92,   25,  104,    4,  226,   65,   16,   38,\n",
              "       1334,   88,   12,   16,  283,    5,   16, 4472,  113,  103,   32,\n",
              "         15,   16, 5345,   19,  178,   32], dtype=int32)"
            ]
          },
          "metadata": {},
          "execution_count": 77
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = Sequential()\n",
        "model.add(SimpleRNN(32,input_shape =(50,1),return_sequences=False))\n",
        "model.add(Dense(1, activation ='sigmoid'))"
      ],
      "metadata": {
        "id": "ubGR-E7eNC_i"
      },
      "execution_count": 78,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EueAhBLsOgHi",
        "outputId": "e79cb282-caa5-4370-c062-25feb7c67c1a"
      },
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_7\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " simple_rnn_3 (SimpleRNN)    (None, 32)                1088      \n",
            "                                                                 \n",
            " dense_3 (Dense)             (None, 1)                 33        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 1121 (4.38 KB)\n",
            "Trainable params: 1121 (4.38 KB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(loss ='binary_crossentropy', optimizer = 'adam', metrics =['accuracy'] )\n",
        "model.fit(x_train, y_train,epochs =5, validation_data =(x_test, y_test))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FkLvCuCnQZ2y",
        "outputId": "fbe72bcb-b5f2-4829-e9ff-bdad1bd6d686"
      },
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "782/782 [==============================] - 16s 19ms/step - loss: 0.6945 - accuracy: 0.5072 - val_loss: 0.6952 - val_accuracy: 0.5014\n",
            "Epoch 2/5\n",
            "782/782 [==============================] - 11s 14ms/step - loss: 0.6931 - accuracy: 0.5058 - val_loss: 0.6941 - val_accuracy: 0.5041\n",
            "Epoch 3/5\n",
            "782/782 [==============================] - 11s 14ms/step - loss: 0.6927 - accuracy: 0.5070 - val_loss: 0.6944 - val_accuracy: 0.5052\n",
            "Epoch 4/5\n",
            "782/782 [==============================] - 13s 17ms/step - loss: 0.6929 - accuracy: 0.5039 - val_loss: 0.6955 - val_accuracy: 0.5015\n",
            "Epoch 5/5\n",
            "782/782 [==============================] - 13s 17ms/step - loss: 0.6930 - accuracy: 0.5053 - val_loss: 0.6937 - val_accuracy: 0.5034\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7fdde1d83010>"
            ]
          },
          "metadata": {},
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "myE3IAAPRV7S"
      },
      "execution_count": 80,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font color='coral'>Using embeddings instead of integer encoding allows the model to understand the meaning and context of words, reduces computational complexity, adapts to the task during training, generalizes better to unseen words, and leverages pre-trained representations for improved performance.</font>\n",
        "<p><font color=\\\"tea\\\">Overall, using word embeddings instead of integer encoding enhances the model's ability to understand and process natural language by capturing semantic relationships, reducing dimensionality, enabling learning of parameters, facilitating generalization, and leveraging pre-trained embeddings for improved performance.</p></font>"
      ],
      "metadata": {
        "id": "7VwsRMcnR2QW"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h1><font color='teal'><b>Embedding"
      ],
      "metadata": {
        "id": "3wTy6yK4wJ8L"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "docs = ['go india',\n",
        "\t\t'india india',\n",
        "\t\t'hip hip hurray',\n",
        "\t\t'jeetega bhai jeetega india jeetega',\n",
        "\t\t'bharat mata ki jai',\n",
        "\t\t'kohli kohli',\n",
        "\t\t'sachin sachin',\n",
        "\t\t'dhoni dhoni',\n",
        "\t\t'modi ji ki jai',\n",
        "\t\t'inquilab zindabad']"
      ],
      "metadata": {
        "id": "WEONtwA6R1kd"
      },
      "execution_count": 81,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.preprocessing.text import Tokenizer\n",
        "tokenizer = Tokenizer()"
      ],
      "metadata": {
        "id": "D0Uktyg3R41l"
      },
      "execution_count": 82,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer.fit_on_texts(docs)"
      ],
      "metadata": {
        "id": "YIO09pRjSM7T"
      },
      "execution_count": 83,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "len(tokenizer.word_index)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R2FwFAVISaM7",
        "outputId": "0bbf3735-569e-4fa3-a02f-922e06966d5f"
      },
      "execution_count": 84,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "17"
            ]
          },
          "metadata": {},
          "execution_count": 84
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "sequences = tokenizer.texts_to_sequences(docs)\n",
        "sequences"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NwXV4gDtShrq",
        "outputId": "2590bad6-7321-4285-b658-026615eda21f"
      },
      "execution_count": 85,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[9, 1],\n",
              " [1, 1],\n",
              " [3, 3, 10],\n",
              " [2, 11, 2, 1, 2],\n",
              " [12, 13, 4, 5],\n",
              " [6, 6],\n",
              " [7, 7],\n",
              " [8, 8],\n",
              " [14, 15, 4, 5],\n",
              " [16, 17]]"
            ]
          },
          "metadata": {},
          "execution_count": 85
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.utils import pad_sequences\n",
        "sequences = pad_sequences(sequences,padding='post')\n",
        "sequences"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rZ5SdE3JSvMD",
        "outputId": "b2b11c03-c192-4227-c728-a6f80be2c08e"
      },
      "execution_count": 86,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 9,  1,  0,  0,  0],\n",
              "       [ 1,  1,  0,  0,  0],\n",
              "       [ 3,  3, 10,  0,  0],\n",
              "       [ 2, 11,  2,  1,  2],\n",
              "       [12, 13,  4,  5,  0],\n",
              "       [ 6,  6,  0,  0,  0],\n",
              "       [ 7,  7,  0,  0,  0],\n",
              "       [ 8,  8,  0,  0,  0],\n",
              "       [14, 15,  4,  5,  0],\n",
              "       [16, 17,  0,  0,  0]], dtype=int32)"
            ]
          },
          "metadata": {},
          "execution_count": 86
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model2 = Sequential()\n",
        "\n",
        "model2.add(Embedding(17, output_dim=2, input_length = 5))"
      ],
      "metadata": {
        "id": "WZPmuiFRSyLL"
      },
      "execution_count": 102,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model2.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O9VlycxrTi3C",
        "outputId": "b4ab2dda-ac2f-400c-9e14-fdf3f1b409c2"
      },
      "execution_count": 103,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_10\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " embedding_7 (Embedding)     (None, 5, 2)              34        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 34 (136.00 Byte)\n",
            "Trainable params: 34 (136.00 Byte)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model2.compile('adam','accuracy')"
      ],
      "metadata": {
        "id": "OdaQa5bqTlzT"
      },
      "execution_count": 104,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(sequences)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "F43VZ4cYihQO",
        "outputId": "7d823b02-8fb3-4428-e96b-073b73cc6a3f"
      },
      "execution_count": 106,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[ 9  1  0  0  0]\n",
            " [ 1  1  0  0  0]\n",
            " [ 3  3 10  0  0]\n",
            " [ 2 11  2  1  2]\n",
            " [12 13  4  5  0]\n",
            " [ 6  6  0  0  0]\n",
            " [ 7  7  0  0  0]\n",
            " [ 8  8  0  0  0]\n",
            " [14 15  4  5  0]\n",
            " [16 17  0  0  0]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.datasets import imdb\n",
        "from keras.preprocessing.text import Tokenizer\n",
        "from keras.utils import pad_sequences\n",
        "from keras import Sequential\n",
        "from keras.layers import Dense,SimpleRNN,Embedding,Flatten"
      ],
      "metadata": {
        "id": "3Njxq6avUG9j"
      },
      "execution_count": 132,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "(X_train,Y_train),(X_test,Y_test) = imdb.load_data()"
      ],
      "metadata": {
        "id": "AFrS-nmQUMu6"
      },
      "execution_count": 117,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_train = pad_sequences(X_train,padding='post',maxlen=50)\n",
        "X_test = pad_sequences(X_test,padding='post',maxlen=50)"
      ],
      "metadata": {
        "id": "wi8sZHL_UO35"
      },
      "execution_count": 118,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model2 = Sequential()\n",
        "model2.add(Embedding(10000, 2, input_length=50, embeddings_initializer='uniform'))\n",
        "model2.add(SimpleRNN(32, return_sequences=False))\n",
        "model2.add(Dense(1, activation='sigmoid'))"
      ],
      "metadata": {
        "id": "jcNYRk3_YvNa"
      },
      "execution_count": 126,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model2.summary()"
      ],
      "metadata": {
        "id": "dnosQh2nZTvU",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d0479d86-0713-455a-b9aa-97597509dd89"
      },
      "execution_count": 122,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_12\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " embedding_9 (Embedding)     (None, 50, 2)             20000     \n",
            "                                                                 \n",
            " simple_rnn_6 (SimpleRNN)    (None, 32)                1120      \n",
            "                                                                 \n",
            " dense_6 (Dense)             (None, 1)                 33        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 21153 (82.63 KB)\n",
            "Trainable params: 21153 (82.63 KB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "from keras.datasets import imdb\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Embedding, SimpleRNN, Dense\n",
        "from keras.preprocessing import sequence\n",
        "\n",
        "# Set the parameters\n",
        "max_features = 10000  # Number of words to consider as features\n",
        "maxlen = 100  # Cut texts after this number of words (among top max_features most common words)\n",
        "batch_size = 32\n",
        "\n",
        "print('Loading data...')\n",
        "(input_train, y_train), (input_test, y_test) = imdb.load_data(num_words=max_features)\n",
        "print(len(input_train), 'train sequences')\n",
        "print(len(input_test), 'test sequences')\n",
        "\n",
        "print('Pad sequences (samples x time)')\n",
        "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
        "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
        "print('input_train shape:', input_train.shape)\n",
        "print('input_test shape:', input_test.shape)\n",
        "\n",
        "# Build the model\n",
        "model = Sequential()\n",
        "model.add(Embedding(max_features, 32))\n",
        "model.add(SimpleRNN(32))\n",
        "model.add(Dense(1, activation='sigmoid'))\n",
        "\n",
        "# Compile the model\n",
        "model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\n",
        "\n",
        "# Train the model\n",
        "history = model.fit(input_train, y_train,\n",
        "                    epochs=10,\n",
        "                    batch_size=batch_size,\n",
        "                    validation_split=0.2)\n",
        "\n",
        "# Evaluate the model\n",
        "loss, accuracy = model.evaluate(input_test, y_test)\n",
        "print('Test loss:', loss)\n",
        "print('Test accuracy:', accuracy)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_Y0QLn0sjRnT",
        "outputId": "dc0ff8c9-08f1-49ce-bea7-2a9cf8a16a6d"
      },
      "execution_count": 134,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Loading data...\n",
            "25000 train sequences\n",
            "25000 test sequences\n",
            "Pad sequences (samples x time)\n",
            "input_train shape: (25000, 100)\n",
            "input_test shape: (25000, 100)\n",
            "Epoch 1/10\n",
            "625/625 [==============================] - 20s 30ms/step - loss: 0.5910 - accuracy: 0.6787 - val_loss: 0.4400 - val_accuracy: 0.7986\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 19s 31ms/step - loss: 0.3393 - accuracy: 0.8568 - val_loss: 0.4055 - val_accuracy: 0.8284\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 19s 30ms/step - loss: 0.2144 - accuracy: 0.9168 - val_loss: 0.4591 - val_accuracy: 0.7974\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 18s 29ms/step - loss: 0.1327 - accuracy: 0.9518 - val_loss: 0.5529 - val_accuracy: 0.8062\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 17s 27ms/step - loss: 0.0579 - accuracy: 0.9824 - val_loss: 0.6413 - val_accuracy: 0.7982\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 18s 29ms/step - loss: 0.0301 - accuracy: 0.9911 - val_loss: 0.8393 - val_accuracy: 0.7670\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 18s 29ms/step - loss: 0.0256 - accuracy: 0.9930 - val_loss: 0.8615 - val_accuracy: 0.7922\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 18s 29ms/step - loss: 0.0327 - accuracy: 0.9897 - val_loss: 1.1185 - val_accuracy: 0.7108\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 17s 27ms/step - loss: 0.0217 - accuracy: 0.9931 - val_loss: 1.0260 - val_accuracy: 0.7762\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 17s 27ms/step - loss: 0.0142 - accuracy: 0.9955 - val_loss: 1.0303 - val_accuracy: 0.7810\n",
            "782/782 [==============================] - 6s 8ms/step - loss: 1.0124 - accuracy: 0.7742\n",
            "Test loss: 1.0124222040176392\n",
            "Test accuracy: 0.7741600275039673\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "QNxbBQvUpE58"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}