C3/W1/ungraded_labs/C3_W1_Lab_3_sarcasm.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/hobbang2/TensorFlowCertification/blob/main/C3/W1/ungraded_labs/C3_W1_Lab_3_sarcasm.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IXFlXpalw-Mc"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/https-deeplearning-ai/tensorflow-1-public/blob/master/C3/W1/ungraded_labs/C3_W1_Lab_3_sarcasm.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SdNGfEo2u-r7"
      },
      "source": [
        "# Ungraded Lab: Tokenizing the Sarcasm Dataset\n",
        "\n",
        "In this lab, you will be applying what you've learned in the past two exercises to preprocess the [News Headlines Dataset for Sarcasm Detection](https://www.kaggle.com/rmisra/news-headlines-dataset-for-sarcasm-detection/home). This contains news headlines which are labeled as sarcastic or not. You will revisit this dataset in later labs so it is good to be acquainted with it now."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Twhyfjg0xTkg"
      },
      "source": [
        "## Download and inspect the dataset\n",
        "\n",
        "First, you will fetch the dataset and preview some of its elements."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "33W129a7xgoJ"
      },
      "outputs": [],
      "source": [
        "# Download the dataset\n",
        "!wget https://storage.googleapis.com/tensorflow-1-public/course3/sarcasm.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zJHdzh9FyWa2"
      },
      "source": [
        "The dataset is saved as a [JSON](https://www.json.org/json-en.html) file and you can use Python's [`json`](https://docs.python.org/3/library/json.html) module to load it into your workspace. The cell below unpacks the JSON file into a list."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OkaBMeNDwMel"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "# Load the JSON file\n",
        "with open(\"./sarcasm.json\", 'r') as f:\n",
        "    datastore = json.load(f)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D2aSBvJVzRNV"
      },
      "source": [
        "You can inspect a few of the elements in the list. You will notice that each element consists of a dictionary with a URL link, the actual headline, and a label named `is_sarcastic`. Printed below are two elements with contrasting labels."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RiiFcWU2xnMJ"
      },
      "outputs": [],
      "source": [
        "# Non-sarcastic headline\n",
        "print(datastore[0])\n",
        "\n",
        "# Sarcastic headline\n",
        "print(datastore[20000])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dPuH0bBiz8LJ"
      },
      "source": [
        "With that, you can collect all urls, headlines, and labels for easier processing when using the tokenizer. For this lab, you will only need the headlines but we included the code to collect the URLs and labels as well."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9pxLUQJCxkNB"
      },
      "outputs": [],
      "source": [
        "# Initialize lists\n",
        "sentences = [] \n",
        "labels = []\n",
        "urls = []\n",
        "\n",
        "# Append elements in the dictionaries into each list\n",
        "for item in datastore:\n",
        "    sentences.append(item['headline'])\n",
        "    labels.append(item['is_sarcastic'])\n",
        "    urls.append(item['article_link'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lBHSXJ5V0qqK"
      },
      "source": [
        "## Preprocessing the headlines\n",
        "\n",
        "You can convert the `sentences` list above into padded sequences by using the same methods you've been using in the past exercises. The cell below generates the `word_index` dictionary and generates the list of padded sequences for each of the 26,709 headlines."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5OSTw3uJuvmY"
      },
      "outputs": [],
      "source": [
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
        "\n",
        "# Initialize the Tokenizer class\n",
        "tokenizer = Tokenizer(oov_token=\"<OOV>\")\n",
        "\n",
        "# Generate the word index dictionary\n",
        "tokenizer.fit_on_texts(sentences)\n",
        "\n",
        "# Print the length of the word index\n",
        "word_index = tokenizer.word_index\n",
        "print(f'number of words in word_index: {len(word_index)}')\n",
        "\n",
        "# Print the word index\n",
        "print(f'word_index: {word_index}')\n",
        "print()\n",
        "\n",
        "# Generate and pad the sequences\n",
        "sequences = tokenizer.texts_to_sequences(sentences)\n",
        "padded = pad_sequences(sequences, padding='post')\n",
        "\n",
        "# Print a sample headline\n",
        "index = 2\n",
        "print(f'sample headline: {sentences[index]}')\n",
        "print(f'padded sequence: {padded[index]}')\n",
        "print()\n",
        "\n",
        "# Print dimensions of padded sequences\n",
        "print(f'shape of padded sequences: {padded.shape}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4wyLF5T036W8"
      },
      "source": [
        "This concludes the short demo on using text data preprocessing APIs on a relatively large dataset. Next week, you will start building models that can be trained on these output sequences. See you there!"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "C3_W1_Lab_3_sarcasm.ipynb",
      "private_outputs": true,
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.4"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}