pos_tagging.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "POS_tagging.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyOU0a9XrlHCOkfQcD/F89cs",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/Melvinchen0404/c4b91cbaa33e4ac35a26a20f6165acc1/pos_tagging.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##NLP Technique 2: POS Tagging \n",
        "**Part-of-speech (POS) tagging** (or **tagging**) refers to the process of classifying words into their **parts of speech** and labelling them accordingly \\\n",
        "**Parts of speech** are also known as **word classes** or **lexical categories** \\\n",
        "Each word will have a **part-of-speech (POS) tag** to describe its syntactic category \\\n",
        "The **tokenized words** could be tagged as nouns, verbs, adjectives, adverbs, prepositions, determiners, conjunctions, pronouns, etc \\\n",
        "Here is an online **POS tagging** resource: https://parts-of-speech.info/  \n",
        " \n",
        "\\\n",
        "**STEP 1:** Import the `pos_tag` function from the `nltk` package \\\n",
        "**STEP 2:** Download the `averaged_perceptron_tagger` for tagging the **tokenized words** with their **parts of speech**\n"
      ],
      "metadata": {
        "id": "bcFycCC6sjxM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('punkt')\n",
        "\n",
        "class color:\n",
        "   BOLD = '\\033[1m'\n",
        "   END = '\\033[0m'\n",
        "\n",
        "text = \"Stately, plump Buck Mulligan came from the stairhead, bearing a bowl of lather on which a mirror and a razor lay crossed. A yellow dressinggown, ungirdled, was sustained gently behind him on the mild morning air.\"\n",
        "tokens1 = nltk.word_tokenize(text)\n",
        "tokens2 = [word for word in tokens1 if word.isalnum()]\n",
        "from nltk import pos_tag\n",
        "nltk.download('averaged_perceptron_tagger')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1E5U4zuNvOOB",
        "outputId": "34bd157b-52c8-4cf7-ec92-753f0b2171f8"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n",
            "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
            "[nltk_data]     /root/nltk_data...\n",
            "[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**STEP 3:** Apply the `pos_tag` function to the **tokenized words** to get the **POS tags** for each **tokenized word**  "
      ],
      "metadata": {
        "id": "ozhZLmO5vRXw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "nltk.pos_tag(tokens2)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ksQ6s0nYtA-3",
        "outputId": "2cef763b-6fb6-4089-f099-74a5c1fa9fdb"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('Stately', 'RB'),\n",
              " ('plump', 'JJ'),\n",
              " ('Buck', 'NNP'),\n",
              " ('Mulligan', 'NNP'),\n",
              " ('came', 'VBD'),\n",
              " ('from', 'IN'),\n",
              " ('the', 'DT'),\n",
              " ('stairhead', 'NN'),\n",
              " ('bearing', 'VBG'),\n",
              " ('a', 'DT'),\n",
              " ('bowl', 'NN'),\n",
              " ('of', 'IN'),\n",
              " ('lather', 'NN'),\n",
              " ('on', 'IN'),\n",
              " ('which', 'WDT'),\n",
              " ('a', 'DT'),\n",
              " ('mirror', 'NN'),\n",
              " ('and', 'CC'),\n",
              " ('a', 'DT'),\n",
              " ('razor', 'NN'),\n",
              " ('lay', 'NN'),\n",
              " ('crossed', 'VBD'),\n",
              " ('A', 'NNP'),\n",
              " ('yellow', 'JJ'),\n",
              " ('dressinggown', 'NN'),\n",
              " ('ungirdled', 'VBD'),\n",
              " ('was', 'VBD'),\n",
              " ('sustained', 'VBN'),\n",
              " ('gently', 'RB'),\n",
              " ('behind', 'IN'),\n",
              " ('him', 'PRP'),\n",
              " ('on', 'IN'),\n",
              " ('the', 'DT'),\n",
              " ('mild', 'JJ'),\n",
              " ('morning', 'NN'),\n",
              " ('air', 'NN')]"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**STEP 4:** The `nltk` package provides documentation for each **POS tag** \\\n",
        "This allows queries to be run for each **POS tag** \\"
      ],
      "metadata": {
        "id": "lwgZ_Dyrv_KB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown **POS tag query**\n",
        "#@markdown ---\n",
        "POS_tag = \"Key in abbreviation for POS tag to be queried here\"#@param {type:\"string\"}\n"
      ],
      "metadata": {
        "cellView": "form",
        "id": "1pvVIkUFxRfS"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "nltk.download('tagsets')\n",
        "nltk.help.upenn_tagset(POS_tag)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "klqDZ_dWvxbY",
        "outputId": "b1755a5c-fc19-4d66-ad48-0dd090f0ab06"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "No matching tags found.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package tagsets to /root/nltk_data...\n",
            "[nltk_data]   Unzipping help/tagsets.zip.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**STEP 5:** If you wish, you can tag only a restricted number of **tokenized words**. Suppose you wish to tag only the first *n* **tokenized words**. You could run the line of code `tagged[0:n]`"
      ],
      "metadata": {
        "id": "lEq1jRpRzVD3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tagged = nltk.pos_tag(tokens2)\n",
        "tagged[0:8]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ClM6rQyh0JPH",
        "outputId": "c02f8961-b863-4e21-8990-0d9037c1e377"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('Stately', 'RB'),\n",
              " ('plump', 'JJ'),\n",
              " ('Buck', 'NNP'),\n",
              " ('Mulligan', 'NNP'),\n",
              " ('came', 'VBD'),\n",
              " ('from', 'IN'),\n",
              " ('the', 'DT'),\n",
              " ('stairhead', 'NN')]"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    }
  ]
}