Topic_Modeling_Agglomerative_Clusters.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.9"
    },
    "colab": {
      "name": "Topic Modeling - Agglomerative Clusters.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/KalikaKay/Author-Classification-Project/blob/master/Topic_Modeling_Agglomerative_Clusters.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "okLgwgBMgYv9"
      },
      "source": [
        "# Topic Modeling\r\n",
        "\r\n",
        "Using LSA, LDA, and NNMF; print out top ten words (with their highest loading) for each topic modeling. \r\n",
        "\r\n",
        "Analyze and compare among three methods."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FucHzgF_a8TI"
      },
      "source": [
        "# Data Cleaning\r\n",
        "\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ed65OHqNVkXh"
      },
      "source": [
        "#Topic Models\r\n",
        "from sklearn.decomposition import TruncatedSVD\r\n",
        "from sklearn.decomposition import LatentDirichletAllocation, NMF\r\n",
        "\r\n",
        "#Data Engineering\r\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer\r\n",
        "import numpy as np\r\n",
        "import pandas as pd\r\n",
        "\r\n",
        "\r\n",
        "# file location\r\n",
        "PATH = '/content/drive/MyDrive/Author Classification/Books/Books.parquet'\r\n",
        "books = pd.read_parquet(PATH)"
      ],
      "execution_count": 106,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 196
        },
        "id": "WTmEXurOtBK9",
        "outputId": "0cbd6124-24b8-415c-d52e-57722993c5f2"
      },
      "source": [
        "books.head()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentence</th>\n",
              "      <th>author</th>\n",
              "      <th>tokenized</th>\n",
              "      <th>lemmatized</th>\n",
              "      <th>kbow</th>\n",
              "      <th>abow</th>\n",
              "      <th>dbow</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Produced David Price</td>\n",
              "      <td>Anne Bronte</td>\n",
              "      <td>[produced, david, price]</td>\n",
              "      <td>[produced, david, price]</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Agnes Grey NOVEL</td>\n",
              "      <td>Anne Bronte</td>\n",
              "      <td>[agnes, grey, novel]</td>\n",
              "      <td>[agnes, grey, novel]</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>Illustration Birthplace Charlotte Emily and An...</td>\n",
              "      <td>Anne Bronte</td>\n",
              "      <td>[illustration, birthplace, charlotte, emily, a...</td>\n",
              "      <td>[illustration, birthplace, charlotte, emily, a...</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>All true histories contain instruction though ...</td>\n",
              "      <td>Anne Bronte</td>\n",
              "      <td>[all, true, histories, contain, instruction, t...</td>\n",
              "      <td>[all, true, history, contain, instruction, tho...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>father was clergyman the north England who was...</td>\n",
              "      <td>Anne Bronte</td>\n",
              "      <td>[father, was, clergyman, the, north, england, ...</td>\n",
              "      <td>[father, wa, clergyman, the, north, england, w...</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                             sentence       author  ... abow dbow\n",
              "0                               Produced David Price   Anne Bronte  ...    1    0\n",
              "1                                   Agnes Grey NOVEL   Anne Bronte  ...    1    0\n",
              "5   Illustration Birthplace Charlotte Emily and An...  Anne Bronte  ...    1    0\n",
              "9   All true histories contain instruction though ...  Anne Bronte  ...    0    0\n",
              "10  father was clergyman the north England who was...  Anne Bronte  ...    0    0\n",
              "\n",
              "[5 rows x 7 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Wi1NABMv8uHf"
      },
      "source": [
        "*...emphasis on unsupervised learning*\r\n",
        "\r\n",
        "Perhaps I didn't pay enough attention to the cluster results the agglomerative model provided in my first unsupervised learning notebook. \r\n",
        "\r\n",
        "I'm curious about the topics for the cluster models. What are the topics in the agglomerative clusters? \r\n",
        "\r\n",
        "I will be performing my topic modeling on the agglomerative clusters. \r\n",
        "\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aqj6Be91uxCr"
      },
      "source": [
        "#Number of Keywords and Topics for each model.\r\n",
        "num_keywords = 10\r\n",
        "num_topics = 10"
      ],
      "execution_count": 80,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pkGdl5ksu3kl"
      },
      "source": [
        "# Latent Semantic Analysis (LSA)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8Y1wrpOyCWdW"
      },
      "source": [
        "#Bag of Words Vector\r\n",
        "vectorizer_one = TfidfVectorizer()\r\n",
        "vectorized_one = vectorizer_one.fit_transform(books[books.abow ==1]['lemmatized'].astype(str))\r\n",
        "vectorizer_zero = TfidfVectorizer()\r\n",
        "vectorized_zero= vectorizer_zero.fit_transform(books[books.abow ==0]['lemmatized'].astype(str))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_kda5pBvuzOZ"
      },
      "source": [
        "model = TruncatedSVD(n_components=num_topics)\r\n",
        "model.fit_transform(vectorized_one.toarray())\r\n",
        "\r\n",
        "results = [[(vectorizer_one.get_feature_names()[i], topic[i])\r\n",
        "           for i in topic.argsort()[:-num_keywords - 1:-1]]\r\n",
        "           for topic in model.components_]\r\n",
        "one = [[x[0] for x in i] for i in results]"
      ],
      "execution_count": 84,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zIleOevy0itU",
        "outputId": "cbf95013-eb95-4473-b884-f8eeefd58675"
      },
      "source": [
        "#This format for readability. \r\n",
        "for topic in one:\r\n",
        "  print(*topic)"
      ],
      "execution_count": 98,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "the you and she her that wa said his not\n",
            "you said are what have don know your will can\n",
            "she her said jake yes hand him herself maud nap\n",
            "yes said jake the nap him bunny right lucas well\n",
            "said jake the nap his don bunny lucas maud again\n",
            "what said asked matter jake mean wa the nap cried\n",
            "not that but she have could would had all wa\n",
            "his him that wa and jake not but eye with\n",
            "him she his wa you did asked why the raskolnikov\n",
            "wa that jake you are there the silent maud voice\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pYBn35FflX_q"
      },
      "source": [
        "#redfine the model so it's not tainted by the first one. \r\n",
        "model = TruncatedSVD(n_components=num_topics)\r\n",
        "model.fit_transform(vectorized_zero.toarray())\r\n",
        "\r\n",
        "results = [[(vectorizer_zero.get_feature_names()[i], topic[i])\r\n",
        "           for i in topic.argsort()[:-num_keywords - 1:-1]]\r\n",
        "           for topic in model.components_]\r\n",
        "zero = [[x[0] for x in i] for i in results]"
      ],
      "execution_count": 101,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ei0Uo_Uwn_xg",
        "outputId": "ee42d538-83e0-4f13-8590-11b71d3faa3b"
      },
      "source": [
        "#This format for readability. \r\n",
        "for topic in zero:\r\n",
        "  print(*topic)"
      ],
      "execution_count": 102,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "the and wa that her she had his you for\n",
            "you that she have not for your and her will\n",
            "her she had wa not could herself been would with\n",
            "and his wa that him had not all them they\n",
            "and her they their them will child little our it\n",
            "that they for their all one will have not are\n",
            "wa they but there not and did myself were could\n",
            "his that wa her man hand one she with eye\n",
            "she not but him for would will have very could\n",
            "she that had and him out then went come ivanovna\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VmLe-yDX-OGs"
      },
      "source": [
        "# Latent Dirichlet Allocation (LDA)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ApLvM72cA0B5"
      },
      "source": [
        "#Bag of Words Vector\r\n",
        "vectorizer_one = CountVectorizer()\r\n",
        "vectorized_one = vectorizer_one.fit_transform(books[books.abow ==1]['lemmatized'].astype(str))\r\n",
        "vectorizer_zero = CountVectorizer()\r\n",
        "vectorized_zero= vectorizer_zero.fit_transform(books[books.abow ==0]['lemmatized'].astype(str))"
      ],
      "execution_count": 99,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g0H8Bb_YA-Fc",
        "outputId": "cef614bc-5953-4ee5-99cb-f594501485c6"
      },
      "source": [
        "model = LatentDirichletAllocation(n_components=num_topics, learning_method='online')\r\n",
        "model.fit_transform(vectorized_one.toarray())\r\n",
        "\r\n",
        "results = [[(vectorizer_one.get_feature_names()[i], topic[i])\r\n",
        "           for i in topic.argsort()[:-num_keywords - 1:-1]]\r\n",
        "           for topic in model.components_]\r\n",
        "one = [[x[0] for x in i] for i in results]\r\n",
        "\r\n",
        "#This format for readability. \r\n",
        "for topic in one:\r\n",
        "  print(*topic)"
      ],
      "execution_count": 100,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "jake you back sure what believe don know told yes\n",
            "mind wish father petrovitch hope arkady along raskolnikov ivanovna secret\n",
            "the and wa his had her with from that were\n",
            "you and she the that her not but said for\n",
            "felt life more and since fairfax boy jane rest glad\n",
            "silence spoke sonia ask between silent into moved stopped sent\n",
            "old passed the vasya snap street twenty hundred arm daughter\n",
            "the light miko anita over dead yesterday grantline straight gregg\n",
            "the and into hear through kleig earth stood which it\n",
            "face the side upon his and slowly fell svidriga lov\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZMldLOyhCzn4",
        "outputId": "d05676e2-7aa2-4202-e5b1-3f839c554f55"
      },
      "source": [
        "model = LatentDirichletAllocation(n_components=num_topics, learning_method='online')\r\n",
        "model.fit_transform(vectorized_zero.toarray())\r\n",
        "\r\n",
        "results = [[(vectorizer_zero.get_feature_names()[i], topic[i])\r\n",
        "           for i in topic.argsort()[:-num_keywords - 1:-1]]\r\n",
        "           for topic in model.components_]\r\n",
        "zero = [[x[0] for x in i] for i in results]\r\n",
        "\r\n",
        "#This format for readability. \r\n",
        "for topic in zero:\r\n",
        "  print(*topic)"
      ],
      "execution_count": 105,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "mastakovitch yulian member waggon promotion gas counter flown transcendental seth\n",
            "emma mr harriet miss elton knightley woodhouse churchill weston frank\n",
            "inventor solarium carnes sunburn admiral planet plane quartz operating sanity\n",
            "sophia macdonald sensibility janetta augustus laura scotland tho edinburgh laurina\n",
            "von beyer moonlight lunium cathode cadmium detected ultra spectrum medical\n",
            "ivanovitch ivanovna sonia semyon katerina svidriga lov rouble arkady amalia\n",
            "the and wa his that with had him for but\n",
            "eloisa tho marlowe elizabeth lesley chaise charlotte louisa disgraced 7th\n",
            "and the that you her she wa not for had\n",
            "pyotr petrovitch wud romanovna avdotya alexandrovna pulcheria heave swung yer\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CramJQP0FKFP"
      },
      "source": [
        "# Non-Negative Matrix Factorization"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kAsbQbbtFWBz"
      },
      "source": [
        "#Bag of Words Vector\r\n",
        "vectorizer_one = TfidfVectorizer()\r\n",
        "vectorized_one = vectorizer_one.fit_transform(books[books.abow ==1]['lemmatized'].astype(str))\r\n",
        "vectorizer_zero = TfidfVectorizer()\r\n",
        "vectorized_zero= vectorizer_zero.fit_transform(books[books.abow ==0]['lemmatized'].astype(str))"
      ],
      "execution_count": 107,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DlBGIKMGFYXh",
        "outputId": "597be9cb-8143-472a-eaaa-022a8981f605"
      },
      "source": [
        "model = NMF(n_components=num_topics)\r\n",
        "model.fit_transform(vectorized_one.toarray())\r\n",
        "\r\n",
        "results = [[(vectorizer_one.get_feature_names()[i], topic[i])\r\n",
        "           for i in topic.argsort()[:-num_keywords - 1:-1]]\r\n",
        "           for topic in model.components_]\r\n",
        "one = [[x[0] for x in i] for i in results]\r\n",
        "\r\n",
        "#This format for readability. \r\n",
        "for topic in one:\r\n",
        "  print(*topic)"
      ],
      "execution_count": 108,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "the and from with had room they door out into\n",
            "you are know don your why can will have how\n",
            "she her had herself hand eye with did face upon\n",
            "yes answered sir replied raskolnikov quite sure course now well\n",
            "said jake nap don she bunny maud lucas again anne\n",
            "what asked mean matter cried say want are did raskolnikov\n",
            "not have and but for that all would will very\n",
            "his and eye with hand face her head upon looked\n",
            "him looked with don raskolnikov let and bunny know she\n",
            "wa that there had silent but were voice maud when\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XyKV7_tbFYXj",
        "outputId": "fee3e22e-2316-494c-a590-e69eb135b23e"
      },
      "source": [
        "model = NMF(n_components=num_topics)\r\n",
        "model.fit_transform(vectorized_zero.toarray())\r\n",
        "\r\n",
        "results = [[(vectorizer_zero.get_feature_names()[i], topic[i])\r\n",
        "           for i in topic.argsort()[:-num_keywords - 1:-1]]\r\n",
        "           for topic in model.components_]\r\n",
        "zero = [[x[0] for x in i] for i in results]\r\n",
        "\r\n",
        "#This format for readability. \r\n",
        "for topic in zero:\r\n",
        "  print(*topic)"
      ],
      "execution_count": 109,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "the room door which with from and through window into\n",
            "you your are don know have will that come see\n",
            "her she and the with had from that eye mother\n",
            "very emma and mr harriet the wa weston miss elton\n",
            "his him himself and with the man would vasya ivanovitch\n",
            "that will have and not for ha all what one\n",
            "and wa then went said out came when down were\n",
            "had wa that been not but all could the there\n",
            "she her herself would not him knew that did could\n",
            "they them their and were the are themselves with for\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9QpqIv6kiyis"
      },
      "source": [
        "# Conclusion\r\n",
        "\r\n",
        "\r\n",
        "Three topic models were run against the books dataset. In the interest maintaining the unsupervised learning theme, the books were filtered by the sentences' agglomerative clusters.\r\n",
        "\r\n",
        "Out of the three, the LDA model seems to have the most information regarding the top ten topics for each cluster. LSA seems to come in at a close second. \r\n",
        "I have no idea what NMF is talking about. \r\n",
        "\r\n",
        "There are definitely some other approaches that I could have taken with this topic modeling. I could have filtered out by author or book and retrieved a topic for specific titles or authors. \r\n",
        "\r\n",
        "I took the cluster route because I wanted to see if there was a stark difference in topics between the clusters.\r\n",
        "\r\n",
        "---\r\n",
        "*a Thinkful Project by Kalika Kay Curry*\r\n"
      ]
    }
  ]
}