Detecting_Sports_Content_with_Machine_Learning".ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyNFKiySsUVuaEeTKv5T8P/f",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/hussain0048/Projects-/blob/master/Detecting_Sports_Content_with_Machine_Learning%22.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Introduction**"
      ],
      "metadata": {
        "id": "qQ5HBtBY7FS4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "n the vast landscape of digital content, the ability to categorize and understand text is essential for efficient information retrieval and analysis. Whether it's filtering news articles, organizing research papers, or personalizing recommendations, text classification plays a pivotal role in various applications. In this blog, we delve into the fascinating realm of text classification, focusing on a specific task: detecting sports-related content amidst a sea of diverse documents.\n",
        "\n",
        "Through the lens of machine learning, we embark on a journey to unravel the intricacies of distinguishing sports-related documents from their non-sports counterparts. Armed with a small but illustrative dataset, we navigate the process of training a classifier to recognize the distinctive features of sports content, from the thrill of the game to the nuances of policy discussions surrounding sporting events.\n",
        "\n",
        "Join us as we explore the methodologies, challenges, and insights gleaned from this endeavor. From the basics of natural language processing to the practical application of classification algorithms, this blog offers a comprehensive overview of the techniques employed to unlock the world of text classification.\n",
        "\n",
        "Whether you're a seasoned data scientist, an aspiring machine learning enthusiast, or simply curious about the magic behind intelligent systems, this exploration into the realm of sports text classification promises to inform, inspire, and ignite your curiosity. So, let's embark on this adventure together and discover the hidden patterns that lie within the words we read."
      ],
      "metadata": {
        "id": "zIYaqFu_7MZx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Table of Content**\n",
        "\n",
        "1.   Import library\n",
        "2.   Train and Test Data\n",
        "3.   Data Preprocessing\n",
        "4.   Train the classifier\n",
        "5.   Model Evaluation\n",
        "6.   Model performance Improment\n",
        "7.   Model Evaluation\n",
        "\n"
      ],
      "metadata": {
        "id": "Sv65-9ro7Pec"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Import library**"
      ],
      "metadata": {
        "id": "tufVAEBu7iYQ"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "VY3C6vKJ6vmA"
      },
      "outputs": [],
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.svm import LinearSVC"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Train and test data**"
      ],
      "metadata": {
        "id": "c1WQEG0Y7xFI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Train and test data. Both the full documents and their labels (\"Sports\" vs \"Non Sports\")\n",
        "train_data = ['Football: a great sport', 'The referee has been very bad this season', 'Our team scored 5 goals', 'I love tennis',\n",
        "              'Politics is in decline in the UK', 'Brexit means Brexit', 'The parlament wants to create new legislation',\n",
        "              'I so want to travel the world']\n",
        "train_labels = [\"Sports\",\"Sports\",\"Sports\",\"Sports\", \"Non Sports\", \"Non Sports\", \"Non Sports\", \"Non Sports\"]\n",
        "\n",
        "test_data = ['Swimming is a great sport',\n",
        "             'A lot of policy changes will happen after Brexit',\n",
        "             'The table tennis team will travel to the UK soon for the European Championship']\n",
        "test_labels = [\"Sports\",\"Non Sports\",\"Sports\"]"
      ],
      "metadata": {
        "id": "HehtR6kq75gA"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#**📚Data Preprocessing**"
      ],
      "metadata": {
        "id": "5aDl4YH18Ixw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Representation of the data using TF-IDF\n",
        "vectorizer = TfidfVectorizer()\n",
        "vectorised_train_data = vectorizer.fit_transform(train_data)\n",
        "vectorised_test_data = vectorizer.transform(test_data)"
      ],
      "metadata": {
        "id": "GMGzneMD8J5w"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Train the classifie**"
      ],
      "metadata": {
        "id": "VEgCfCMf8X3I"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Train the classifier given the training data\n",
        "classifier = LinearSVC()\n",
        "classifier.fit(vectorised_train_data, train_labels)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        },
        "id": "nuko3o2N8e3R",
        "outputId": "2fa9930d-baae-4613-f593-b31fed9f777f"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearSVC()"
            ],
            "text/html": [
              "<style>#sk-container-id-3 {color: black;background-color: white;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearSVC()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearSVC</label><div class=\"sk-toggleable__content\"><pre>LinearSVC()</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#**📚Model Evaluation**"
      ],
      "metadata": {
        "id": "3RFtUMEh8t-3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Train the classifier given the training data\n",
        "classifier = LinearSVC()\n",
        "classifier.fit(vectorised_train_data, train_labels)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        },
        "id": "8rvRhRuu84q4",
        "outputId": "4ea8cf52-3af2-448e-b051-7f4cc946e3c8"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearSVC()"
            ],
            "text/html": [
              "<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearSVC()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearSVC</label><div class=\"sk-toggleable__content\"><pre>LinearSVC()</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Predict the labels for the test documents (not used for training)\n",
        "print(classifier.predict(vectorised_test_data))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ndwRlgXP8-Y3",
        "outputId": "d3fcba7a-00ad-4d3a-8d4b-640639294fec"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['Sports' 'Non Sports' 'Non Sports']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "However, the third case is wrongly classified. Why do you think that might be?\n",
        "\n",
        "Matching problems (e.g., \"car\" is different than \"Cars\")\n",
        "\n",
        "Cases never seen before (e.g., the classifier has never seen the word \"table\")\n",
        "\n",
        "\"Spurious\" correlations and bias (\"car\" appears only in the positive category)\n",
        "\n",
        "Lets look into how we are representing our documents"
      ],
      "metadata": {
        "id": "xZhVsYfNLGzu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "# Vectorize the test data using the same TF-IDF vectorizer used for training\n",
        "vectorized_test_data = vectorizer.transform(test_data)\n",
        "\n",
        "# Predict labels for the test data\n",
        "predicted_labels = classifier.predict(vectorized_test_data)\n",
        "\n",
        "# Calculate accuracy\n",
        "accuracy = accuracy_score(test_labels, predicted_labels)\n",
        "print(\"Accuracy:\", accuracy)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VmPNc0dA9PUX",
        "outputId": "d982632b-9717-47ba-92ed-95dc11552329"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 0.6666666666666666\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Model Performance improment**"
      ],
      "metadata": {
        "id": "UV92jApAL-Gl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "# Function to show the feature weights of a document (to be explained later)\n",
        "def feature_values(doc, representer):\n",
        "    doc_representation = representer.transform([doc])\n",
        "    features = representer.get_feature_names_out()\n",
        "    return [(features[index], doc_representation[0, index]) for index in doc_representation.nonzero()[1]]\n",
        "\n",
        "pprint([feature_values(doc, vectorizer) for doc in test_data])\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hWwPbRGmLckG",
        "outputId": "79923fbc-fa64-4385-b078-b28ea0672852"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[('sport', 0.5773502691896258),\n",
            "  ('is', 0.5773502691896258),\n",
            "  ('great', 0.5773502691896258)],\n",
            " [('brexit', 1.0)],\n",
            " [('uk', 0.3466689227843291),\n",
            "  ('travel', 0.3466689227843291),\n",
            "  ('to', 0.29053561299308733),\n",
            "  ('the', 0.6594480187891556),\n",
            "  ('tennis', 0.3466689227843291),\n",
            "  ('team', 0.3466689227843291)]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Lets try again, with stop-word removal this time\n",
        "\n"
      ],
      "metadata": {
        "id": "I7s-V_-GMKI-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5Pv-AfXgMU6H",
        "outputId": "ea8fc9f9-3bd1-4642-bdc2-ea330c81eaef"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.corpus import stopwords\n",
        "\n",
        "# Load the list of (english) stop-words from nltk\n",
        "stop_words = stopwords.words(\"english\")\n",
        "\n",
        "# Represent, train, predict\n",
        "vectorizer = TfidfVectorizer(stop_words=stop_words)\n",
        "vectorised_train_data = vectorizer.fit_transform(train_data)\n",
        "vectorised_test_data = vectorizer.transform(test_data)\n",
        "classifier = LinearSVC()\n",
        "classifier.fit(vectorised_train_data, train_labels)\n",
        "binary_predictions=classifier.predict(vectorised_test_data)\n",
        "\n",
        "print(classifier.predict(vectorised_test_data))\n",
        "# Expected: [Sports, Non Sports, Sports]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a9qVbLbiMOOy",
        "outputId": "cb9c9f47-465d-4371-82b8-eb6f8d67e07f"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['Sports' 'Non Sports' 'Sports']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Model Evaluation**"
      ],
      "metadata": {
        "id": "HpeHSeIQMdBc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.metrics import f1_score, precision_score, recall_score\n",
        "\n",
        "# Binary problem\n",
        "binary_labels = [1, 0, 1]\n",
        "binary_predictions = [1, 0, 0]\n",
        "\n",
        "# Quality values (with respect to class 1 by default)\n",
        "# Show our quality\n",
        "precision = precision_score(binary_labels, binary_predictions)\n",
        "recall = recall_score(binary_labels, binary_predictions)\n",
        "f1 = f1_score(binary_labels, binary_predictions)\n",
        "print(\"Micro-average quality numbers\")\n",
        "print(\"Precision: {:.4f}, Recall: {:.4f}, F1-measure: {:.4f}\".format(precision,\n",
        "                                                                     recall,\n",
        "                                                                     f1))\n",
        "\n",
        "binary_labels = [\"A\", \"B\", \"A\"]\n",
        "binary_predictions = [\"A\", \"B\", \"B\"]\n",
        "precision = precision_score(binary_labels, binary_predictions, pos_label=\"A\")\n",
        "recall = recall_score(binary_labels, binary_predictions, pos_label=\"A\")\n",
        "f1 = f1_score(binary_labels, binary_predictions, pos_label=\"A\")\n",
        "print(\"Precision: {:.4f}, Recall: {:.4f}, F1-measure: {:.4f}\".format(precision,\n",
        "                                                                     recall,\n",
        "                                                                     f1))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m9tc6gHxMhp2",
        "outputId": "63c4cc05-c4ff-4123-8bdf-2b7e56bc2d3b"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Micro-average quality numbers\n",
            "Precision: 1.0000, Recall: 0.5000, F1-measure: 0.6667\n",
            "Precision: 1.0000, Recall: 0.5000, F1-measure: 0.6667\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "# Vectorize the test data using the same TF-IDF vectorizer used for training\n",
        "vectorized_test_data = vectorizer.transform(test_data)\n",
        "\n",
        "# Predict labels for the test data\n",
        "predicted_labels = classifier.predict(vectorized_test_data)\n",
        "\n",
        "# Calculate accuracy\n",
        "accuracy = accuracy_score(test_labels, predicted_labels)\n",
        "print(\"Accuracy:\", accuracy)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-4AsR4UdMw9J",
        "outputId": "9fb02429-85b2-4cb4-e027-915db7196cdd"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 1.0\n"
          ]
        }
      ]
    }
  ]
}