ProjectProposal.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a155ff5b-51f9-4232-a49c-f1efb2614ffb",
   "metadata": {},
   "source": [
    "# 204k News Articles and Essays\n",
    "Topic modeling of the news based on its content and title.\n",
    "\n",
    "## OVERVIEW\n",
    "News come in different formats, different types and different categories. Here we attempt to use Topic modeling and Clustering to get answers on what each content containt based on its content and then we try to do it based only on its title. <br>\n",
    "The process would be: \n",
    "We load the data. Keep what we need from the data. Clean the text(ex:stopwords).<br> \n",
    "Build the bag of words for all documents. Build the bag of words for each document. <br>\n",
    "Vectorize the data. Run the LDA model. Run the model on all data and save the output to dataframe <br>\n",
    "Run the Clustering algorithm. Save the data to csv. Make the charts.\n",
    "\n",
    "## Data\n",
    "The data is acquired from: https://components.one/datasets/all-the-news-articles-dataset <br>\n",
    "The Raw data containts 12 features: id, title, author, date, content, year, month, publication, category, digital, section,\turl.<br>\n",
    "<br>\n",
    "The features we are using are only the 'title' and 'content'.<br>\n",
    "The data we are not interested in will be dropped/ignored.<br>\n",
    "<br>\n",
    "The 'title' is the headling/name/title of the news/Article/Essay.<br>\n",
    "The 'Content' is the body/content/Essay/Article/News itself.<br>\n",
    "\n",
    "\n",
    "\n",
    "## TOOLS  \n",
    "- Pandas\n",
    "- Numpy\n",
    "- Scikit-learn\n",
    "- Matplotlib\n",
    "- Seaborn\n",
    "- nltk\n",
    "- gensim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5aaeabf-d4d4-4dea-818c-6aea363e64f6",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}