Reddit is a collection of online discussion boards known as "subreddits", which cover a variety of topics. The goal of this project is to classify which subreddit a given post came from. As there are over 1.5 million subreddits on reddit, we will be classifying posts from two subreddits, /r/personalfinance and /r/relationship_advice, to make the project manageable. The choice of these two subreddits is motivated by their text-heavy posts. We will be creating and comparing two models: a logistic regression and a multinomial naive Bayes classifier. Our results may be useful for any reddit user who is unsure which subreddit is the most appropriate to submit his new post, so that he can attract the most comments.
- 1. Data Collection
- 2. Data Cleaning and EDA
- 3. Pre-processing and Modeling
- 4. Evaluation and Conceptual Understanding
- 5. Conclusion and Recommendations
| Feature | Type | Description |
|---|---|---|
| name | object | Unique id of a post. |
| title | object | Title of a post. |
| text | object | Body text of a post. |
| subreddit | object | Which subreddit the post came from. |
- Our multinomial naive Bayes classifier performed well with a test accuracy score of 98.73%. This is within expectations because the topics of our two chosen subreddits differ significantly.
- Subreddit Classifier - A proof of concept web application was developed to demonstrate potential use case. Users can submit their draft post in the application to determine which of the two subreddits should the post be submitted to.
- Scope for future improvements:
- Optimize stop words and explore strategies for stemming and lemmatization
- Try ensemble models, such as random forest classifier
- Ability for model to classify more than two subreddits