This is a dataset of the all-time top 1,000 posts, from the top 2,500 subreddits by subscribers, pulled from reddit between August 15–20, 2013.
Switch branches/tags
Nothing to show
Clone or download
umbrae Strip author from CSV files.
A few folks have written in to have their name removed from this
dataset for various reasons (privacy, other sensitivity reasons,
etc). In hindsight author doesn't really provide any value in this
dataset, so I've decided to remove it from all rows.

so I've decided to remove it from all
Latest commit f6585ba Jan 19, 2017

README.md

Reddit Top 2.5 Million

What is this?

This is a dataset of top posts from reddit. It contains the top 1,000 all-time posts from the top 2,500 subreddits, so 2.5 million posts in total. The top subreddits were determined by subscriber count and are located in the manifest file within.

Ths data was pulled between August 15-20 of August 2013.

Each file is a CSV with the related subreddit as its filename. Each CSV file contains a header line.

A good example of the structure of this data is here.

Why?

I plan to use this for some data analysis I'm putting together for /r/serendipity. You're free to do whatever you like with it.

What use is this?

Who knows! Here's one simple example: This is a breakdown of the top voted domains on Serendipity. (Yes, as a pie chart.)

For my use, I'm considering finding important terms for a subreddit using TF-IDF.

Have other ideas? I'd love to hear what you do with this. Let me know.

Gotchas?

Yes, one: This only includes subreddits that are not marked NSFW. This is related to my particular use case, in which NSFW subreddits aren't useful for me. They'd be relatively easy to scrape if you're interested, you'll just need to get the list of NSFW subreddits from http://www.reddit.com/reddits.