This subreddit downloader downloads all posts and comments in a subreddit
For a tutorial to use this program please follow this medium article - https://medium.com/@guneet.android/download-subreddit-data-for-nlp-projects-c8fa85c0df87
This script makes use of the pushshift API ( https://github.com/pushshift/api ) to download the entire dump of posts and comments in a specific subreddit
Simply replace the subreddit name in the fourth line of code and run the script
It will download two files one for all the posts and one for all the comments
Patched UTF-8 encoding*
Please note pushshift has rate limitations -
repeated 429 error codes will result in blocklisting of IP
If your data format is in single quotes
Please use this JSON fixer by @mbrzusto