Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
src
 
 
 
 
 
 
 
 
 
 

reddit-scraper

What is this?

This is a Reddit scraper that monitors a subreddit and stores new threads in Amazon DynamoDB.

Install

npm i

Create .env at root with the following variables, specific to your Reddit app: https://www.reddit.com/prefs/apps/

AWS_PROFILE=***
CLIENT_ID=***
CLIENT_SECRET=***
POST_FLAIR_TEXT=***
POST_MINIMUM_UPVOTES=1
REDDIT_USER=***
REDDIT_PASS=***
SUBREDDIT=***

AWS_PROFILE refers to the corresponding section in your ~/.aws/credentials file. For example:

[default] ; default profile
aws_access_key_id = <DEFAULT_ACCESS_KEY_ID>
aws_secret_access_key = <DEFAULT_SECRET_ACCESS_KEY>

[personal-account] ; personal account profile
aws_access_key_id = <PERSONAL_ACCESS_KEY_ID>
aws_secret_access_key = <PERSONAL_SECRET_ACCESS_KEY>

Usage

npm start

About

A customisable Reddit scraper, built on Node. It runs on AWS Lambda and stores data in DynamoDB.

Topics

Resources

License

Releases

No releases published

Packages

No packages published