Feedsearch Gateway

Serverless AWS API using Zappa for the Feedsearch Crawler Python package.

Live at https://feedsearch.dev

Setup

Install Pipenv.

Install and configure the AWS CLI.

The following environment variables are required:

USER_AGENT="Mozilla/5.0 (compatible; Feedsearch-Crawler; +https://feedsearch.dev)"
FLASK_S3_BUCKET_NAME="feedsearch-bucket"
DYNAMODB_TABLE="feedsearch-table"
SERVER_NAME="feedsearch.dev" #Exclude this value in dev for localhost

• USER_AGENT : HTTP User-Agent string.
• FLASK_S3_BUCKET_NAME : The name of the S3 bucket that serves static files.
• DYNAMODB_TABLE : The name of the DynamoDB table for storing found feeds.
• SERVER_NAME : The host url of the site.

For local development, add the environment variables to a .env file.

For production or testing in AWS, add them to the Environment Variables in Lambda, either directly or in in the zappa_settings.json file.

Update the other settings in the zappa_settings.json file as required.

Run the create_table.py script.

python3 scripts/create_table.py

Development

Run the dev server:

export FLASK_APP=gateway/application.py
export FLASK_DEBUG=true

flask run

Deployment

Upload static assets to S3:

export FLASK_APP=gateway/application.py

flask upload

Deploy with:

zappa update production