podverse-api

Data API, database migration scripts, and backend services for the Podverse ecosystem

Getting started

Local Development and Deployment

This repo contains steps for running podverse-api locally for development.

For stage/prod deployment instructions, please refer to the podverse-ops docs.

Prereqs

For podverse-api to work you will need a local Postgres database running.

You can setup your own database, or go to the podverse-ops repo, add the podverse-db-local.env file as explained in the docs, then run this command:

docker-compose -f docker-compose.local.yml up -d podverse_db

Setup environment variables

For local development, environment variables are provided by a local .env file. Duplicate the .env.example file, rename it to .env, and update all of the environment variables to match what is needed for your environment.

Install node_modules

npm install

Start dev server

npm run dev

Sample database data

If you want to add entities to the database quickly for development purposes only, run the following:

npm run dev:seeds:sampleEntities

Add podcast categories to the database

npm run dev:seeds:categories

Add feed urls to the database

To add podcasts to the database, you first need to add feed urls to the database, and then run the podcast parser with those feed urls.

You can pass multiple feed urls as a comma-delimited string parameter to the npm run dev:scripts:addFeedUrls command.

A list of sample podcast feed urls can be found in podverse-api/docs/sampleFeedUrls.txt.

npm run dev:scripts:addFeedUrls <feed urls>

OR you can pass an array of feed urls in the /src/config/parser/addFeedUrlsFile.json file, then run the npm run dev:scripts:addFeedUrlsFromFile command.

npm run dev:scripts:addFeedUrlsFromFile

Parse feed urls to add podcasts and episodes to the database

Orphan feed urls do not have a podcast associated with them.

npm run scripts:parseOrphanFeedUrls

To parse all non-orphan and public feed urls, you can run:

npm run scripts:parsePublicFeedUrls

Use SQS to add feed urls to a queue, then parse them

This project uses AWS SQS for its remote queue.

npm run dev:scripts:addAllOrphanFeedUrlsToQueue

or:

npm run dev:scripts:addAllPublicFeedUrlsToQueue

After you have added feed urls to a queue, you can retrieve and then parse the feed urls by running:

npm run dev:scripts:parseFeedUrlsFromQueue

Request Google Analytics pageview data and save to database

Below are sample commands for requesting unique pageview data from Google Analytics, which is used throughout the site for sorting by popularity (not a great/accurate system for popularity sorting...).

npm run dev:scripts:queryUniquePageviews -- clips month
npm run dev:scripts:queryUniquePageviews -- episodes week
npm run dev:scripts:queryUniquePageviews -- podcasts allTime

See the podverse-ops repo for a sample cron configuration for querying the Google API on a timer.