Skip to content

simddev/aggregator

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Gator

Gator is a CLI RSS feed aggregator built with TypeScript, Node.js, PostgreSQL, and Drizzle ORM.

It lets you:

  • register/login users
  • add RSS feeds
  • follow/unfollow feeds
  • run an aggregator loop to fetch posts
  • browse recent posts from feeds you follow

Requirements

Before running Gator, make sure you have:

  • Node.js (Node 18+ recommended, Node 20/22 works great)
  • npm
  • PostgreSQL running locally
  • psql (Postgres CLI client)

Installation

Clone the repo and install dependencies:

git clone https://github.com/YOUR-USERNAME/gator.git
cd gator
npm install

PostgreSQL Setup

  1. Start PostgreSQL.
  2. Create a database named gator:
CREATE DATABASE gator;
  1. Make sure you can connect with psql.

Example connection string (works if you use the default postgres role with no password locally):

psql "postgres://postgres:@localhost:5432/gator"

If your Postgres user/password is different, adjust the connection string accordingly.

Arch Linux note

If you're on Arch Linux, install and start PostgreSQL first:

sudo pacman -S postgresql
sudo -iu postgres initdb --locale=C.UTF-8 -D /var/lib/postgres/data
sudo systemctl enable --now postgresql

Then enter psql as the postgres user:

sudo -u postgres psql

Config File

Create this file in your home directory:

~/.gatorconfig.json

Example:

{
  "db_url": "postgres://postgres:@localhost:5432/gator?sslmode=disable"
}

Notes:

  • db_url is required
  • current_user_name will be added automatically by the CLI after login/register

Database Migrations

Generate and run migrations:

npx drizzle-kit generate
npx drizzle-kit migrate

If you added scripts in package.json, you can also use:

npm run generate
npm run migrate

Running the CLI

Use:

npm run start -- <command> [args...]

Example:

npm run start -- register simon

Commands

User Commands

Register a user

npm run start -- register <username>

Creates a new user and sets them as the current user.

Login

npm run start -- login <username>

Sets the current user in the config file (user must already exist).

List users

npm run start -- users

Shows all users and marks the current user.

Reset database (development helper)

npm run start -- reset

Deletes all users (and cascades into feeds, follows, and posts).


Feed Commands

Add a feed

npm run start -- addfeed "<feed name>" "<feed url>"

Creates a feed and automatically follows it for the current user.

Example:

npm run start -- addfeed "Boot.dev Blog" "https://blog.boot.dev/index.xml"

List feeds

npm run start -- feeds

Shows all feeds in the database, including the user who added them.

Follow a feed

npm run start -- follow "<feed url>"

Follows an existing feed by URL.

Unfollow a feed

npm run start -- unfollow "<feed url>"

Unfollows a feed for the current user.

Show followed feeds

npm run start -- following

Lists all feeds followed by the current user.


Aggregation Commands

Run the aggregator

npm run start -- agg <duration>

Runs a continuous loop that fetches posts from feeds and stores them in the database.

Duration examples:

  • 500ms
  • 5s
  • 1m
  • 1h

Example:

npm run start -- agg 30s

Press Ctrl+C to stop the aggregator gracefully.


Post Commands

Browse posts

npm run start -- browse

Shows the latest posts from feeds the current user follows.

Default limit is 2 posts.

Browse with a custom limit

npm run start -- browse 10

Shows the latest 10 posts.


Example Workflow

npm run start -- register simon
npm run start -- addfeed "Boot.dev Blog" "https://blog.boot.dev/index.xml"
npm run start -- addfeed "Hacker News" "https://news.ycombinator.com/rss"
npm run start -- agg 10s
# let it run for a bit, then press Ctrl+C
npm run start -- browse 10

Project Stack

  • TypeScript
  • Node.js
  • PostgreSQL
  • Drizzle ORM
  • drizzle-kit
  • fast-xml-parser

Notes

  • Feed URLs are unique in the database.
  • Post URLs are unique, so duplicate posts are ignored safely.
  • Deleting users cascades and removes related feeds, follows, and posts.
  • Run the aggregator with a reasonable interval to avoid hammering third-party servers.

License

This project was built as part of the Boot.dev guided project curriculum.

About

A CLI RSS aggregator written in TypeScript.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors