DESIGN.md

ReadErl Design

My inspiration for ReadErl was Google Reader. When Google Reader shut down, I added an RSS aggregator to Lisplog. That works quite well, for me, but it is configured with a text file containing a lisp list of RSS feed URLs. I want a tool for everybody to use, like Google Reader. I suppose I should survey the field; there are other tools out there. But I'm going to just build something original, and see if anybody wants to use it.

ReadErl.com is currently running on a $5/month Digital Ocean VM, but I want it to be able to scale big. This has a some implications:

1. The database hides behind an API, with synchronous and asynchronous entry points. I will use mnesia initially, likely via the Amnesia Elixir wrapper, but I want to be able to switch to CouchDB or Riak, if necessary for scale.

2. Pluggable feeds. I'll initially do RSS and Twitter. I'd like to add Medium, but right now the only way to do that is to screen scrape the HTML. Hopefully, they'll fix that soon, by expanding their API to allow more than just posting, or shipping recommended stories to IFTTT.

I was originally going to write ReadErl in Erlang, with the Chicago Boss web framework. Then I discovered Elixir and Phoenix. Popeye, meet Jessica Rabbit.

Reminder links:

• GitHub Flavored Markdown
• Daring Fireball's Markdown Docs

Initial exploratory tasks:

1. Write some code to use feeder_ex to parse RSS feeds. [** done: lib/readerl/fetcher.ex **]

2. Switch the dummy ReadErl.com web page from static HTML to Phoenix. [** done **]

3. Design the feed plugin API.

4. Build the RSS plugin.

5. Work on the web interface allowing creation of user accounts, and addition of feeds. Probably use Sonny Scroggin's oauth2 for this.

6. Write the code that periodically pulls in, parses, and stores in the database, all the known feeds.

7. Write the code that creates the feed web page from a user's feeds.

8. Add Twitter feeds (with twitter_kit).

9. Go live.

10. Profit (Google ads and monthly payments to disable them or get more history)

11. Add an external API for aggregated feeds, likely as RSS.

Other features, a grab bag as I think of them:

• Multiple named feeds per user.

• Public URLs for a subset of your named feeds. Global names, first come first served, and /<you>/<name>, where <name> defaults to what you call it internally, but can be different.

• An RSS feed for each of those public URLs.

• JavaScript to insert a public feed on your web site. This probably already exists.