A converter from a raw HTML to an Atom feed. You can use it to watch sites that do not provide its own feed.
Ruby JavaScript
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
doc
etc
lib/bwkfanboy
plugins
public
test
views
.gitignore
Gemfile
Gemfile.lock
Procfile
README.rdoc
Rakefile
bwkfanboy.gemspec
config.ru
shotgun.rb

README.rdoc

About

A live example: bwkfanboy.herokuapp.com

bwkfanboy is a converter from a raw HTML to an Atom feed. You can use it to watch sites that do not provide its own feed.

The converter is not a magick tool: you'll need to write a plugin (in Ruby) for an each site you want to watch. bwkfanboy gives guidelines and a general assistance.

Version 2.0.0 was rewritten form scratch. Plugins from previous versions will not work. See github.com/gromnitsky/bwkfanboy_old repo for old versions.

Included Plugins

NAME                 VER DESCRIPTION

bwk                    2 Brian Kernighan's articles from Daily Princetonian
freebsd-ports-update   3 News from FreeBSD ports
inc                    1 Articles (per-user) from inc.com
econlib                1 Latest articles from econlib.org

Installation

Install bundler:

# gem install bundler

Install all dependencies. You have 2 choices:

  1. Usual via

    # bundle install
  2. Or only for this program

    % bundle install --path vendor/bundle

    You'll have to prefix other commands with 'bundle exec' though.

Run application:

% rackup -E production

Point a browser to 127.0.0.1:9292

Architecture

Plugins

Plugins can be in system

`gem env gemdir`/gems/bwkfanboy-x.y.z/plugins

or user's home

~/.bwkfanboy/plugins

directory.

Pipeline

The program consists of 3 parts:

  1. bwkfanboy script that takes 1 parameter: the name of a file in plugin directories (without the .rb suffix). So, for example to get an atom feed from dailyprincetonian.com, you type:

    % bwkfanboy bwk

    and it will load /usr/local/lib/ruby/gems/1.9/gems/bwkfanboy-2.0.0/plugins/bwk.rb file on my FreeBSD machine, fetch and parse the html from dailyprincetonian.com and generate the required feed, dumping it to stdout.

    The script is just a convenient wrapper for 2 separate utils below. (Although it doesn't executes them, but uses their internal api.)

  2. bwkfanboy_parse

    Takes 1 parameter: a plugin name.

    This util reads stdin, expecting it to be a html, parses it and dumps the result to stdout as MessagePack object in UTF-8.

  3. bwkfanboy_generate

    Reads stdin expecting it to be a proper MessagePack object.

    The result will be an Atom feed dumped to stdout in UTF-8.

So, without the wrapper all this together looks like:

% curl http://example.org | bwkfanboy_parse myplugin | bwkfanboy_generate

HTTP

bwkfanboy includes a small sinatra app. See the installation phase.