A live example: bwkfanboy.herokuapp.com
bwkfanboy is a converter from a raw HTML to an Atom feed. You can use it to watch sites that do not provide its own feed.
The converter is not a magick tool: you'll need to write a plugin (in Ruby) for an each site you want to watch. bwkfanboy gives guidelines and a general assistance.
Version 2.0.0 was rewritten form scratch. Plugins from previous versions will not work. See github.com/gromnitsky/bwkfanboy_old repo for old versions.
NAME VER DESCRIPTION bwk 2 Brian Kernighan's articles from Daily Princetonian freebsd-ports-update 3 News from FreeBSD ports inc 1 Articles (per-user) from inc.com econlib 1 Latest articles from econlib.org
# gem install bundler
Install all dependencies. You have 2 choices:
# bundle install
Or only for this program
% bundle install --path vendor/bundle
You'll have to prefix other commands with 'bundle exec' though.
% rackup -E production
Point a browser to 127.0.0.1:9292
Plugins can be in system
`gem env gemdir`/gems/bwkfanboy-x.y.z/plugins
or user's home
The program consists of 3 parts:
bwkfanboy script that takes 1 parameter: the name of a file in plugin directories (without the .rb suffix). So, for example to get an atom feed from dailyprincetonian.com, you type:
% bwkfanboy bwk
and it will load /usr/local/lib/ruby/gems/1.9/gems/bwkfanboy-2.0.0/plugins/bwk.rb file on my FreeBSD machine, fetch and parse the html from dailyprincetonian.com and generate the required feed, dumping it to stdout.
The script is just a convenient wrapper for 2 separate utils below. (Although it doesn't executes them, but uses their internal api.)
Takes 1 parameter: a plugin name.
This util reads stdin, expecting it to be a html, parses it and dumps the result to stdout as MessagePack object in UTF-8.
Reads stdin expecting it to be a proper MessagePack object.
The result will be an Atom feed dumped to stdout in UTF-8.
So, without the wrapper all this together looks like:
% curl http://example.org | bwkfanboy_parse myplugin | bwkfanboy_generate
bwkfanboy includes a small sinatra app. See the installation phase.