munchr + friends
various utils to enhance web2.0 productivity. Munchr provides tools to interact
with websites, rss feeds and instant messaging:

    * nbox is a general widget for displaying information
    * munchr fetches rss feeds (possibly to nbox)
    * wachr combines nbox&munchr for your convenience
    * im_mcabber&pidgin feed IM messages to nbox
    * feedr POSTs data to websites
    * gaskr is an input widget, that interacts with feedr
    * bildr helps prepare gaskr configurations

Keep it simple stupid was the main goal, so all components do only one thing.
You can mix and match the functionality to suit your needs. If you don’t like
the looks of gaskr it is easy to write another GUI frontend. Or you don’t like
dzen? Your OS has something similar? Use that if you prefer.

Generally speaking for input related tasks you will end up with a bunch of
desktop or keyboard shortcuts to gaskr/feedr invocations. For output you will
probably have a wachr running, which polls rss feeds (munchr) and displays
(nbox) various incomming information in the dzen window.

So which websites could be integrated more thightly into your desktop?

Availability
munchr+friends is released under the GPLv3 and avaible via git:
   git clone git://github.com/stef/munchr.git
the script itself is very short, download it directly

wow, another flexible framework, so what?
sure it is a framework, but i am using it for daily work.

Using munchr I monitor a bunch of work related rss feeds (document changes in
various document management systems, new sw releases, svn updates, etc), all
information this is fed into the nbox.

Usually I am using mcabber for my rather minimal instant messaging needs, so
any new messages addressed at me, shows up also in the nbox.

I use gaskr/feedr to submit new ideas, appointments and tasks into my online
todo list, which is based on the great tikiwiki and its trackers.

installation
    * dependencies:

      apt-get install ksh dzen2 imagemagick python2.5 python-magic python-feedparser \
        python-yaml python-4suite-xml python-dbus python-egenix-mxdatetime

    * use

      ./setup

unfortunately there is no packaging available. everything can be run from the
directory where the source has been extracted.

related files
    * doc/munchrrc: basic configuration for all compontents
    * setup: script to setup all files, directories and pipes for basic functioning.

Getting Information
For daily work small RSS updates, IM messages and other various bits of
information flow in. Munchr is a simple RSS parser which can collect data from
restricted websites. im_mcabber is an extension of the console im client
mcabber, so that incomming IM messages are routed to nbox. nbox is a small GUI
display for all collected information. Initially it is only one line high, but
if you move your cursor over it, it expands and you can scroll in the history,
behold the magic of dzen2 ;)

nbox
This is is the widget displaying the various informations gathered by other
utilities.

nbox waits for information written to ~/.config/munchr/notifications and
displays them in a dzen widget at the bottom right of your screen. Hover over
the widget and you get a scrollable backlog of all previous events. Comming up,
klicking on events makes something meaningful.

munchr
Reads the contents of

~/.config/munchr/feeds

– one line per url, gets each feed and emits the new items to nbox

munchr can access sites using the HTTP Basic Authentication mechanism, so you
can also use munchr on restricted sites.

wachr
wachr is a convenience wrapper around nbox and munchr. wachr starts an nbox and
polls munchr every 2 minutes.

pidgin
pidgin is an experimental feature to route pidgin/purple IM messages to nbox.

im_mcabber
im_mcabber routes IM messages from mcabber to nbox. mcabber needs to be
configured to use im_mcabber by setting in ~/.mcabberrc:

   set events_command = <pathto>im_mcabber
   set event_log_files = 1
   set event_log_dir = ~/.mcabber/event_files

make sure, that ~/.mcabber/event_files exists:

   mkdir -p ~/.mcabber/event_files

Giving Information
The other important aspect of munchr+friends is posting information to the net.
The primary way of doing this, is to post data on various webserver
applications. For example it is possible to post a query to google and process
it with munchr+friends.

Posting information is done by feedr, it takes a set of parameters and values
and submits it to the appropriate webservice. This set of parameters can be
static – comming from a file. This is handy to automatize logins for example
(beware to protect your files containing passwords). Other information might be
asked from the user, gaskr provides a GUI to prepare data ready for feedr.
Unfortunately we must know in advance what kind of data the webservice
requires, so we must set up some config files for gaskr. Scraping initial
information for gaskr config files is the job of bildr, it can login onto sites
and scrape all form data from html content. This information should be enough
to start gaskr with, but it can be further customized by editing the config
files directly.

feedr
feedr submits values via HTTP GET/POST requests to servers.
feedr gets the data to be submitted via files and standard input. The format is
based on YAML (yet another meta language – iirc).

feedr parses the output according to a specified xpath expression, matches
printed on completion.
feedr config files can be used to automate tasks that don’t need user input. It
is also possible to store login credentials in such files. These files can then
be used to login to protected sites using the ‘-l ’ parameter.

   { action: 'https://www.example.com/tiki-login.php',
      resultxpath: 'string(//h:div[@class="simplebox error"]/text())',
      postfields: {
         user: mstef,
         pass: password,
      }
   }

gaskr
gaskr is a graphical frontend to feedr.

gaskr parses a YAML configfile and displays the corresponding graphical input
form. After submission of the form gaskr formats the results so that feedr can
send it to the webserver.

gaskr configfiles always have ‘resultpath’, ‘action’ and ‘fields’ values.
here is an example of cfgs/gaskr/google:

   resultxpath: '//h:h3[@class="r"]/h:a'
   action: 'http://www.google.com/search'
   fields: [ 
      { type: hidden, name: hl, value: en, },
      { type: text, label: Google Search, name: q, primari: 1},
      { type: submit, name: btnG, value: Google Search, },
      { type: submit, name: btnI, value: I'm Feeling Lucky, }, 
      ]

bildr
bildr helps you prepare configfiles for gaskr. bildr connects to the webserver
and dumps all forms found on the page.

bildr accepts a ‘-l ’ parameter, so it can help build config files for gaskr.
In the example below I prepare a configfile for an authenticated webbased task application:

   ./bildr -l cfgs/feedr/company.site.login 'https://www.example.com/tiki-view_tracker.php?trackerId=3' >cfgs/gaskr/company.addtodo

the result can be edited, e.g. * non-important fields deleted or * hide static values

Examples
complex example, query google feed results to nbox (needs links2 to dump html):

   ./gaskr <cfgs/gaskr/google | ./feedr -n |
         sed 's/^$/<br \/>/' >/tmp/asdf.html && 
         links2 -dump /tmp/asdf.html >>~/.config/munchr/notifications && 
         rm /tmp/asdf.html