Skip to content
Psycojoker edited this page Jan 14, 2012 · 5 revisions

munchr + friends

various utils to enhance web2.0 productivity. Munchr provides tools to interact with websites, rss feeds and instant messaging:

  • nbox is a general widget for displaying information
  • munchr fetches rss feeds (possibly to nbox)
  • wachr combines nbox&munchr for your convenience
  • im_mcabber&pidgin feed IM messages to nbox
  • feedr POSTs data to websites
  • gaskr is an input widget, that interacts with feedr
  • bildr helps prepare gaskr configurations

Keep it simple stupid was the main goal, so all components do only one thing. You can mix and match the functionality to suit your needs. If you don’t like the looks of gaskr it is easy to write another GUI frontend. Or you don’t like dzen? Your OS has something similar? Use that if you prefer.

Generally speaking for input related tasks you will end up with a bunch of desktop or keyboard shortcuts to gaskr/feedr invocations. For output you will probably have a wachr running, which polls rss feeds (munchr) and displays (nbox) various incoming information in the dzen window.

So which websites could be integrated more tightly into your desktop?

Availability

munchr+friends is released under the GPLv3 and available via git:

git clone git://github.com/stef/grindr.git

the script itself is very short,
download it directly
- don’t worry about the long filename.

wow, another flexible framework, so what?

sure it is a framework, but i am using it for daily work.

Using munchr I monitor a bunch of work related RSS feeds (document changes in various document management systems, new SW releases, SVN updates, etc), all information this is fed into the nbox.

Usually I am using mcabber for my rather minimal instant messaging needs, so any new messages addressed at me, shows up also in the nbox.

I use gaskr/feedr to submit new ideas, appointments and tasks into my online todo list, which is based on the great tikiwiki and its trackers.

installation

  • dependencies:
    apt-get install ksh dzen2 imagemagick python2.5 python-magic python-feedparser \
    python-yaml python-4suite-xml python-dbus python-egenix-mxdatetime python-4suite-xml
  • use
    ./setup

unfortunately there is no packaging available. everything can be run from the
directory where the source has been extracted.

related files

  • doc/munchrrc: basic configuration for all components
  • setup: script to setup all files, directories and pipes for basic functioning.

Getting Information

For daily work small RSS updates, IM messages and other various bits of information flow in. Munchr is a simple RSS parser which can collect data from restricted websites. im_mcabber is an extension of the console IM client mcabber, so that incoming IM messages are routed to nbox. nbox is a small GUI display for all collected information. Initially it is only one line high, but if you move your cursor over it, it expands and you can scroll in the history, behold the magic of dzen2 ;)

nbox

This is is the widget displaying the various informations gathered by other utilities.

nbox waits for information written to ~/.config/munchr/notifications and displays them in a dzen widget at the bottom right of your screen. Hover over the widget and you get a scrollable backlog of all previous events. Coming up, clicking on events makes something meaningful.

munchr

Reads the contents of

~/.config/munchr/feeds
– one line per URL, gets each feed and emits the new items to nbox

munchr can access sites using the HTTP Basic Authentication mechanism, so you can also use munchr on restricted sites.

wachr

wachr is a convenience wrapper around nbox and munchr. wachr starts an nbox and polls munchr every 2 minutes.

pidgin

pidgin is an experimental feature to route pidgin/purple IM messages to nbox.

im_mcabber

im_mcabber routes IM messages from mcabber to nbox. mcabber needs to be configured to use im_mcabber by setting in ~/.mcabberrc:


set events_command = im_mcabber
set event_log_files = 1
set event_log_dir = ~/.mcabber/event_files

make sure, that ~/.mcabber/event_files exists:

mkdir -p ~/.mcabber/event_files

Giving Information

The other important aspect of munchr+friends is posting information to the net. The primary way of doing this, is to post data on various webserver applications. For example it is possible to post a query to google and process it with munchr+friends.

Posting information is done by feedr, it takes a set of parameters and values and submits it to the appropriate webservice. This set of parameters can be static – coming from a file. This is handy to automatize logins for example (beware to protect your files containing passwords). Other information might be asked from the user, gaskr provides a GUI to prepare data ready for feedr. Unfortunately we must know in advance what kind of data the webservice requires, so we must set up some config files for gaskr. Scraping initial information for gaskr config files is the job of bildr, it can login onto sites and scrape all form data from HTML content. This information should be enough to start gaskr with, but it can be further customized by editing the config files directly.

feedr

feedr submits values via HTTP GET/POST requests to servers.

feedr gets the data to be submitted via files and standard input. The format is based on YAML (yet another meta language – iirc).

feedr parses the output according to a specified XPATH expression, matches printed on completion.

feedr config files can be used to automate tasks that don’t need user input. It is also possible to store login credentials in such files. These files can then be used to login to protected sites using the ‘-l ’ parameter.


{ action: ‘https://www.example.com/tiki-login.php’,
resultxpath: ‘string(//h:div[@class=“simplebox error”]/text())’,
postfields: {
user: mstef,
pass: password,
}
}

gaskr

gaskr is a graphical frontend to feedr.

gaskr parses a YAML configfile and displays the corresponding graphical input form. After submission of the form gaskr formats the results so that feedr can send it to the webserver.

gaskr configfiles always have ‘resultpath’, ‘action’ and ‘fields’ values.

here is an example of cfgs/gaskr/google:


resultxpath: ‘//h:h3[@class=“r”]/h:a’
action: ‘http://www.google.com/search’
fields: [
{ type: hidden, name: hl, value: en, },
{ type: text, label: Google Search, name: q, primari: 1},
{ type: submit, name: btnG, value: Google Search, },
{ type: submit, name: btnI, value: I’m Feeling Lucky, },
]

bildr

bildr helps you prepare config files for gaskr. bildr connects to the webserver and dumps all forms found on the page.

bildr accepts a ‘-l ’ parameter, so it can help build config files for gaskr.

In the example below I prepare a config file for an authenticated webbased task application:


./bildr -l cfgs/feedr/company.site.login ‘https://www.example.com/tiki-view_tracker.php?trackerId=3’ >cfgs/gaskr/company.addtodo

the result can be edited, e.g.
  • non-important fields deleted or
  • hide static values

Examples

complex example, query google feed results to nbox (needs links2 to dump html):


./gaskr <cfgs/gaskr/google | ./feedr -n |
sed ‘s/^$/<br \/>/’ >/tmp/asdf.html &&
links2 -dump /tmp/asdf.html >>~/.config/munchr/notifications &&
rm /tmp/asdf.html

nbox screenshot