Added initial browser access for tasks, and updated the README.

README.md

@@ -2,7 +2,9 @@ Shovel
 ======
 
 Shovel is like Rake for python. Turn python funcitons into tasks simply, and 
-access and invoke them from the command line. 'Nuff said.
+access and invoke them from the command line. 'Nuff said. __New__ Shovel also
+now has support for invoking the same tasks in the browser you'd normally run
+from the command line, without any modification to your shovel scripts.
 
 Philosophy of Shovel
 --------------------
@@ -12,6 +14,7 @@ Philosophy of Shovel
 - Arguments are strings -- we're not going to try to guess (there's an exception)
 - We value specificity
 - We'll inspect your tasks as much as we can
+- Tasks should be _accessible_
 
 Installing Shovel
 -----------------
@@ -116,4 +119,58 @@ if we executed the following, then `a` and `b` would be passed as True:
 	shovel foo.bar --a --b
 
 The reason for this is that flags are common for tasks, and it's a relatively
-unambiguous syntax. To a human, the meaning is clear, and now it is to shovel.
+unambiguous syntax. To a human, the meaning is clear, and now it is to shovel.
+
+Browser
+-------
+
+Shovel also now comes with a small utility using the [`bottle`](http://bottlepy.org/docs/dev/)
+framework, designed to make all your shovel tasks accessible from a browser.
+At some point, I'd like to make it accessible as an API as well, returning 
+JSON blobs instead of HTML output when requested. That said, it's not a high
+priority -- if it's something you're after, let me know!
+
+You can access the browser utility by starting up the `shovel-server` utility
+from the same directory where you'd normally run your shovel tasks. You may 
+optionally supply the `--port` option to specify the port on which you'd like
+it to listen, and `--verbose` for additional output:
+
+	# From the directory where your shovel tasks are
+	shovel-server
+
+By default, the `shovel-server` listens on port 3000, and you can access many
+of the same utilities you would from the command line. For instance, help is
+available through the [/help](http://localhost:3000/help) endpoint. Help for
+a specific function is available by providing the name of the task (or group)
+as a query parameter. To get more help on task `foo.bar`, you'd visit
+[/help?foo.bar](http://localhost:3000/help?foo.bar), etc.
+
+Tasks are executed by visiting the `/<task-name>` end-point, and the query 
+parameters are what gets provided to the function. Query parameters without
+values are considered to be positional, and query parameters with values are
+considered to be keyword arguments. For example, the following are equivalent:
+
+	# Accessing through the HTTP interface
+	curl http://localhost:3000/foo.bar?hello&and&how&are=you
+	# Accessing through the command-line utility
+	shovel foo.bar hello and how --are=you
+	# Executing the original python function
+	...
+	>>> bar('hello', 'and', 'how', are='you')
+
+In this way, we can support conventional arguments, variable arguments, and
+keyword arguments. That said, there is a slight difference in the invocation
+from the command-line and through the browser. In the command-line tool, 
+keyword arguments without values are interpreted as flags, where in the url,
+that is not the case. For example, in the following invocation, both 'a' and
+'b' would be passed as 'True' into the function, but there is no equivalent
+in the URL form:
+
+	shovel foo.bar --a --b
+
+To-Do
+=====
+
+1. Ensure that the `shovel-server` utility can detect task failures
+1. Allow the `shovel-server` utility to return `application/json` when requested
+1. Have the `shovel-server` recognize when scripts have been updated, without restart

bin/shovel-server

@@ -0,0 +1,177 @@
+#! /usr/bin/env python
+
+# Copyright (c) 2011 SEOmoz
+# 
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+# 
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+# 
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# ======================================================
+# This is to turn your shovel tasks into HTTP endpoints
+# ======================================================
+
+import argparse
+import pkg_resources
+# First off, read the arguments
+parser = argparse.ArgumentParser(description='Shovel with URLs')
+
+parser.add_argument('--port', dest='port', default=3000, help='What port to run on')
+parser.add_argument('--verbose', dest='verbose', action='store_true',
+    help='Be extra talkative')
+
+ver = pkg_resources.require('shovel')[0].version
+parser.add_argument('--version', action='version', version='Shovel v %s' %(ver), help='print the version of Shovel.')
+
+# Parse our arguments
+clargs, remaining = parser.parse_known_args()
+
+import shovel
+import logging
+if clargs.verbose:
+    shovel.logger.setLevel(logging.DEBUG)
+
+# Load in all the functions from shovel files
+shovel.load()
+
+import os
+import sys
+import pkgutil
+import traceback
+from cStringIO import StringIO
+
+import bottle
+from bottle import Bottle, run, request
+
+# This is the base path from which we'll import our templates
+bottle.TEMPLATE_PATH.append(os.path.join(shovel.__path__[0], 'templates/'))
+
+# And this is our bottle app
+app = Bottle()
+
+# This is a helper to allow us to run a task, and then get
+# back a dictionary of the various outputs -- stdout, stderr,
+# exceptions, return value
+def capture(f, *args, **kwargs):
+    stdout, stderr = sys.stdout, sys.stderr
+    sys.stdout = out = StringIO()
+    sys.stderr = err = StringIO()
+    result = {
+        'exception': None,
+        'stderr'   : None,
+        'stdout'   : None,
+        'return'   : None
+    }
+    try:
+        result['return'] = f(*args, **kwargs)
+    except:
+        result['exception'] = traceback.format_exc()
+    sys.stdout, sys.stderr = stdout, stderr
+    result['stderr'] = err.getvalue()
+    result['stdout'] = out.getvalue()
+    return result
+
+# And let's begin our bottle application
+# Our url format shall be:
+# host:port/task.name.and.so.forth?arg1&arg2&kwarg1=val1&arg3
+#
+# In particular, the task name is like it would be provided on the command
+# line, like 'foo.bar'. Any query parameters without a value will be considered
+# positional arguments, and all others will be considered keyword arguments.
+
+def help_helper(tasks):
+    # This tries to print the reported tasks in a nice, heirarchical fashion
+    modules = {}
+    for task in tasks:
+        m = task.fullname.split('.')
+        # Pop off the last name of the module
+        n = m.pop(-1)
+        mod = modules
+        for name in m:
+            mod.setdefault(name, {})
+            mod = mod[name]
+
+        mod[task.name] = {
+            'name'  : task.name,
+            'full'  : task.fullname,
+            'file'  : task.file,
+            'line'  : task.line,
+            'module': task.module,
+            'doc'   : task.doc,
+            'args'  : repr(shovel.Args(task.spec))
+        }
+
+    return modules
+
+def _parse(qs):
+    args   = []
+    kwargs = {}
+    for pair in qs.split('&'):
+        key, sep, value = pair.partition('=')
+        if key and not value:
+            args.append(key)
+        elif key:
+            kwargs[key] = value
+
+    return args, kwargs
+
+@app.route('/help')
+def _help():
+    args, kwargs = _parse(request.query_string)
+    if len(args):
+        tasks = []
+        for name in args:
+            tasks.extend(shovel.Task.find(name))
+        tasks = [t for t in tasks if t]
+    else:
+        tasks = shovel.Task.find()
+
+    return bottle.template(pkgutil.get_data('shovel', 'templates/help.tpl'), tasks=[{
+        'name'  : task.name,
+        'full'  : task.fullname,
+        'file'  : task.file,
+        'line'  : task.line,
+        'module': task.module,
+        'doc'   : task.doc,
+        'args'  : repr(shovel.Args(task.spec))
+    } for task in tasks])
+
+# Now, we'll catch the task names and execute accordingly
+@app.route('/<task>')
+def _task(task):
+    args, kwargs = _parse(request.query_string)
+    tasks = shovel.Task.find(task)
+    if len(tasks) == 1:
+        t = tasks[0]
+        arg = shovel.Args(t.spec)
+        arg.eval(*args, **kwargs)
+        return bottle.template(
+            pkgutil.get_data('shovel', 'templates/results.tpl'),
+            task   =t,
+            args   =repr(arg),
+            results=capture(t, *args, **kwargs))
+    else:
+        return {
+            'error': 'Found %i tasks matching %s' % (len(tasks), task)
+        }
+
+# And then we have all of our static content
+@app.route('/static/<path:path>')
+def callback(path):
+    return pkgutil.get_data('shovel', 'static/' + path)
+
+run(app, host='', port=clargs.port)

setup.py

@@ -11,18 +11,21 @@
 		'dependencies' : ['argparse']
 	}
 
-setup(name           = 'shovel',
-	version          = '0.1.3',
-	description      = 'Not Rake, but Shovel',
-	long_description = 'Execute python functions as tasks',
-	url              = 'http://github.com/seomoz/shovel',
-	author           = 'Dan Lecocq',
-	author_email     = 'dan@seomoz.org',
-    license          = "MIT License",
-	keywords         = 'tasks, shovel, rake',
-	packages         = ['shovel'],
-	scripts          = ['bin/shovel'],
-	classifiers      = [
+setup(name               = 'shovel',
+	version              = '0.1.4',
+	description          = 'Not Rake, but Shovel',
+	long_description     = 'Execute python functions as tasks',
+	url                  = 'http://github.com/seomoz/shovel',
+	author               = 'Dan Lecocq',
+	author_email         = 'dan@seomoz.org',
+    license              = "MIT License",
+	keywords             = 'tasks, shovel, rake',
+	packages             = ['shovel'],
+	package_dir          = {'shovel': 'shovel'},
+    package_data         = {'shovel': ['templates/*.tpl', 'static/css/*']},
+    include_package_data = True,
+	scripts              = ['bin/shovel', 'bin/shovel-server'],
+	classifiers          = [
         'License :: OSI Approved :: MIT License',
 		'Programming Language :: Python',
 		'Intended Audience :: Developers',

shovel/static/css/style.css

@@ -0,0 +1,45 @@
+body {
+	font-family: "Lucida Grande", Lucida, Verdana, sans-serif;
+}
+
+.task-container {
+	background-color: whitesmoke;
+	padding-bottom: 10px;
+}
+
+.task-name {
+	font:bold 24px Lucida Grande;
+	background-color: black;
+	color: white;
+	margin-top: 10px;
+	padding: 5px;
+}
+
+.task-file {
+	background-color: black;
+	color: white;
+	font-weight: bold;
+	font-size: 8px;
+	padding-left: 5px;
+	padding-bottom: 10px;
+}
+
+.task-doc {
+	margin-top: 10px;
+	margin-left: 15px;
+}
+
+.task-result {
+	margin-top: 10px;
+	margin-bottom: 5px;
+	margin-left: 15px;
+	font-family: "Courier New", Courier, mono;
+	font-weight: bold;
+}
+
+.task-stdout, .task-stderr, .task-returned, .task-exception {
+	background: lightgrey;
+	margin-top: 10px;
+	font-weight: normal;
+	padding: 10px;
+}

shovel/templates/help.tpl

@@ -0,0 +1,15 @@
+<html>
+	<head>
+		<title>Help</title>
+		<link rel='stylesheet' href='static/css/style.css' type='text/css' />
+	</head>
+	<body>
+		%for task in tasks:
+		<div class='task-container'>
+			<div class='task-name'>{{ task['full'] }}{{ task['args'] }}</div>
+			<div class='task-file'>In {{ task['file'] }}:{{ task['line'] }}</div>
+			<div class='task-doc' >{{ task['doc'] }}</div>
+		</div>
+		%end
+	</body>
+</html>

shovel/templates/results.tpl

@@ -0,0 +1,39 @@
+<html>
+	<head>
+		<title>{{ task.fullname }}</title>
+		<link rel='stylesheet' href='static/css/style.css' type='text/css' />
+	</head>
+	<body>
+		<div class='task-container'>
+			<div class='task-name'>
+				{{ task.fullname }}
+				{{ args }}
+			</div>
+
+			%import cgi
+			%if results['stdout']:
+			<div class='task-result'>
+				stdout: <div class='task-stdout'>{{! cgi.escape(results['stdout']).replace('\n', '<br/>') }}</div>
+			</div>
+			%end
+
+			%if results['stderr']:
+			<div class='task-result'>
+				stderr: <div class='task-stderr'>{{! cgi.escape(results['stderr']).replace('\n', '<br/>')}}</div>
+			</div>
+			%end
+
+			%if results['return']:
+			<div class='task-result'>
+				returned: <div class='task-returned'>{{! cgi.escape(results['return']).replace('\n', '<br/>')}}</div>
+			</div>
+			%end
+
+			%if results['exception']:
+			<div class='task-result'>
+				exception: <div class='task-exception'>{{! cgi.escape(results['exception']).replace('\n', '<br/>')}}</div>
+			</div>
+			%end
+		</div>
+	</body>
+</html>