Now ttrack uses cmdparser and own time parsing.

All ttrack commands now use the cmdparser library to automatically
generate a parser for the command based on the command syntax in the
docstring. Command error reporting is quite different as a result
(hopefully more helpful) and a lot of the boiler-plate error checking
and command completion functions have been stripped out.

The new datetimeparse library, based on classes in cmdparser, now
replaces the third party parsedatetime library. This is mostly because
this library made some annoying assumptions - for example, days of the
week defaulted to the next instance, which is almost never what you want
when adjusting start/stop times. The new library also allows a much
nicer syntax for specifying periods (see next paragraph).

The summary command has seen some significant syntactic differences -
instead of choosing from a fixed list of period types (day, week,
month), a fairly freeform date/time syntax is now permitted, which can
run between arbitrary dates. The underlying library always supported
this, but there wasn't a good way to express it in the command syntax.

This commit also introduces considerable new functionality into
cmdparser, adding a Subtree class to allow an entire parse tree to be
"hidden" behind a single token in the top-level tree, including a
conversion function which can squash syntax down to useful values (for
example, datetimeparse uses this to convert to a datetime instance).

README.md

@@ -15,11 +15,11 @@ tags can be applied to tasks for flexible categorisation.
 Installation
 ------------
 
-Python 2.5 or later should be sufficient to execute the scripts. If the
-parsedatetime library (http://code.google.com/p/parsedatetime/) then this will
-be used for more friendly date and time parsing.
+Python 2.5 or later should be sufficient to execute the scripts. Just ensure
+tracklib.py, cmdparser.py and datetimeparse.py are available on the PYTHONPATH
+and run ttrack.py.
 
-Just ensure tracklib.py is available on the PYTHONPATH and run ttrack.py.
+> **NOTE:** Proper packaging is on the todo list!
 
 When first run, an SQLite database is created in your home directory in a file
 called `.timetrackdb`. It's currently not possible to change the name used
@@ -198,12 +198,18 @@ are currently four types:
 * `entries`: Shows raw task times.
 
 Following the report type, the period over which the report should be run is
-specified - this can be `day`, `week` or `month`. If no other arguments are
-specified, the report for the current day/week/month is displayed. Otherwise,
-the next argument can be a number indicating how many days/weeks/months ago
-the report should be run. Specifying 0 indicates the current period (the
-default if the value is omitted), 1 indicates the previous day/week/month,
-2 indicates the period before that, etc.
+specified - the syntax for this is fairly flexible and some examples of
+what will be accepted are:
+
+* "yesterday"
+* "2 weeks ago"
+* "last month"
+* "December 2012"
+* "between 15/10/2011 and today"
+
+> **NOTE:** When providing two dates to run the report, bear in mind that the
+> first date will be inclusive but the second date will be exclusive (so the
+> example "between 15/10/2011 and today" won't include today).
 
 Finally, if splitting by task (only), a the keyword `tag` followed by a tag
 name can be specified at the end of the command. If so, the list of tasks
@@ -213,20 +219,20 @@ In case you're thinking that all sounds a bit too complicated, here are some
 simple examples which probably cover most of what you need, followed by an
 explanation of what will be displayed.
 
-    summary task time week
+    summary task time this week
 
 Display a summary of the time spent on each task so far this week.
 
-    summary tag time day 1
+    summary tag time yesterday
 
 Display a summary of the time spent yesterday on tasks in each tag.
 
-    summary task switches month 1
+    summary task switches last month
 
 Display the number of times each task interrupted another one in the previous
 month.
 
-    summary task diary month tag projects
+    summary task diary this month tag projects
 
 Display diary entries recorded so far this month for all tasks with tag
 `projects`.
@@ -243,13 +249,21 @@ For this to work you'll have to create tasks to track all the things which
 disturb you - for example, if you are interrupted by calls from customers,
 you could create a task `customersupport` to track this.
 
+Remember that context switches are budgeted against the new task (i.e. the
+"interrupting" task), not the old one (i.e. the "interrupted" task).
+
 To count as a context switch and be included in the totals for the `switches`
 report, a task must be different to the previous task and start less than a
 minute after the first one ended. When reporting by tag rather than task,
-the definition is changed to the new task must have at least one tag which
-the old task does not. For example, if two different tasks both have only the
-`coding` tag then switching between them will count as a context switch in a
-`task` report, but not in a `tag` report.
+the switch is only counted if the new task has at least one tag which the old
+task does not.
+
+For example, if two different tasks both have only the `coding`
+tag then switching between them will count as a context switch in a `task`
+report, but not in a `tag` report. By comparison, if the old task was tagged
+with "A", "B" and "C" and the new task tagged with "C", "D" and "E" then
+the context switches count would be incremented for tags "D" and "E" only
+due to the change.
 
 
 Advanced Usage

cmdparser.py

@@ -110,8 +110,8 @@ def match(self, compare_items, fields=None, completions=None, trace=None,
     def check_match(self, items, fields=None, trace=None, context=None):
         """Return None if the specified command-line is valid and complete.
 
-        If the command-line doesn't match, the first non-matching item is
-        returned, or the empty string if the command was incomplete.
+        If the command-line doesn't match, an appropriate error explaining the
+        lack of match is returned.
 
         Calling code should typically use this instead of calling match()
         directly. Derived classes shouldn't typically override this method.
@@ -120,7 +120,9 @@ def check_match(self, items, fields=None, trace=None, context=None):
             unparsed = self.match(items, fields=fields, trace=trace,
                                   context=context)
             if unparsed:
-                return "unused parameters from %r onwards" % (unparsed[0],)
+                suffix = " ".join(unparsed)
+                suffix = suffix[:29] + "..." if len(suffix) > 32 else suffix
+                return "command invalid somewhere in: %r" % (suffix,)
             else:
                 return None
         except MatchError, e:
@@ -239,6 +241,64 @@ def match(self, compare_items, fields=None, completions=None, trace=None,
 
 
 
+class Subtree(ParseItem):
+    """Matches an entire parse tree, converting the result to a single value.
+
+    This item is intended for use in applications which wish to present
+    a potentially complicated potion of the parse tree as a single argument.
+    A good example of this is a time specification, which might accept
+    strings such as "yesterday at 3:34" or "25 minutes ago", but wish to
+    store the result in the fields dictionary as a single datetime instance.
+
+    By default, command completion within the subtree will be enabled - if the
+    tree should be treated more like a token then it may be useful to disable
+    completion (i.e. always return no completions), and this can be done by
+    setting the suppress_completion parameter to the constructor to True.
+    """
+
+    def __init__(self, name, spec, ident_factory=None,
+                 suppress_completion=False):
+        self.name = name
+        self.suppress_completion = suppress_completion
+        # Allow any parsing exceptions to be passed out of constructor.
+        self.parse_tree = parse_spec(spec, ident_factory=ident_factory)
+
+
+    def __str__(self):
+        return '<' + str(self.name) + '>'
+
+
+    def convert(self, args, fields, context):
+        """Convert matched items into field value(s).
+
+        This method is called when the subtree matches and is passed the
+        subset of the argument list which matched as well as the fields
+        array that was filled in. It should return a list of values which
+        will be appended to those for the field name.
+
+        The base class instance simply appends the list of matched arguments
+        to the field values list.
+        """
+        return args
+
+
+    def match(self, compare_items, fields=None, completions=None, trace=None,
+              context=None):
+
+        tracer = CallTracer(trace, self, compare_items)
+        subtree_fields = {}
+        completions = None if self.suppress_completion else completions
+        new_items = self.parse_tree.match(compare_items, fields=subtree_fields,
+                                          completions=completions, trace=trace,
+                                          context=context)
+        consumed = compare_items[:len(compare_items)-len(new_items)]
+        if fields is not None:
+            field_value = fields.setdefault(str(self), [])
+            field_value.extend(self.convert(consumed, subtree_fields, context))
+        return new_items
+
+
+
 class Alternation(ParseItem):
     """Matches any of a list of alternative Sequence items.
 
@@ -349,6 +409,21 @@ def get_values(self, context):
         return [self.token]
 
 
+    def convert(self, arg, context):
+        """Argument conversion hook.
+
+        A matched argument is filtered through this method before being placed
+        in the "fields" dictionary passed on the match() method. This allows
+        derived classes to, for example, convert the type of the argument to
+        something that's more useful to the code using the value.
+
+        The first argument (after self) is the matched token string, the second
+        is the context passed to match(). The return value should be a list to
+        be added to the list of values for the field.
+        """
+        return [arg]
+
+
     def match(self, compare_items, fields=None, completions=None, trace=None,
               context=None):
         """See ParseItem.match()."""
@@ -359,13 +434,15 @@ def match(self, compare_items, fields=None, completions=None, trace=None,
                 completions.update(self.get_values(context))
             tracer.fail([])
             raise MatchError("insufficient args for %r" % (str(self),))
+        arg = compare_items[0]
         for value in self.get_values(context):
-            if compare_items[0] == value:
+            if arg == value:
                 if fields is not None:
-                    fields.setdefault(str(self), []).append(value)
+                    arg_list = fields.setdefault(str(self), [])
+                    arg_list.extend(self.convert(arg, context))
                 return compare_items[1:]
         tracer.fail(compare_items)
-        raise MatchError("%r doesn't match %r" % (compare_items[0], str(self)))
+        raise MatchError("%r doesn't match %r" % (arg, str(self)))
 
 
 
@@ -380,8 +457,38 @@ def __str__(self):
         return "<" + self.name + ">"
 
 
-    def validate(self, item):
-        return True
+    def validate(self, arg, context):
+        """Validation hook.
+
+        Derived classes can use this to indicate whether a given parameter
+        value is accpetable. Return True if yes, False otherwise. The base
+        class version returns True unless convert() raises ValueError, in
+        which case False (note that no other exceptions are caught).
+
+        For cases where a small set of values is acceptable it may be more
+        appropriate to derive from Token and override get_values(), which has
+        the advantage of also allowing tab-completion.
+        """
+        try:
+            self.convert(arg, context)
+            return True
+        except ValueError:
+            return False
+
+
+    def convert(self, arg, context):
+        """Argument conversion hook.
+
+        A matched argument is filtered through this method before being placed
+        in the "fields" dictionary passed on the match() method. This allows
+        derived classes to, for example, convert the type of the argument to
+        something that's more useful to the code using the value.
+
+        The first argument (after self) is the matched token string. The second
+        is the context passed to match(). The return value should be a list to
+        be added to the list of values for the field.
+        """
+        return [arg]
 
 
     def match(self, compare_items, fields=None, completions=None, trace=None,
@@ -390,15 +497,35 @@ def match(self, compare_items, fields=None, completions=None, trace=None,
         if not compare_items:
             tracer.fail([])
             raise MatchError("insufficient args for %r" % (str(self),))
-        if not self.validate(compare_items[0]):
-            raise MatchError("%r is not a valid %s"
-                             % (compare_items[0], str(self)))
+        arg = compare_items[0]
+        if not self.validate(arg, context):
+            raise MatchError("%r is not a valid %s" % (arg, str(self)))
         if fields is not None:
-            fields.setdefault(str(self), []).append(compare_items[0])
+            fields.setdefault(str(self), []).extend(self.convert(arg, context))
         return compare_items[1:]
 
 
 
+class IntegerToken(AnyToken):
+    """As AnyToken, but only accepts integers and converts to int values."""
+
+    def __init__(self, name, min_value=None, max_value=None):
+        AnyToken.__init__(self, name)
+        self.min_value = min_value
+        self.max_value = max_value
+
+
+    def convert(self, arg, context):
+        value = int(arg)
+        if (self.min_value is not None and value < self.min_value or
+            self.max_value is not None and value > self.max_value):
+            raise ValueError("integer value %d outside range %d-%d" %
+                             (value, self.min_value, self.max_value))
+        return [value]
+
+
+
+
 class AnyTokenString(ParseItem):
     """Matches the remainder of the command string."""
 
@@ -410,22 +537,50 @@ def __str__(self):
         return "<" + self.name + "...>"
 
 
-    def validate(self, items):
-        return True
+    def validate(self, items, context):
+        """Validation hook.
+
+        Derived classes can use this to indicate whether a given parameter
+        list is accpetable. Return True if yes, False otherwise. The base
+        class version returns True unless convert() raises ValueError, in
+        which case False (note that no other exceptions are caught).
+        """
+        try:
+            self.convert(items, context)
+            return True
+        except ValueError:
+            return False
+
+
+    def convert(self, items, context):
+        """Argument conversion hook.
+
+        A matched argument list is filtered through this method before being
+        placed in the "fields" dictionary passed on the match() method. This
+        allows derived classes to, for example, convert the types of arguments
+        or concatenate them.
+
+        The first argument (after self) is the list of matched arguments. The
+        second is the context argument passed to match(). The return value
+        should be a list which is added to the list of values for the field -
+        the list need not be the same length as the input.
+        """
+        return items
 
 
     def match(self, compare_items, fields=None, completions=None, trace=None,
               context=None):
         tracer = CallTracer(trace, self, compare_items)
         if not compare_items:
             raise MatchError("insufficient args for %r" % (str(self),))
-        if not self.validate(compare_items):
+        if not self.validate(compare_items, context):
             args = " ".join(compare_items)
             args = args[:20] + "[...]" if len(args) > 25 else args
             tracer.fail([])
             raise MatchError("%r is not a valid %s" % (args, str(self)))
         if fields is not None:
-            fields.setdefault(str(self), []).extend(compare_items)
+            arg_list = fields.setdefault(str(self), [])
+            arg_list.extend(self.convert(compare_items, context))
         return []
 
 
@@ -581,7 +736,7 @@ def wrapper(cmd_self, args):
 
         # Ensure wrapper has correct docstring, and also store away the parse
         # tree for the class wrapper to use for building completer methods.
-        wrapper.__doc__ = self.new_docstring
+        wrapper.__doc__ = "\n" + self.new_docstring + "\n"
         wrapper._cmdparser_decorator = self
 
         return wrapper