HOWTO update

HOWTO.rst

@@ -20,6 +20,7 @@ Specifying Tokens
  - PYCOMMENT # python-style comment
  - NUMBER # an integer or float
  - INT # an integer
+ - HEX # an 0xabc012 hex number.
  - ID # usually [a-zA-Z\_][a-zA-Z_0-9]*
  - ANY # matches any single char
 
@@ -37,6 +38,10 @@ Specifying Tokens
  - matches 'one of the specified strings' *followed by a non-id character*
  - example: RESERVED_WORDS
 
+- IIdToken
+
+ - same as IdToken, but caseInsensitive
+
 - ReToken
 
  - this is the most flexible, but also the slowest. Use only when needed
@@ -59,10 +64,13 @@ Defining Rules
 
     - ``"string literal"``
     - ``[optional, children, ...]``
+    - ``(nested, (tuples, are), collapsed)``
     - ``star(zero, or_more)``
     - ``plus(one, or_more)``
     - ``_or(one, of, these)``
-    - ``TOKEN``
+    - ``_not(this, stuff)`` # checks the content, and if it *does not* match,
+      consumes **a single** token.
+    - ``TOKEN_NAME``
 
 Abstract Syntax Tree Attributes
 *******************************
@@ -91,7 +99,7 @@ Simple
 
 :atype:
 
-    match the first element of atype [raises an error if there are no elements
+    match the first element of atype [becomes None if there are no elements
     matching atype]
 
 :[atype]: match all elements of atype [becomes a list]
@@ -106,11 +114,13 @@ The complex definition is a dictionary, where the ``type`` parameter follows
 the *simple* definition above.
 
 :type: atype | [atype] | [atype, anothertype]
+:single: (bool) only use if you want to override the normal inference.
 :start: (int) used for slicing (default: 0)
 :end: (int) also for slicing (default: 0 [means no limit])
 :step: (int) (default: 1)
 
-As you can see, if you don't need slicing, you can just use the simple spec.
+As you can see, if you don't need to slice or override the "single" aspect,
+you can just use the simple spec.
 
 And here's an example from a calculator:
 
@@ -144,10 +154,15 @@ Actually Making the Grammar
 
 .. code-block:: python
 
-    grammar = Grammar(start, tokens, indent=False, ignore=[], ast_tokens=[])
+    grammar = Grammar(start, tokens, idchars='', indent=False, ignore=[], ast_tokens=[])
 
 :start: the start rule
 :tokens: a list of tokens to use
+:idchars:
+
+    extra characters you want to be considered ID-like (e.g. '$' for
+    javascript, PHP)
+
 :indent:
 
     (bool) if true, insert INDENT and DEDENT tokens in the appropriate places
@@ -215,7 +230,7 @@ Within a ``deal_with_baz`` function, if you want to translate a child node,
 call ``t.translate(node.somechild, scope)`` -- it will deal with that node in
 the way you'd expect.
 
-The ``scope`` variable that you saw me passing around is an anonymous object
+The ``scope`` variable that you saw me passing around is an object
 that is really useful if you need to maintain any kind of state while
 translating (local variables, etc.). To "turn on" scope usage, pass some
 keyword arguments to the translator, which will populate the default
@@ -229,7 +244,7 @@ The ``scope`` object that gets passed around will then have the attributes
 "variables" and "call_stack". For a good example of using the translation
 scope, look at `CleverCSS2 <http://jaredforsyth.com/projects/clevercss2/>`_.
 *If you don't "turn on" the scope, it doesn't get passed around -- your
-translator functions should only take one argument.*
+translating functions should only take one argument.*
 
 Once you've populated your translator, you can call ``t.from_string(text)`` to
 first turn the ``text`` into an AST, and then translate the AST.