fixed a bug in error reporting and some small documentation updates

--HG--
branch : trunk

docs/api.rst

@@ -15,7 +15,8 @@ Jinja2 uses a central object called the template :class:`Environment`.
 Instances of this class are used to store the configuration, global objects
 and are used to load templates from the file system or other locations.
 Even if you are creating templates from string by using the constructor of
-:class:`Template` class, an environment is created automatically for you.
+:class:`Template` class, an environment is created automatically for you,
+albeit a shared one.
 
 Most applications will create one :class:`Environment` object on application
 initialization and use that to load templates.  In some cases it's however
@@ -43,6 +44,10 @@ To render it with some variables, just call the :meth:`render` method::
 
     print template.render(the='variables', go='here')
 
+Using a template loader rather then passing strings to :class:`Template`
+or :meth:`Environment.from_string` has multiple advantages.  Besides being
+a lot easier to use it also enables template inheritance.
+
 
 Unicode
 -------
@@ -52,6 +57,40 @@ objects to the render function or bytestrings that only consist of ASCII
 characters.  Additionally newlines are normalized to one end of line
 sequence which is per default UNIX style (``\n``).
 
+Python 2.x supports two ways of representing string objects.  One is the
+`str` type and the other is the `unicode` type, both of which extend a type
+called `basestring`.  Unfortunately the default is `str` which should not
+be used to store text based information unless only ASCII characters are
+used.  With Python 2.6 it is possible to my `unicode` the default on a per
+module level and with Python 3 it will be the default.
+
+To explicitly use a unicode string you have to prefix the string literal
+with a `u`: ``u'Hänsel und Gretel sagen Hallo'``.  That way Python will
+store the string as unicode by decoding the string with the character
+encoding from the current Python module.  If no encoding is specified this
+defaults to 'ASCII' which means that you can't use any non ASCII identifier.
+
+To set a better module encoding add the following comment to the first or
+second line of the Python module using the unicode literal::
+
+    # -*- coding: utf-8 -*-
+
+We recommend utf-8 as Encoding for Python modules and templates as it's
+possible to represent every Unicode character in utf-8 and because it's
+backwards compatible to ASCII.  For Jinja2 the default encoding of templates
+is assumed to be utf-8.
+
+It is not possible to use Jinja2 to process non unicode data.  The reason
+for this is that Jinja2 uses Unicode already on the language level.  For
+example Jinja2 treats the non-breaking space as valid whitespace inside
+expressions which requires knowledge of the encoding or operating on an
+unicode string.
+
+For more details about unicode in Python have a look at the excellent
+`Unicode documentation`_.
+
+
+.. _Unicode documentation: http://docs.python.org/dev/howto/unicode.html
 
 High Level API
 --------------
@@ -448,7 +487,8 @@ Low Level API
 
 The low level API exposes functionality that can be useful to understand some
 implementation details, debugging purposes or advanced :ref:`extension
-<jinja-extensions>` techniques.
+<jinja-extensions>` techniques.  Unless you know exactly what you are doing we
+don't recommend using any of those.
 
 .. automethod:: Environment.lex
 

docs/faq.rst

@@ -21,10 +21,10 @@ benchmark different engines in different situations.  The benchmarks from the
 testsuite show that Jinja2 has a similar performance to `Mako`_ and is between
 10 and 20 times faster than Django's template engine or Genshi.  These numbers
 should be taken with tons of salt as the benchmarks that took these numbers
-only test a few performance related situations such as looping.  They are
-not a good indicator for the templates used in the average application.
-Additionally you should keep in mind that for most web applications
-templates are clearly not the bottleneck.
+only test a few performance related situations such as looping.  Generally
+speaking the performance of a template engine doesn't matter much as the
+usual bottleneck in a web application is either the database or the application
+code.
 
 .. _Mako: http://www.makotemplates.org/
 

docs/intro.rst

@@ -34,7 +34,8 @@ Prerequisites
 
 Jinja2 needs at least **Python 2.4** to run.  Additionally a working C-compiler
 that can create python extensions should be installed for the debugger.  If no
-C-compiler is available the `ctypes`_ module should be installed.
+C-compiler is available and you are using Python 2.4 the `ctypes`_ module
+should be installed.
 
 .. _ctypes: http://python.net/crew/theller/ctypes/
 
@@ -97,8 +98,9 @@ Basic API Usage
 This section gives you a brief introduction to the Python API for Jinja2 templates.
 
 The most basic way to create a template and render it is through
-:class:`Template`.  This however is not the recommended way to work with it,
-but the easiest
+:class:`Template`.  This however is not the recommended way to work with it if
+your templates are not loaded from strings but the file system or another data
+source:
 
 >>> from jinja2 import Template
 >>> template = Template('Hello {{ name }}!')
@@ -110,3 +112,7 @@ object that provides a method called :meth:`~Template.render` which when
 called with a dict or keyword arguments expands the template.  The dict
 or keywords arguments passed to the template are the so-called "context"
 of the template.
+
+What you can see here is that Jinja2 is using unicode internally and the
+return value is an unicode string.  So make sure that your application is
+indeed using unicode internally.

docs/templates.rst

@@ -149,10 +149,11 @@ a list of numbers from ``1`` to ``9`` the output would be ``123456789``.
 
 Note that you must not use a whitespace between the tag and the minus sign:
 
-    valid:
+    valid::
+
         {%- if foo -%}...{% endif %}
 
-    invalid:
+    invalid::
 
         {% - if foo - %}...{% endif %}
 
@@ -1032,8 +1033,8 @@ For multiple placeholders always use keyword arguments to `format` as other
 languages may not use the words in the same order.
 
 
-do
-~~
+Expression Statement
+~~~~~~~~~~~~~~~~~~~~
 
 If the expression-statement extension is loaded a tag called `do` is available
 that works exactly like the regular variable expression (``{{ ... }}``) just

jinja2/loaders.py

@@ -255,7 +255,7 @@ class ChoiceLoader(BaseLoader):
     >>> loader = ChoiceLoader([
     ...     FileSystemLoader('/path/to/user/templates'),
     ...     PackageLoader('myapplication')
-    .. ])
+    ... ])
 
     This is useful if you want to allow users to override builtin templates
     from a different location.

jinja2/parser.py

@@ -29,7 +29,7 @@ def __init__(self, environment, source, name=None, filename=None):
         self.name = name
         self.filename = filename
         self.closed = False
-        self.stream = environment.lexer.tokenize(self.source, filename)
+        self.stream = environment.lexer.tokenize(self.source, name, filename)
         self.extensions = {}
         for extension in environment.extensions.itervalues():
             for tag in extension.tags:
@@ -43,7 +43,7 @@ def fail(self, msg, lineno=None, exc=TemplateSyntaxError):
         """
         if lineno is None:
             lineno = self.stream.current.lineno
-        raise TemplateSyntaxError(msg, lineno, self.name, self.filename)
+        raise exc(msg, lineno, self.name, self.filename)
 
     def is_tuple_end(self, extra_end_rules=None):
         """Are we at the end of a tuple?"""

jinja2/runtime.py

@@ -9,7 +9,6 @@
     :license: GNU GPL.
 """
 import sys
-from types import FunctionType
 from itertools import chain, imap
 from jinja2.utils import Markup, partial, soft_unicode, escape, missing, concat
 from jinja2.exceptions import UndefinedError, TemplateRuntimeError