made a big refactoring of Twig internals (and added a bunch of unit t…

…ests and phpdoc, fixes twigphp#53)

CHANGELOG

@@ -1,8 +1,15 @@
 * 0.9.7-DEV
 
 Backward incompatibilities:
- * The short notation of the `block` tag changed.
-
+ * added a 'as' string to the block tag short notation ({% block title "Title" %} must now be {% block title as "Title" %})
+ * removed the sandboxed attribute of the include tag (use the new sandbox tag instead)
+ * refactored the Node system (if you have custom nodes, you will have to update them to use the new API)
+
+ * removed the Twig_Resource::resolveMissingFilter() method
+ * fixed the filter tag which did not apply filtering to included files
+ * added a bunch of unit tests
+ * added a bunch of phpdoc
+ * added a sandbox tag in the sandbox extension
  * fixed iterator_to_array() usage
  * changed the date filter to support any date format supported by DateTime
  * added strict_variable setting to throw an exception when an invalid variable is used in a template (disabled by default when debug is false)
@@ -12,7 +19,6 @@ Backward incompatibilities:
  * added three interfaces: Twig_NodeInterface, Twig_TokenParserInterface, and Twig_FilterInterface
  * changed the generated code to match the new coding standards
  * fixed sandbox mode (__toString() method check was not enforced if called implicitly from a simple statement like {{ article }})
- * added a 'as' string to the block tag short notation ({% block title "Title" %} must now be {% block title as "Title" %})
  * added an exception when a child template has a non-empty body (as it is always ignored when rendering)
 
 * 0.9.6 (2010-05-12)

doc/02-Twig-for-Template-Designers.markdown

@@ -641,12 +641,6 @@ rendered contents of that file into the current namespace:
 
 Included templates have access to the variables of the active context.
 
-An included file can be evaluated in the sandbox environment by appending
-`sandboxed` at the end if the `escaper` extension has been enabled:
-
-    [twig]
-    {% include 'user.html' sandboxed %}
-
 You can also restrict the variables passed to the template by explicitly pass
 them as an array:
 
@@ -656,16 +650,15 @@ them as an array:
     {% set vars as ['foo': 'bar'] %}
     {% include 'foo' with vars %}
 
-The most secure way to include a template is to use both the `sandboxed` mode,
-and to pass the minimum amount of variables needed for the template to be
-rendered correctly:
-
-    [twig]
-    {% include 'foo' sandboxed with vars %}
-
 >**NOTE**
 >The `with` keyword is supported as of Twig 0.9.5.
 
+-
+
+>**TIP**
+>When including a template created by an end user, you should consider
+>sandboxing it. More information in the "Twig for Developers" chapter.
+
 ### Import
 
 Twig supports putting often used code into macros. These macros can go into

doc/03-Twig-for-Developers.markdown

@@ -389,10 +389,12 @@ The policy object is the first argument of the sandbox constructor:
     $twig->addExtension($sandbox);
 
 By default, the sandbox mode is disabled and should be enabled when including
-untrusted templates:
+untrusted template code by using the `sandbox` tag:
 
-    [php]
-    {% include "user.html" sandboxed %}
+    [twig]
+    {% sandbox %}
+      {% include 'user.html' %}
+    {% endsandbox %}
 
 You can sandbox all templates by passing `true` as the second argument of the
 extension constructor:

doc/04-Extending-Twig.markdown

@@ -481,22 +481,16 @@ The `Project_Set_Node` class itself is rather simple:
     [php]
     class Project_Set_Node extends Twig_Node
     {
-      protected $name;
-      protected $value;
-
       public function __construct($name, Twig_Node_Expression $value, $lineno)
       {
-        parent::__construct($lineno);
-
-        $this->name = $name;
-        $this->value = $value;
+        parent::__construct(array('value' => $value), array('name' => $name), $lineno);
       }
 
       public function compile($compiler)
       {
         $compiler
           ->addDebugInfo($this)
-          ->write('$context[\''.$this->name.'\'] = ')
+          ->write('$context[\''.$this['name'].'\'] = ')
           ->subcompile($this->value)
           ->raw(";\n")
         ;
@@ -533,7 +527,7 @@ developer generate beautiful and readable PHP code:
  * `outdent()`: Outdents the generated code (see `Twig_Node_Block` for a usage
    example).
 
-Creating a Node Transformer
----------------------------
+Creating a Node Visitor
+-----------------------
 
 To be written...

lib/Twig/Autoloader.php

@@ -37,11 +37,9 @@ static public function register()
     static public function autoload($class)
     {
         if (0 !== strpos($class, 'Twig')) {
-            return false;
+            return;
         }
 
         require dirname(__FILE__).'/../'.str_replace('_', '/', $class).'.php';
-
-        return true;
     }
 }

lib/Twig/Compiler.php

@@ -41,6 +41,16 @@ public function setEnvironment(Twig_Environment $env)
         $this->env = $env;
     }
 
+    /**
+     * Returns the environment instance related to this compiler.
+     *
+     * @return Twig_Environment The environment instance
+     */
+    public function getEnvironment()
+    {
+        return $this->env;
+    }
+
     /**
      * Gets the current PHP code after compilation.
      *
@@ -211,19 +221,4 @@ public function outdent($step = 1)
 
         return $this;
     }
-
-    /**
-     * Returns the environment instance related to this compiler.
-     *
-     * @return Twig_Environment The environment instance
-     */
-    public function getEnvironment()
-    {
-        return $this->env;
-    }
-
-    public function getTemplateClass($name)
-    {
-        return $this->getEnvironment()->getTemplateClass($name);
-    }
 }

lib/Twig/Environment.php

@@ -178,13 +178,14 @@ public function getTemplateClass($name)
     /**
      * Loads a template by name.
      *
-     * @param  string $name The template name
+     * @param  string  $name  The template name
+     * @param  Boolean $macro Whether to return the macro object if any, or the template one
      *
      * @return Twig_TemplateInterface A template instance representing the given template name
      */
-    public function loadTemplate($name)
+    public function loadTemplate($name, $macro = false)
     {
-        $cls = $this->getTemplateClass($name);
+        $cls = $this->getTemplateClass($name).($macro ? '_Macro' : '');
 
         if (isset($this->loadedTemplates[$cls])) {
             return $this->loadedTemplates[$cls];
@@ -309,6 +310,10 @@ public function hasExtension($name)
 
     public function getExtension($name)
     {
+        if (!isset($this->extensions[$name])) {
+            throw new LogicException(sprintf('The "%s" extension is not enabled.', $name));
+        }
+
         return $this->extensions[$name];
     }
 

lib/Twig/ExpressionParser.php

@@ -86,14 +86,15 @@ public function parseCompareExpression()
             ||
             $this->parser->getStream()->test(Twig_Token::NAME_TYPE, 'in')
         ) {
-            $ops[] = array($this->parser->getStream()->next()->getValue(), $this->parseAddExpression());
+            $ops[] = new Twig_Node_Expression_Constant($this->parser->getStream()->next()->getValue(), $lineno);
+            $ops[] = $this->parseAddExpression();
         }
 
         if (empty($ops)) {
             return $expr;
         }
 
-        return new Twig_Node_Expression_Compare($expr, $ops, $lineno);
+        return new Twig_Node_Expression_Compare($expr, new Twig_Node($ops), $lineno);
     }
 
     public function parseAddExpression()
@@ -279,6 +280,7 @@ public function parsePrimaryExpression($assignment = false)
                     throw new Twig_SyntaxError(sprintf('Unexpected token "%s" of value "%s"', Twig_Token::getTypeAsString($token->getType()), $token->getValue()), $token->getLine());
                 }
         }
+
         if (!$assignment) {
             $node = $this->parsePostfixExpression($node);
         }
@@ -358,14 +360,16 @@ public function parseRangeExpression($node)
 
         $end = $this->parseExpression();
 
-        return new Twig_Node_Expression_Filter($node, array(array('range', array($end))), $lineno);
+        $filters = new Twig_Node(array(new Twig_Node_Expression_Constant('range', $lineno), new Twig_Node(array($end))));
+
+        return new Twig_Node_Expression_Filter($node, $filters, $lineno);
     }
 
     public function parseSubscriptExpression($node)
     {
         $token = $this->parser->getStream()->next();
         $lineno = $token->getLine();
-        $arguments = array();
+        $arguments = new Twig_Node();
         if ($token->getValue() == '.') {
             $token = $this->parser->getStream()->next();
             if ($token->getType() == Twig_Token::NAME_TYPE || $token->getType() == Twig_Token::NUMBER_TYPE) {
@@ -398,7 +402,8 @@ public function parseFilterExpressionRaw()
         while (true) {
             $token = $this->parser->getStream()->expect(Twig_Token::NAME_TYPE);
 
-            $filters[] = array($token->getValue(), $this->parseArguments());
+            $filters[] = new Twig_Node_Expression_Constant($token->getValue(), $token->getLine());
+            $filters[] = $this->parseArguments();
 
             if (!$this->parser->getStream()->test(Twig_Token::OPERATOR_TYPE, '|')) {
                 break;
@@ -407,13 +412,13 @@ public function parseFilterExpressionRaw()
             $this->parser->getStream()->next();
         }
 
-        return $filters;
+        return new Twig_Node($filters);
     }
 
     public function parseArguments()
     {
         if (!$this->parser->getStream()->test(Twig_Token::OPERATOR_TYPE, '(')) {
-            return array();
+            return new Twig_Node();
         }
 
         $args = array();
@@ -426,14 +431,13 @@ public function parseArguments()
         }
         $this->parser->getStream()->expect(Twig_Token::OPERATOR_TYPE, ')');
 
-        return $args;
+        return new Twig_Node($args);
     }
 
     public function parseAssignmentExpression()
     {
         $lineno = $this->parser->getCurrentToken()->getLine();
         $targets = array();
-        $is_multitarget = false;
         while (true) {
             if (!empty($targets)) {
                 $this->parser->getStream()->expect(Twig_Token::OPERATOR_TYPE, ',');
@@ -449,13 +453,9 @@ public function parseAssignmentExpression()
             if (!$this->parser->getStream()->test(Twig_Token::OPERATOR_TYPE, ',')) {
                 break;
             }
-            $is_multitarget = true;
-        }
-        if (!$is_multitarget && count($targets) == 1) {
-            return array(false, $targets[0]);
         }
 
-        return array(true, $targets);
+        return new Twig_Node($targets);
     }
 
     public function parseMultitargetExpression()
@@ -479,10 +479,7 @@ public function parseMultitargetExpression()
             }
             $is_multitarget = true;
         }
-        if (!$is_multitarget && count($targets) == 1) {
-            return array(false, $targets[0]);
-        }
 
-        return array(true, $targets);
+        return array($is_multitarget, new Twig_Node($targets));
     }
 }

lib/Twig/Extension/Core.php

@@ -33,16 +33,6 @@ public function getTokenParsers()
         );
     }
 
-    /**
-     * Returns the node visitor instances to add to the existing list.
-     *
-     * @return array An array of Twig_NodeVisitorInterface instances
-     */
-    public function getNodeVisitors()
-    {
-        return array(new Twig_NodeVisitor_Filter());
-    }
-
     /**
      * Returns a list of filters to add to the existing list.
      *

lib/Twig/Extension/Sandbox.php

@@ -20,6 +20,16 @@ public function __construct(Twig_Sandbox_SecurityPolicyInterface $policy, $sandb
         $this->sandboxedGlobally = $sandboxed;
     }
 
+    /**
+     * Returns the token parser instance to add to the existing list.
+     *
+     * @return array An array of Twig_TokenParser instances
+     */
+    public function getTokenParsers()
+    {
+        return array(new Twig_TokenParser_Sandbox());
+    }
+
     /**
      * Returns the node visitor instances to add to the existing list.
      *