Rendering the FST
This section is quite advanced and you will maybe never need to use what is in here. But if you want to process the whole rendered fst or part of it as a chunk, please read along since several helpers are provided.
Understanding core rendering
Baron renders the FST back into source code by following the instructions given by the :file:`nodes_rendering_order` dictionary. It gives, for every FST node, the order in which the node components must be rendered and the nature of those components.
.. ipython:: python from baron import nodes_rendering_order, parse from baron.helpers import show_node nodes_rendering_order["name"] show_node(parse("a_name")) nodes_rendering_order["tuple"] show_node(parse("(a_name,another_name,yet_another_name)")) nodes_rendering_order["comma"]
For a "name" node, it is a list containing a unique component stored in a tuple but it can contain multiple ones like for a "tuple" node.
To render a node, you just need to render each element of the list, one by one, in the given order. As you can see, they are all formatted as a 3-tuple. The first column is the type which is one of the following:
.. ipython:: python from baron.render import node_types node_types
With the exception of the "constant" node, the second column contains the key of the FST node which must be rendered. The first column explains how that key must be rendered. We'll see the third column later.
- A :file:`node` node is one of the nodes in the :file:`nodes_rendering_order` we just introduced, it is rendered by following the rules mentionned here. This is indeed a recursive definition.
- A :file:`key` node is a branch of the tree that contains another node (a python dictionary).
- A :file:`string` node is a leaf of the tree that contains a variable value, like the name of a function. former case, it is rendered by rendering its content.
- A :file:`list` node is like a :file:`key` node but can contain 0, 1 or several other nodes stored in a python list. For example, Baron root node is a :file:`list` node since a python program is a list of statements. It is rendered by rendering each of its elements in order.
- A :file:`formatting` node is similar in behaviour to a :file:`list` node but contains only formatting nodes. This is basically where Baron distinguish itself from other ASTs.
- A :file:`constant` node is a leaf of the FST tree. The second column always contain a string which is outputted directly. Compared to a :file:`string` node, the :file:`constant` node is identical for every instance of the nodes (e.g. the left parenthesis character :file:`(` in a function call node or the :file:`def` keyword of a function definition) while the :file:`string` node's value can change (e.g. the name of the function in a function definition node).
- A :file:`bool` node is a node used exclusively for conditional rendering. It's exact use will be explained later on with the tuple's third column but the main point for now is to know that they are never rendered.
Let's see all this is in action by rendering a "lambda" node. First, the root node is always a "list" node and since we are only parsing one statement, the root node contains our "lambda" node at index 0:
.. ipython:: python fst = parse("lambda x, y = 1: x + y") fst["type"]
For reference, you can find the (long) FST produced by the lambda node at the end of this section.
Now, let's see how to render a "lambda" node:
.. ipython:: python nodes_rendering_order["lambda"]
Okay, first the string constant "lambda", then a first_formatting node which represents the space between the string "lambda" and the variable "x".
.. ipython:: python fst["first_formatting"]
The "first_formatting" contains a list whose unique element is a "space" node.
.. ipython:: python fst["first_formatting"] nodes_rendering_order["space"]
Which in turn is rendered by printing the value of the string of the space node.
.. ipython:: python fst["first_formatting"]["value"]
So far we have outputted "lambda ". Tedious but exhaustive.
We have exhausted the "first_formatting" node so we go back up the tree. Next is the "list" node representing the arguments:
.. ipython:: python fst["arguments"]
Rendering a "list" node is done one element at a time. First a "def_argument", then a "comma" and again a "def_argument".
.. ipython:: python fst["arguments"] nodes_rendering_order["def_argument"]
The first "def_argument" is rendered by first outputting the content of a name "string" node:
.. ipython:: python fst["arguments"]["name"]
Now, we have outputted "lambda x". At first glance we could say we should render the second element of the "def_argument" node but as we'll see in the next section, it is not the case because of the third column of the tuple.
For reference, the FST of the lambda node:
.. ipython:: python show_node(fst)
Sometimes, some node elements must not be outputted. In our "def_argument" example, all but the first are conditional. They are only rendered if the FST's "value" node exists and is not empty. Let's compare the two "def_arguments" FST nodes:
.. ipython:: python fst["arguments"] fst["arguments"] nodes_rendering_order[fst["arguments"]["type"]]
The "value" is empty for the former "def_argument" but not for the latter because it has a default value of "= 1".
.. ipython:: python from baron import dumps dumps(fst["arguments"]) dumps(fst["arguments"])
The rule here is that the third column of a node is one of:
- True, it is always rendered;
- False, it is never rendered;
- A string, it is rendered conditionnally. It is not rendered if the key it references is either empty or False. It also must reference an existing key. In our example above, it references the existing "value" key which is empty in the first case and not empty in the second.
This is how "bool" nodes are never outputted: their third column is always False.
We will conclude here now that we have seen an example of every aspect of FST rendering. Understanding everything is not required to use Baron since several helpers like :file:`render`, :file:`RenderWalker` or :file:`dumps` handle all the complexity under the hood.
Baron provides a render function helper which walks recursively the :file:`nodes_rendering_order` dictionnary for you:
.. autofunction:: baron.render.render
But even easier, Baron provides a walker class whose job is to walk the fst while rendering it and to call user-provided callbacks at each step:
.. autoclass:: baron.render.RenderWalker
from baron.render import RenderWalker def dumps(tree): return Dumper().dump(tree) class Dumper(RenderWalker): def before_constant(self, constant, key): self.dump += constant def before_string(self, string, key): self.dump += string def dump(self, tree): self.dump = '' self.walk(tree) return self.dump
If while walking you need to know the current path of the node, then you should subclass :file:`PathWalker` instead:
.. autoclass:: baron.path.PathWalker
Here is a succint example of what you should expect when using the :file:`PathWalker`:
.. ipython:: python from baron.path import PathWalker fst = parse("a = 1") class PathWalkerPrinter(PathWalker): def before(self, key_type, item, render_key): super(PathWalkerPrinter, self).before(key_type, item, render_key) print(self.current_path) def after(self, key_type, item, render_key): print(self.current_path) super(PathWalkerPrinter, self).after(key_type, item, render_key) walker = PathWalkerPrinter() walker.walk(fst)
Like in the example, don't forget to call the before and after methods of the parent class. Furthermore, you need to respect the order specified above, that is: