Background: This is the result of a discussion between DagSverreSeljebotn and MartinCMartin; see here. Long-term it would be an approach to unify CEP 511 (macro meta-programming) with CEP 508 (compile-time unrolling meta-programming). (The latter is a bit poorly explained in the wiki, however if you are really interested read the emails linked to for a better explanation.)
The word macro everywhere refers to the LISP meaning (not the much less ambitious C meaning, which is basically covered by inline functions).
In the below I'll assume
#!python from cython.meta import uneval, codenode, PLUS, EXP, VALUE, VAR, CALL, MULTIPLY
It is a compile-time code introspection tool. Basically, it takes an expression and returns a tree representing the code that computes the expression. The tree can then be modified, and eval() called on it.
This kind of behaviour cannot happen at any time though. For now the suggestion is to only allow use within macros as described in the next section, however limited support could be added for use within functions as well. For now, I'll use the cython.macro keyword to specify a macro.
For demonstration purposes only, consider however the following code-snippet (while completely ignoring feasability, which is the subject of the next section!):
#!python >>> a = 23 >>> b = a ** global_var - sin(23) >>> tree = uneval(b) >>> print repr(tree) codenode(name="b", op=PLUS, args=[ codenode(name=None, op=EXP, args=[ codenode(name="a", op=VALUE, args=), codenode(name="global_var", op=VAR, args=) ]), codenode(name=None, op=CALL, args=[sin, codenode(name=None, op=VALUE, args=) ]) ])
Notice that a and b has their name attribute set, while "temporaries" in the calculation has no name.
What should happen if the value is determined by if-tests, for-loops etc. is a tough question that's not dealt with now. However, note that at any time the tree can be "cut short" and provide a "magic black box" for some values (ie like global_var above). So either the code tree is extended to follow the program flow back through if-tests and for-loops, or it is cut off and only contain the resulting calculations as op=VAR.
The code tree should overload str in order to provide nice formatting.
#!python >>> #...continuing... >>> print str(tree) a = 23 b = a ** global_var - sin(23) b
(Or "return b"...not sure...) Note that this is a "reformatting" of the tree so might not be identical with the input code snippet.
After the desired modifications etc. are made to the tree, it can be injected back into the Python code by using the eval() function.
#!python >>> #...continuing... >>> print tree.eval() # same result as "print b"
On eval(), side-effects will be fired again up to the first named node in the tree. Ie:
#!python >>> def a(x): ... print "Hello", x ... return 1 ... >>> >>> b = a(1) Hello 1 >>> tree = uneval(a(2) + a(3) + b + b) >>> # Notice that the parameter to uneval is not calculated yet, no side-effects >>> print tree.eval() Hello 2 Hello 3 4 >>> print tree.eval() Hello 2 Hello 3 4
Note: In macros below, it is actually wanted to have side-effects run one level further down, ie basically ignore that the root node is named. This must be solved somehow (a parameter "param=True" in the root would do, but perhaps one can do better).
Of course, this functionality cannot be used in any run-time code. All calls to code-trees must "disappear" at compile-time. That's where CEP 508 enters. In CEP 508 the decorator @cython.unroll is used, but below I'll use @cython.macro.
With the addition of uneval, CEP 508 can provide much macro behaviour. The main difference to CEP 511 is that the calling convention is similar to Python (the expressions themselves go in and out) so one can keep the def keyword.
#!python @cython.macro def deriv(x, var): node = uneval(x) if node.op == ADD: node.args = [deriv(x) for x in node.args] elif node.op = MULTIPLY: node.op = ADD node.args = [ codenode(MULTIPLY, [node.args, deriv(node.args)]), codenode(MULTIPLY, [deriv(node.args), node.args]) ] elif node.name == var: return codenode(VALUE, ) else: return codenode(VALUE, ) # node now represents the derivated expression # return it _by value_ to the caller return node.eval()
Note that the above would remove any side-effects coming from expressions that are derivated away, ie:
#!python deriv(3*x*x + sin(34), "x")
Here, sin is never even called. In order to always have side-effects run, one can insert an extra node.eval() before modifying the tree (in the case above, one would insert "node.eval()" lines before returning 1 and 0, in order to fire off side-effects but discard the results, since that is when nodes are dropped).
This gets much more complicated as one starts adding expressions that can only be known run-time into the macros. For the full version, please read the mentioned emails first for much more discusssion about this.
At least it is clear that if, for instance, instead of "x" above a string that is only known run-time is passed, a compile-time error must be raised.