Merge pull request #7 from scnerd/lambda_lift_class

Lambda lift class

README.rst

@@ -59,6 +59,12 @@ Such a modification can only be done by the programmer if the dynamic features a
 This example is converted, in place and at runtime, to exactly the unrolled code above.
 
 
+Documentation
+=============
+
+Complete documentation can be found over at `RTFD <http://pypragma.readthedocs.io/en/latest/>`_.
+
+
 Installation
 ============
 
@@ -133,7 +139,7 @@ Certain keywords are reserved, of course, as will be defined in the documentatio
 
 A side effect of the proper Python syntax is that functions can have their source code retrieved by any normal Pythonic reflection:
 
-.. code-block:: ipython
+.. code-block:: python
 
    In [1]: @pragma.unroll(num_pows=3)
       ...: def pows(i):
@@ -159,9 +165,7 @@ Quick Examples
 Collapse Literals
 +++++++++++++++++
 
-:doc:`Complete documentation <collapse_literals>`:
-
-.. code-block:: ipython
+.. code-block:: python
 
    In [1]: @pragma.collapse_literals(x=5)
       ...: def f(y):
@@ -179,9 +183,7 @@ Collapse Literals
 De-index Arrays
 +++++++++++++++
 
-:doc:`Complete documentation <deindex>`:
-
-.. code-block:: ipython
+.. code-block:: python
 
    In [1]: fns = [math.sin, math.cos, math.tan]
 
@@ -208,7 +210,7 @@ De-index Arrays
 
 Note that, while it's not evident from the above printed source code, each variable ``fns_X`` is assigned to the value of ``fns[X]`` at the time when the decoration occurs:
 
-.. code-block:: ipython
+.. code-block:: python
 
    In [4]: call(0, math.pi)
    Out[4]: 1.2246467991473532e-16  # AKA, sin(pi) = 0
@@ -219,9 +221,7 @@ Note that, while it's not evident from the above printed source code, each varia
 Unroll
 ++++++
 
-:doc:`Complete documentation <unroll>`:
-
-.. code-block:: ipython
+.. code-block:: python
 
    In [1]: p_or_m = [1, -1]
 
@@ -246,9 +246,7 @@ Unroll
 Inline
 ++++++
 
-:doc:`Complete documentation <inline>`:
-
-.. code-block:: ipython
+.. code-block:: python
 
    In [1]: def sqr(x):
       ...:     return x ** 2
@@ -276,3 +274,45 @@ Inline
        _sqr_return_1 = _sqr_0.get('return', None)
        del _sqr_0
        return _sqr_return_0 + _sqr_return_1  # Substitute the returned values for the function calls
+
+Stacking Transformations
+++++++++++++++++++++++++
+
+The above examples demonstrate how to perform `pragma` transformations to a function. It should be especially noted, however, that since each transformer returns a proper Python function, they can stack seamlessly:
+
+.. code-block:: python
+
+    In [1]: def make_dynamic_caller(*fns):
+       ...:     @pragma.deindex(fns, 'fns')
+       ...:     @pragma.unroll(num_fns=len(fns))
+       ...:     def dynamic_call(i, x):
+       ...:         for j in range(num_fns):
+       ...:             if i == j:
+       ...:                 return fns[j](x)
+       ...:
+       ...:     return dynamic_call
+
+    In [2]: f = make_dynamic_caller(math.sin, math.cos, math.tan)
+
+    In [3]: f??
+    Signature: f(i, x)
+    Source:
+    def dynamic_call(i, x):
+        if i == 0:
+            return fns_0(x)
+        if i == 1:
+            return fns_1(x)
+        if i == 2:
+            return fns_2(x)
+    File:      /tmp/tmpf9tjaffi
+    Type:      function
+
+    In [4]: g = pragma.collapse_literals(i=1)(f)
+
+    In [5]: g??
+    Signature: g(i, x)
+    Source:
+    def dynamic_call(i, x):
+        return fns_1(x)
+    File:      /tmp/tmpbze5i__2
+    Type:      function

docs/inline.rst

@@ -23,16 +23,17 @@ To inline a function ``f`` into the code of another function ``g``, use ``pragma
         z = y + 3
         return f(z * 4)
 
-    # ... g Becomes ...
+    # ... g Becomes something like ...
 
     def g(y):
         z = y + 3
-        _f = {}
-        _f['x'] = z * 4
-        for ____ in [None]:
-            _f = _f['x'] ** 2
-            break
-        return _f
+        _f = dict(x=z * 4)  # Store arguments
+        for ____ in [None]:  # Function body
+            _f['return'] = _f['x'] ** 2  # Store the "return"ed value
+            break  # Return, terminate the function body
+        _f_return = _f.get('return', None)  # Retrieve the returned value
+        del _f  # Discard everything else
+        return _f_return
 
 This loop can be removed, if it's not necessary, using :func:``pragma.unroll``. This can be accomplished if there are no returns within a conditional or loop block. In this case::
 
@@ -54,4 +55,17 @@ This loop can be removed, if it's not necessary, using :func:``pragma.unroll``.
         _f = _f['x'] ** 2
         return _f
 
-Eventually, this could be collapsed using :func:``pragma.collapse_literals``, to produce simply ``return ((y + 3) * 4) ** 2``, but dictionaries aren't yet supported for collapsing.
+It needs to be noted that, besides arguments getting stored into a dictionary, other variable names remain unaltered when inlined. Thus, if there are shared variable names in the two functions, they might overwrite each other in the resulting inlined function.
+
+.. todo:: Fix name collision by name-mangling non-free variables
+
+Eventually, this could be collapsed using :func:``pragma.collapse_literals``, to produce simply ``return ((y + 3) * 4) ** 2``, but dictionaries aren't yet supported for collapsing.
+
+When inlining a generator function, the function's results are collapsed into a list, which is then returned. This will break in two main scenarios:
+
+- The generator never ends, or consumes excessive amounts of resources.
+- The calling code relies on the resulting generator being more than just iterable.
+
+In general, either this won't be an issue, or you should know better than to try to inline the infinite generator.
+
+.. todo:: Support inlining a generator into another generator by merging the functions together. E.g., ``for x in my_range(5): yield x + 2`` becomes ``i = 0; while i < 5: yield i + 2; i += 1`` (or something vaguely like that).

docs/todo.rst

@@ -1,6 +1,7 @@
 TODO List
 =========
 
+.. todo:: Replace custom stack implementation with ``collections.ChainMap``
 .. todo:: Implement decorator to eliminate unused lines of code (assignments to unused values)
 .. todo:: Technically, ``x += y`` doesn't have to be the same thing as ``x = x + y``. Handle it as its own operation of the form ``x += y; return x``
 

pragma/collapse_literals.py

@@ -1,12 +1,17 @@
 import ast
 
-from .core import TrackedContextTransformer, make_function_transformer, resolve_literal, log
+from .core import TrackedContextTransformer, make_function_transformer, primitive_ast_types
+import logging
+log = logging.getLogger(__name__)
 
 
 # noinspection PyPep8Naming
 class CollapseTransformer(TrackedContextTransformer):
     def visit_Name(self, node):
-        return self.resolve_literal(node)
+        res = self.resolve_literal(node)
+        if isinstance(res, primitive_ast_types):
+            return res
+        return node
 
     def visit_BinOp(self, node):
         return self.resolve_literal(self.generic_visit(node))
@@ -21,13 +26,17 @@ def visit_Compare(self, node):
         return self.resolve_literal(self.generic_visit(node))
 
     def visit_Subscript(self, node):
-        return self.resolve_literal(node)
+        return self.resolve_literal(self.generic_visit(node))
 
     def visit_Call(self, node):
-        return self.resolve_literal(self.generic_visit(node))
+        node = self.generic_visit(node)
+        try:
+            return self.resolve_literal(node)
+        except (AssertionError, TypeError, KeyError, IndexError):
+            return node
 
     def visit_If(self, node):
-        cond = resolve_literal(node.test, self.ctxt, True)
+        cond = self.resolve_literal(node.test, raw=True)
         # print("Attempting to collapse IF conditioned on {}".format(cond))
         if not isinstance(cond, ast.AST):
             log.debug("Collapsing if condition ({} resolved to {})".format(node.test, cond))

pragma/core/__init__.py

@@ -10,14 +10,14 @@
 
 
 def _is_iterable(x):
-    try:
-        iter(x)
-        return True
-    except Exception:
-        return False
+    return hasattr(x, '__iter__')
+
+def _is_indexable(x):
+    return hasattr(x, '__getitem__')
 
 
 safe_new_contract('iterable', _is_iterable)
+safe_new_contract('indexable', _is_indexable)
 safe_new_contract('literal', 'int|float|str|bool|tuple|list|None')
 for name, tp in inspect.getmembers(ast, inspect.isclass):
     safe_new_contract(name, tp)
@@ -37,29 +37,56 @@ def _pretty_str(o):
     if isinstance(o, ast.AST):
         if isinstance(o, ast.Name):
             return o.id
-        if isinstance(o, ast.Call):
-            return _pretty_str(o.func)
-        if isinstance(o, ast.Attribute):
+        elif isinstance(o, ast.Call):
+            return "{}(...)".format(_pretty_str(o.func))
+        elif isinstance(o, ast.Attribute):
             return "{}.{}".format(_pretty_str(o.value), o.attr)
 
         return astor.to_source(o).strip()
     else:
-        return str(o)
+        return repr(o)
+
+
+_log_call_depth = 0
 
 
 def _log_call(f):
     @functools.wraps(f)
     def inner(*args, **kwargs):
-        result = f(*args, **kwargs)
-        log.debug("{}({}) -> {}".format(
+        global _log_call_depth
+
+        result = None
+        ex = None
+        log.debug("START {}{}({})".format(
+            ' ' * _log_call_depth,
             f.__name__,
             ', '.join(
                 [_pretty_str(a) for a in args] +
                 ["{}={}".format(_pretty_str(k), _pretty_str(v)) for k, v in kwargs.items()]
-            ),
-            result
+            )
         ))
-        return result
+
+        _log_call_depth += 1
+        try:
+            result = f(*args, **kwargs)
+            return result
+
+        except Exception as e:
+            ex = e
+            raise e
+        finally:
+            _log_call_depth -= 1
+
+            log.debug("END   {}{}({}) -> {}".format(
+                ' ' * _log_call_depth,
+                f.__name__,
+                ', '.join(
+                    [_pretty_str(a) for a in args] +
+                    ["{}={}".format(_pretty_str(k), _pretty_str(v)) for k, v in kwargs.items()]
+                ),
+                _pretty_str(result)
+            ), exc_info=ex)
+
     return inner