Skip to content

Commit

Permalink
Added de-indexing, improved pragma module with bugfixes.
Browse files Browse the repository at this point in the history
  • Loading branch information
scnerd committed Nov 20, 2017
1 parent 09b2284 commit 6d3e93b
Show file tree
Hide file tree
Showing 9 changed files with 544 additions and 209 deletions.
2 changes: 1 addition & 1 deletion docs/source/caching.rst
@@ -1,5 +1,5 @@
Property Cache
++++++++++++++
==============

In some cases, an object has properties that don't need to be computed until necessary, and once computed are generally static and could just be cached. This could be accomplished using the following simple recipe::

Expand Down
7 changes: 4 additions & 3 deletions docs/source/index.rst
Expand Up @@ -23,12 +23,13 @@ Welcome to miniutils's documentation!
progress_bar
caching
python2
pragma
misc
api


Overview
--------
++++++++

This module provides numerous helper utilities for Python3.X code to add functionality with minimal code footprint. It has tools for the following tasks:

Expand All @@ -38,7 +39,7 @@ This module provides numerous helper utilities for Python3.X code to add functio
- More intuitive contract decorator (leveraging ``pycontracts``)

Installation
------------
++++++++++++

As usual, you can install the latest code version directly from Github::

Expand All @@ -49,7 +50,7 @@ Or you can ``pip`` install the latest release from PyPi::
pip install miniutils

Examples
--------
++++++++

To get started, you can import your desired utilities directly from ``miniutils``. For example, to use the ``CachedProperty`` decorator::

Expand Down
154 changes: 6 additions & 148 deletions docs/source/misc.rst
@@ -1,8 +1,8 @@
Miscellaneous
+++++++++++++
=============

Code Contracts
==============
++++++++++++++

Code contracting seems like a great way to define and document your code's expected behavior, easily integrate bounds checking, and just generally write code that tries to avoid bugs. The `pycontracts <https://andreacensi.github.io/contracts/>`_ package provides this capability within python, but as soon as I started using it I realized that it was meant primarily to be robust, not concise. For example, consider the following code::

Expand Down Expand Up @@ -51,7 +51,7 @@ And now the function works like you'd expect. If you want to do something more c
.. autofunction:: miniutils.magic_contract.magic_contract

Simplifying Decorators
======================
++++++++++++++++++++++

When writing a decorator that could be used like ``@deco`` or ``@deco()``, there's a little code I've found necessary in order to make both cases function identically. I've isolated this code into another decorator (meta-decorator?) to keep my other decorators simple (since, let's be honest, decorators are usually convoluted enough as is).

Expand Down Expand Up @@ -115,7 +115,7 @@ This makes sense, but is somewhat annoying when parameters aren't required, such
.. autofunction:: miniutils.opt_decorator.optional_argument_decorator

Logging Made Easy
=================
+++++++++++++++++

The standard ``logging`` module provides a lot of great functionality, but there are a few simplifications missing:

Expand Down Expand Up @@ -158,7 +158,7 @@ The ``coloredlogs`` module didn't quite work as expected when I tried to use it.


Timing
======
++++++

Simple ``printf``-like timing utilities when proper profiling won't quite work.

Expand Down Expand Up @@ -212,146 +212,4 @@ Use ``tic``/``toc`` to time and report the run times of different chunks of code

This utility is just less verbose than tracking various times yourself. The output is printed to the log for later review. It can also accept a custom print format string, including information about the code calling ``toc()`` and runtimes since the last ``tic``/``toc``.

.. autofunction:: miniutils.timing.tic

Pragma
======

When Python code is being executed abnormally, or being replaced entirely (e.g., by ``numba.jit``), it's sometimes highly relevant how your code is written. However, writing it that way isn't always practical, or you might want the code itself to be dependant on runtime data. In these cases, basic code templating or modification can be useful. This sub-module provides some simple utilities to perform Python code modification at runtime, similar to compiler directives in C.

Unroll
------

Unroll constant loops. If the `for`-loop iterator is a known value at function definition time, then replace it with its body duplicated for each value. For example::

def f():
for i in [1, 2, 4]:
yield i

could be identically replaced by::

def f():
yield 1
yield 2
yield 4

The ``unroll`` decorator accomplishes this by parsing the input function, performing the unrolling transformation on the function's AST, then compiling and returning the defined function.

If using a transformational decorator of some sort, such as ``numba.jit`` or ``tangent.grad``, if that function isn't yet able to unwrap loops like this, then using this function might yield cleaner results on constant-length loops.

``unroll`` is currently smart enough to notice singly-defined variables and literals, as well as able to unroll the ``range`` function and unroll nested loops::

@pragma.unroll
def summation(x=0):
a = [x, x, x]
v = 0
for _a in a:
v += _a
return v

# ... Becomes ...

def summation(x=0):
a = [x, x, x]
v = 0
v += x
v += x
v += x
return v

# ... But ...

@pragma.unroll
def f():
x = 3
for i in [x, x, x]:
yield i
x = 4
a = [x, x, x]
for i in a:
yield i

# ... Becomes ...

def f():
x = 3
yield 3
yield 3
yield 3
x = 4
a = [x, x, x]
yield 4
yield 4
yield 4

# Even nested loops and ranges work!

@pragma.unroll
def f():
for i in range(3):
for j in range(3):
yield i + j

# ... Becomes ...

def f():
yield 0 + 0
yield 0 + 1
yield 0 + 2
yield 1 + 0
yield 1 + 1
yield 1 + 2
yield 2 + 0
yield 2 + 1
yield 2 + 2

You can also request to get the function source code instead of the compiled callable by using ``return_source=True``::

In [1]: @pragma.unroll(return_source=True)
...: def f():
...: for i in range(3):
...: print(i)
...:

In [2]: print(f)
def f():
print(0)
print(1)
print(2)

It also supports limited recognition of externally and internally defined values::

@pragma.unroll(a=range)
def f():
for b in a(3):
print(b)

# Is equivalent to:

a = range
@pragma.unroll
def f():
for b in a(3):
print(b)

# Both of which become:

def f():
print(0)
print(1)
print(2)

Currently not-yet-supported features include:

- Handling constant sets and dictionaries (since the values contained in the AST's, not the AST nodes themselves, must be uniquely identified)
- Tuple assignments (``a, b = 3, 4``)
- ``zip``, ``reversed``, and other known operators, when performed on definition-time constant iterables

.. autofunction:: miniutils.pragma.unroll

Collapse Literals
-----------------

Collapse literal operations in code to their results, e.g. ``x = 1 + 2`` gets converted to ``x = 3``.

.. autofunction:: miniutils.pragma.collapse_literals
.. autofunction:: miniutils.timing.tic

0 comments on commit 6d3e93b

Please sign in to comment.