Complete documentation in full color.
pipetools
is a python package that enables function composition similar to
using Unix pipes.
Inspired by Pipe and Околомонадное (whatever that means...)
It allows piping of arbitrary functions and comes with a few handy shortcuts.
Source is on github.
Pipetools attempt to simplify function composition and make it more readable.
I believe it to be easier to read, write and think about from left to right /
top to bottom in the order that it's actually executed, instead of reversed
order as it is with regular function composition ((f • g)(x) == f(g(x))
).
Say you want to create a list of python files in a given directory, ordered by filename length, as a string, each file on one line and also with line numbers:
>>> print pyfiles_by_length('../pipetools') 0. main.py 1. utils.py 2. __init__.py 3. ds_builder.py
So you might write it like this:
def pyfiles_by_length(directory): all_files = os.listdir(directory) py_files = [f for f in all_files if f.endswith('.py')] py_files.sort(key=len) numbered = enumerate(py_files) rows = ("{0}. {1}".format(i, f) for i, f in numbered) return '\n'.join(rows)
Or perhaps like this:
def pyfiles_by_length(directory): return '\n'.join('{0}. {1}'.format(*x) for x in enumerate(sorted( [f for f in os.listdir(directory) if f.endswith('.py')], key=len)))
Or, if you're a mad scientist, you would probably do it like this:
pyfiles_by_length = lambda d: (reduce('{0}\n{1}'.format, map(lambda x: '%d. %s' % x, enumerate(sorted( filter(lambda f: f.endswith('.py'), os.listdir(d)), key=len)))))
But there should be one -- and preferably only one -- obvious way to do it.
So which one is it? Well, to redeem the situation, pipetools
give you yet
another possibility!
pyfiles_by_length = (pipe | os.listdir | where(X.endswith('.py')) | sort_by(len) | enumerate | foreach("{0}. {1}") | '\n'.join )
So is this The Right Way™? Probably not, but I think it's pretty cool, so you should give it a try! Read on to see how it works.
$ pip install pipetools
The pipe
object can be used to pipe functions together to
form new functions, and it works like this:
from pipetools import pipe f = pipe | a | b | c f(x) == c(b(a(x)))
A real example, sum of odd numbers from 0 to x:
from functools import partial from itertools import ifilter from pipetools import pipe odd_sum = pipe | xrange | partial(ifilter, lambda x: x % 2) | sum odd_sum(10) # -> 25
Note that the chain up to the sum is lazy.
As partial application is often useful when piping things together, it is done automatically when the pipe encounters a tuple, so this produces the same result as the previous example:
odd_sum = pipe | xrange | (ifilter, lambda x: x % 2) | sum
As of 0.1.9
, this is even more powerful, see X-curry.
Pipetools contain a set of pipe-utils that solve some common tasks. For example there is a shortcut for the ifilter from our example, called where():
from pipetools import pipe, where odd_sum = pipe | xrange | where(lambda x: x % 2) | sum
Well that might be a bit more readable, but not really a huge improvement, but wait!
If a pipe-util is used as first or second item in the pipe (which happens
quite often) the pipe
at the beginning can be omitted:
odd_sum = xrange | where(lambda x: x % 2) | sum
See pipe-utils' documentation.
where(), but also foreach(),
sort_by() and other pipe-utils can be
quite useful, but require a function as an argument, which can either be a named
function -- which is OK if it does something complicated -- but often it's
something simple, so it's appropriate to use a lambda
. Except Python's
lambdas are quite verbose for simple tasks and the code gets cluttered...
X object to the rescue!
from pipetools import where, X odd_sum = xrange | where(X % 2) | sum
How 'bout that.
Read more about the X object and it's limitations.
Since it doesn't make sense to compose functions with strings, when a pipe (or a pipe-util) encounters a string, it attempts to use it for (advanced) formatting:
>>> countdown = pipe | (xrange, 1) | reversed | foreach('{0}...') | ' '.join | '{0} boom' >>> countdown(5) u'4... 3... 2... 1... boom'
Sometimes it's useful to create a one-off pipe and immediately run some input through it. And since this is somewhat awkward (and not very readable, especially when the pipe spans multiple lines):
result = (pipe | foo | bar | boo)(some_input)
It can also be done using the >
operator:
result = some_input > pipe | foo | bar | boo
Which also isn't ideal, but I couldn't think of anything better so far...
See the full documentation.