coroutines for dataflow pipelines
Switch branches/tags
Nothing to show
Pull request Compare This branch is 1 commit behind timClicks:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


==>  COP
     coroutines for dataflow pipelines

==>  ABOUT
     Cop is designed to make it easy
     to write readable dataflow programs.
     Speed is important, but probably
     isn't there yet.

     This library is a learning exercise.
     At this stage, it should really be
     seen as a source for you to see
     how to implement your own coroutines.

     The primary motivation for cop is
     to facilitate readable and flexible
     dataflow programming. Once the 
     project is more mature, its focus
     could shift to enhancing performance.

     There are three basic terms that are
     used within cop: sources, steps and

        /sources/   Feed data to steps.
                    (cop.file, cop.web)

        /steps/     Process data, then
                    feed it to a target.

        /sinks/     Process data.

==>  USAGE
     Using cop is straight forward. Examples
     generally follow the following pattern:

       from cop.sources import source
       from cop.steps import step1, step2
       from cop.sinks import sink


       >>> from cop.sources import web
       >>> from cop.steps import delimit 
       >>> from cop.stops import grep
       >>> from cop.steps import transform
       >>> from cop.sinks import printer

       >>> url = ''
       >>> web(url,
       ...    delimit('\n',
       ...    transform(lambda l: l.lower(),
       ...    grep('monty', 
       ...    printer())))
     In this case, nothing comes out. That's
     basically because there's very little
     about Monty Python at

     /non-linear pipelines/
     Sometimes, a linear pathway isn't enough.
     Perhaps we would like to do something
     with intermediate results.

     Enter `cop.flow`. It provides `broadcast`
     for sending data to multiple steps, and
     `loadbalance`, which implements a round-
     robin load balancing system between steps.

     The start of this whole thing is from 
     code written by David Beazley, especially
     his tutorial on coroutines[0]. That
     work is excellent and should basically
     be considered the manual for this library. 

     I read the PDF of the tutorial in the
     morning. By the end of the evening I had
     a few dozen workable functions that could
     be strung together in an almost arbitary

     I would really like to see interacting
     with multiple cores and hosts become
     trivally easy. 

     If you would like to contribute to cop, 
     that's great. It's easy to do and will
     help people. At the moment, test 
     coverage is at a healthy 0%.

     The plan is to use heavy use of doctests. 
     `cop.printer` should be quite useful:

        >> source(sink(printer())) 

     I use >> as prompt when I don't want 
     doctest to claim it. I find this 
     preferable to using the skip directive.

==>  LEGAL
     Apart from code written by David Beazley,
     which is used with with permission, the
     code the copyright of Tim McNamara[1].
     The code is released under the Apache 2

     All documentation is the copyright of 
     Tim McNamara. All docs are released under
     the "Creative Commons Attribution 3.0
     New Zealand (CC BY 3.0)" licence[3], in
     addition to the Apache 2 licence where it
     /trade marks/
     "cop" is an unregistered trade mark of
     Tim McNamara under New Zealand law.
     /Consumer Guarantees Act 1993/ 
     If you use this software for personal use,
     you have certain rights under the Consumer
     Guarantees Act 1993. They are explained by
     the Ministry of Consumer Affairs[4].
     Generally speaking, cop must be of 
     "acceptable" quality and be fit for the 
     purposes described.
     However, when judging acceptable quality, 
     please consider that you are downloading
     free software from the Internet. 
     I'm not entirely certain whether
     software code counts as a good or a 
     service for the purpose of the Act
     No, we can't opt out of these provisions.
     Irrespective of the terms of the licence. 


     "Structure and Interpretation of Computer Programs"*
     Chaper 5: Sequences and Coroutines (esp. 5.3 "Coroutines")

     This is not the original SICP, written in Scheme, but
     a version which is Python specific.


     [1] @timClicks