A green threading library with automated concurrency for python based on a processes, signal handlers and greenlets
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
LICENSE
README.md
conpig.py
test0.py
test1.py
test2.py
test3.py
test4.py
test5.py
test6.py

README.md

Conpig Threading Library

Conpig is a green threading library using green threads from Gevent which are based on greenlets. along side a signal based scheduler. This allows one true concurrency using green threads instead of OS processes or OS threads such that spawning a new thread is still cheap.

Want

  • Fast, cheap, easy, traditional concurrency.

Have

  • Cpython's GIL prevents OS threads from running concurrently and switching context for any reason but to perform IO.

  • OS threads are bulky and slow. Processes are even worse.

  • Current green thread implementations require manual context switches. This is tedious. This is also bug prone for programmers who are used to having a scheduler.

  • Processes based libraries like parallel python prevent you from communicating between "threads" with the datastructures you'd usually use. Again, processes are also slow and buggy.

Solution

  • Create a global scheduler using signals to schedule the next context switch.

Drawbacks & Limitations

  • A solution to concurrency, not parallelism.

  • Conpig threads still can only run on one core of a processor.

  • Context switches still only occur every CPython tick.

  • You still need cooperative yields if you want true fairness for programs that don't do as much IO.

Installing (pip to come)

  1. Install Greenlet with pip.

  2. Install Gevent

    • The old version depends on libevent. The new github version depends on libev
    • On OSX the install path for libevent is wrong even if you use Mac Ports.
    • The easiest install method is to download v.1 from github, and install from source using the provided "setup.py".

Usage

  • You can run from a main method
import conpig

def test(arg):
    for i in range(0,4000):
        print arg


conpig.spawn(test, "X")    
conpig.spawn(test, "O")

conpig.wait_all()

Authors

  • Matthew Mirman

  • Eric Faust

License

See LICENSE file.