Skip to content
/ conpig Public

A green threading library with automated concurrency for python based on a processes, signal handlers and greenlets

License

Notifications You must be signed in to change notification settings

mmirman/conpig

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

35 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Conpig Threading Library

Conpig is a green threading library using green threads from Gevent which are based on greenlets. along side a signal based scheduler. This allows one true concurrency using green threads instead of OS processes or OS threads such that spawning a new thread is still cheap.

Want

  • Fast, cheap, easy, traditional concurrency.

Have

  • Cpython's GIL prevents OS threads from running concurrently and switching context for any reason but to perform IO.

  • OS threads are bulky and slow. Processes are even worse.

  • Current green thread implementations require manual context switches. This is tedious. This is also bug prone for programmers who are used to having a scheduler.

  • Processes based libraries like parallel python prevent you from communicating between "threads" with the datastructures you'd usually use. Again, processes are also slow and buggy.

Solution

  • Create a global scheduler using signals to schedule the next context switch.

Drawbacks & Limitations

  • A solution to concurrency, not parallelism.

  • Conpig threads still can only run on one core of a processor.

  • Context switches still only occur every CPython tick.

  • You still need cooperative yields if you want true fairness for programs that don't do as much IO.

Installing (pip to come)

  1. Install Greenlet with pip.

  2. Install Gevent

    • The old version depends on libevent. The new github version depends on libev
    • On OSX the install path for libevent is wrong even if you use Mac Ports.
    • The easiest install method is to download v.1 from github, and install from source using the provided "setup.py".

Usage

  • You can run from a main method
import conpig

def test(arg):
    for i in range(0,4000):
        print arg


conpig.spawn(test, "X")    
conpig.spawn(test, "O")

conpig.wait_all()

Authors

  • Matthew Mirman

  • Eric Faust

License

See LICENSE file.

About

A green threading library with automated concurrency for python based on a processes, signal handlers and greenlets

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages