Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Shrapnel is a scalable, high-performance cooperative threading library for Python.
Python Other

This branch is 257 commits behind master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
coro
coroutine
docs
include
pyrex
test
.gitignore
INSTALL
LICENSE
README.rst
distribute_setup.py
setup.py

README.rst

This Python library was evolved at IronPort Systems and has been provided as open source by Cisco Systems under an MIT license.

Intro

Shrapnel is a library for high-performance concurrency. It uses coroutines to implement user threads on top of either kqueue (FreeBSD, OS X) or /dev/epoll (linux), and is written mostly in Pyrex/Cython, supporting both 32-bit and 64-bit platforms. It is the culmination of about 8 years of work at IronPort Systems, a provider of high-speed mail appliances. It was open-sourced by Cisco Systems in late 2011.

Features

  • Lightweight threads, event-driven scheduler.
  • Underneath: non-blocking operations on descriptors, like sockets and pipes.
  • On top, synchronous API for straight-line, simple code.
  • Highly scalable - tens or hundreds of thousands of connections/threads.
  • Thread synchronization primitives, like mutexes, semaphores, etc...
  • with_timeout(): wrap any funcall with a timeout.
  • Wait on kqueue events like file/directory changes, signals, processes, etc... [kqueue only]
  • DNS stub resolver (full-fledged resolver may be forthcoming)
  • HTTP server and client (plus WebSocket, RFC6455 & hixie-76)
  • Support for TLS via tlslite (openssl interface may be forthcoming)
  • other protocols/codecs: ldap, asn1, ftp, mysql, postgres, AMQP.
  • MIT License.

Advantages

Compared to other concurrency packages available for Python, Shrapnel gives you:

  • Speed and Efficiency: the entire scheduler, poller, socket layer, synchronization objects, etc... are written in Cython, with an emphasis on performance and low memory usage.
  • Stock Python: Shrapnel works with out-of-the-box CPython [2.X]. No special variants of Python are needed, it will even work with your OS's OEM python installation. So you can use all the external libraries/modules you've come to rely on.
  • No Callbacks: no need to cuisinart your application into a thousand callbacks. No need to decompose every action into a state machine. Write simple, performant code now without having to send your programmers to class.
  • Drop to Cython for speed: all the capabilities of the system are available from Cython, so you can e.g. write a server entirely in Cython for speed. You can interface with external libraries, and do thread switches from Cython or C. It's even possible to have external C code call back into shrapnel. This makes it easy to prototype your application in Python, and then push only the hot spots into Cython.
  • Timeouts: Shrapnel provides a general timeout mechanism that can be used to wrap any function call with a timeout.
  • Profiler: Thread-aware profiler generates HTML reports.

Tutorial

See http://ironport.github.com/shrapnel/tutorial.html

API Documentation

See http://ironport.github.com/shrapnel/

Something went wrong with that request. Please try again.