Code for High Performance Computing tutorial for EuroPython 2011
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Source code for High Performance Computing tutorial at EuroPython 2011 

The 4 hour tutorial will cover various ways of speeding up the provided Mandelbrot code with a variety of Python packages that let us go from bytecode to C, run on many CPUs and many machines and also use a GPU. The presentation for the tutorial should give the necessary background.

All the files are in subdirectories and are independent of each other, the general pattern is:
python 1000 1000
where "" might be named e.g. "" or "", the first 1000 is the pixel width and height, the second 1000 is the number of iterations. 1000x1000px plots with 1000 iterations are pretty. Use the arguments "100 30" for a super quick test to validate that things are working (it makes a 100x100px image using only 30 iterations).

The tutorial starts by using cProfile, RunSnakeRun and line_profiler to find the bottleneck, we then improve the code and add libraries to keep making things faster.

Overview of the versions:
pure_python: Python implementations for python and pypy
cython_pure_python: a converstion of the python code using cython
numpy_loop: a conversion of the python code using numpy vectors (but run without vector calls)
cython_numpy_loop: as numpy_loop but compiled with cython
numpy_vector: using vector calls on numpy vectors
numpy_vector_numexpr: adding numexpr on the numpy vectors
shedskin: minor conversion to get good speed using shedskin
multiprocessing: using built-in multiprocessing module to run on all cores using pure python implementation
parallelpython_pure_python: using parallelpython module to run across machines and cpus
parallelpython_cython_pure_puthon: showing compiled cython version of pure_python running over machines 
pycuda: gpuarray, elementwisekernel and sourcemodule examples of numpy-like and C code on CUDA GPUs via python

Note that the pure_python examples run fine using PyPy or Python 2.7.1.

Blog write-up (TO FOLLOW)

Versions of packages used to create this tutorial:
Cython 0.14.1
numexpr 1.4.2
numpy 1.5.1
pyCUDA HEAD from git as of 14th June 2011 (with CUDA 4.0 drivers)
PyPy 1.5
Python 2.7.1
ParallelPython 1.6.1
ShedSkin 0.7.1