Skip to content
Parallel Python ( with py2exe
Find file
Pull request Compare This branch is 1 commit ahead of master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Parallel Python and Py2EXE

This project is demostration how to bundle a program which is using Parallel Python ( into the .exe via py2exe.

* Requirement (what I have used):

Windows XP, Python 2.5 from, py2exe module

* Introduction:

Parallel Python starts new processes with subprocess.popen() function (on a local machine). The input and output is encoded with "pickle" and transfered via pipe between the processes. Traditionaly the new workers are started as: "python -u" with complete path.
Location of python interpreter is detected as sys.executable, location of is derived from __file__.

Pickle must be able to read the source code of the function, so .pyc is not enough. A simple proxy function with available source code is enough.

Py2exe exectuble is a stripped version of python interpreter, renamed according your script name (so script.exe instead of python.exe). Your script is embeded inside of the .exe file resources and executed during the start of the exe. All dependent python modules are compiled into .pyc/.pyo and zipped into (which can have different name or can be bundled in the .exe resources as well). Details:

* Usage:


cd dist

* Notes:

In the is an extra "zip" step to include the source code of function necessary for pickle functionality used in parallel python.

We must distribute the python.exe binary as well, because py2exe does not correctly implement the "unbuffered" option.
Once the "python -u" equivalent is available via py2exe, we have more options:
 - Distribute special ppworker.exe (compiled from
 - Implement something like for pp. This means we can distribute only one executable (sum_primes.exe), which when start with argument containing "" behaves like ppworker.

To test correct implementation of the unbuffered mode just change the pp/ line 131-133:
- "python -u" runs well
- "python" ends with EOFError during __unpickle
- "ppworker.exe" end with the same error as "python" - so unbuffered is not correct

If you remove the whole condition at line 130, the sum_primes.exe is called as "sum_primes.exe -u", which is correctly catched by argv test, but again, we end on EOFError. :-(

Something went wrong with that request. Please try again.