Add configuration for setting the degree of compilation parallelism.

[yaoyuannnn]

Maybe the first question is how many cppimport will use by default. Since I'm running it on a machine with a small memory, sometimes I can see the compilation goes out of memory.

[tbenthompson]

Currently, parallelism is off by default. If you turn parallelism on, the number of cores is equal to multiprocessing.cpu_count().

This would be quite easy to modify. I have two proposals for how we could do this:

• Change the parallelism flag to instead take a parameter indicating the number of threads to use. Ideally, this would be done in a way that maintains backward compatibility with the current True/False behavior. So, perhaps, True == multiprocessing.cpu_count, False = 1, and then any integer uses that number of threads.
• A slightly different, but similar proposal would be to just add another configuration field that contains the number of threads to use.

Some locations in the code that would be useful for this:

cppimport/cppimport/templating.py

Line 46 in 00c40d9

parallel = False,

cppimport/cppimport/build_module.py

Line 57 in 00c40d9

def parallel_compile(self, sources, output_dir = None, macros = None,

cppimport/cppimport/build_module.py

Line 150 in 00c40d9

parallelize = cfg.get('parallel') and py33orgreater

I would be excited to accept a pull request if you have the time to implement this!

[tbenthompson]

Reopened because this is something I want to do eventually.