Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

Concurrent Computations on Multicore Processors

Slides from the talk Concurrent Computations on Multicore Processors given at PyCon SG 2015 on June 18, 2015 in Singapore (see the announcement of the talk here). A recording of the talk can be found here.

Description

In this talk we will discuss the multiprocessing module, which allows parallel programming on multicore processors. We will do a quick comparison with the threading module, cover the core API and finally discuss common pitfalls.

Abstract

When trying to improve the performance of a Python application on a multicore machine by means of concurrency, one typically employs either the threading or the multiprocessing module. While the former is used to spawn threads, the latter one allows the creation of subprocesses. Although threads are useful for e.g. I/O-bound problems, their usage is limited due to CPython's Global Interpreter Lock (GIL). On the other hand, the multiprocessing module can spawn subprocesses with their own interpreter instances, hence circumventing the GIL and enabling true parallel computations on multicore architectures.

In many cases, the multiprocessing package allows to gain a significant speed-up by parallelization of computationally intensive code sections. We will cover the most important functionality of this module such as parallel mapping, asynchronous function evaluations, shared memory maps and interprocess communication. In particular, a parallel map can be often used as a drop-in alternative for a sequential mapping. We will go through some examples and discuss common pitfalls.

About

Talk on Concurrent Computations on Multicore Processors in Python

Resources

License

Releases

No releases published

Packages

No packages published