Skip to content
Enabling CPython multi-core parallelism via subinterpreters.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE Initial commit Aug 17, 2018
README.md Update README.md Aug 24, 2018

README.md

multi-core-python

Enabling CPython multi-core parallelism via subinterpreters.

This repo is for tracking the effort and as a place to keep any tooling.

wiki | projects

Project Summary

From my python-ideas post (June 2015):

Python's multi-core story is murky at best.  Not only can we be more
clear on the matter, we can improve Python's support.  The result of any
effort must make multi-core (and concurrency) support in Python obvious,
unmistakable, and undeniable (and keep it Pythonic).

The goal of this project is to do just that, by using the existing subinterpreter C-API.

Solution

The minimal multi-core solution will involve:

  • resolving existing bugs and blockers
  • exposing the existing support (from C-API) in a stdlib module
  • adding a mechanism to safely "share" basic immutable objects between interpreters
  • no longer sharing the GIL between interpreters

Once a minimal solution is in place we can expand from there.

At a high level, we're doing the following concrete tasks:

Constraints / Requirements

  1. no significant impact on single-threaded performance
  2. maintain backward compatibility (C-API, etc.)
  3. (pseudo-)compatibility with multiprocessing/threading/concurrent.futures APIs
  4. a multi-core concurrency model/approach that fits our brains
  5. Python APIs
  6. supportable on other Python implementations
You can’t perform that action at this time.