python_performance.md

Performance Optimization for Python

Basics

• Basic tips for optimizing the performace of your Python code

• Code for the book "High Performance Python" by Micha Gorelick and Ian Ozsvald with OReilly

Interfacing with C/C++

• Interfacing with C from Scipy lecture notes
  • Very nice overview and examples of four approaches.
  1. Python/C API
     • Extending Python3.7 with C or C++
  2. ctypes - A foreign function library for Python
     • It provides C compatible data types, and allows calling functions in DLLs or shared libraries. It can be used to wrap these libraries in pure Python.
  3. SWIG
     • SWIG is an interface compiler that connects programs written in C and C++ with scripting languages such as Python
  4. Cython - C-Extensions for Python
     • Basic tutorial of Cython
     • Tutorial on how to use Cython to optimize Python code by Adrian Price-Whelan
     • Cython tutorial by Pauli Virtanen from 2011

Only for C++

• Boost.Python

  • Boost.Python, a C++ library which enables seamless interoperability between C++ and the Python programming language.

• pybind11 - Seamless operability between C++11 and Python

  • pybind11 is a lightweight header-only library that exposes C++ types in Python and vice versa, mainly to create Python bindings of existing C++ code.
  • This is used by the LSST developers, please see the DM Pybind11 style guide for details.

Using Just-in-Time (JIT) Compiler

• pypy - a fast, compliant alternative implementation of the Python language

• numba - makes Python code fast

  • Numba is an open source JIT compiler that translates a subset of Python and NumPy code into fast machine code. __Numba__only supports LLVM.
  • Numba offers a range of options for parallelizing your code for CPUs and GPUs, often with only minor code changes.
  • Numpy supports in Numba
  • Numba has a vectorize and guvectorize decorators that can be very useful.

• hope - A Python Just-In-Time compiler for astrophysical computations

  • hope is a specialized method-at-a-time JIT compiler written in Python for translating Python source code into C++ and compiles this at runtime.
  • Has not been updated for three yeears.

Tutorial and notes

• Optimizing Python in the Real World: NumPy, Numba, and the NUFFT by Jake VanderPlas

Making Numpy faster

• jax - GPU- and TPU-backed NumPy with differentiation and JIT compilation by Google

  • JAX is Autograd and XLA, brought together for high-performance machine learning research.

• autograd - Efficiently computes derivatives of numpy code

Other packages

• How to speed up Python by scikit-learn

Parallel computing in Python

Tutorial

• An introduction to parallel programming using Python's multiprocessing module
• Speed Up Your Algorithms Part 3 — Parallel-ization

Software

Common tools:

• mpi4py - Python bindings for MPI
• joblib - a set of tools to provide lightweight pipelining in Python
• loky - Robust and reusable Executor for joblib
• schwimmbad - A common interface to processing pools

More "Big Data" approach:

• Dask - Parallel computing with task scheduling
  • Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you love
  • Dask is open source and freely available. It is developed in coordination with other community projects like Numpy, Pandas, and Scikit-Learn.
  • Official Dask tutorial using Jupyter notebook