README.md

The repo contains the source code for the Making Python 100x faster with less than 100 lines of Rust blog post (there's a copy in this repo as well).

It's a demo library in Python that we can convert parts of to Rust, to improve its performance.

Here's a table summarizing the perf gains:

Version	Avg time per iteration (ms)	Multiplier
v1 - Baseline implementation (Python)	293.41	1x
v2 - Naive line-to-line translation of find_close_polygons	23.44	12.50x
v3 - Polygon implementation in Rust	6.29	46.53x
v4 - Optimized allocations	2.90	101.16x
v5 - Move outer loop to Rust (*)	0.81	362.23x

(*) v5 was added after the post was published, showing how to reduce Python to Rust overhead even more.

There are also two versions which use "vectorizing" (doing more of the work directly in numpy). While not applicable to the original library, they also show major performance gains:

Version	Avg time per iteration (ms)	Multiplier
v1 - Baseline implementation (Python)	293.41	1x
v1.5 - A bit of numpy vectorization	41.79	7.02x
v1.6 - A lot of numpy vectorization, by @dangrie158	5.18	56.68x

Setup

The code should work on all supported platforms (Python & Rust), but --native profiling is only supported by py-spy on x86 Linux and Windows.

(macOS / Arm Linux can still generate profiles viewing just the Python code.)

For the complete Rust setup, you'll need Rust (use rustup).

To setup with rye, run:

rye sync

To build the native extension, run:

source .venv/bin/activate
(cd poly_match_rs && maturin develop --release)

To run the benchmarks, run:

rye run python ./measure_all.py

For profiling,

sudo rye run py-spy record -o profile.svg -- python measure.py

Exploring more optimizations

There are a few more optimizations we can try.

For example, we could use (dx^2 + dy^2) < dist^2 (calculating dist^2 only once, saving a sqrt). According to the profiler outputs, can we expect this to make a big difference?

We could build a Rust bench using criterion, which would require us to "split" the Python & pure-Rust parts (Maybe using AsRef<Polygon>).

We could also try to convert the list of Polygons only once to Rust at the start of the run. According to the profiler outputs, can we expect this to make a big difference? (See v5)

License

This repo is licensed under either of

Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.