Skip to content

Commit

Permalink
Further extend and clarify the documentation of the py-clone and disa…
Browse files Browse the repository at this point in the history
…ble-reference-pool features
  • Loading branch information
adamreichold committed Apr 28, 2024
1 parent 2c5bf1c commit 9e43e2b
Show file tree
Hide file tree
Showing 4 changed files with 24 additions and 6 deletions.
4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -106,10 +106,10 @@ auto-initialize = []
# Allows use of the deprecated "GIL Refs" APIs.
gil-refs = []

# Enables `Clone`ing references to Python objects which aborts if the GIL is not held.
# Enables `Clone`ing references to Python objects `Py<T>` which panics if the GIL is not held.
py-clone = []

# Disables global reference pool which allows `Drop`ing references to Python objects without the GIL being held.
# Disables global reference pool which allows `Drop`ing references to Python objects `Py<T>` without the GIL being held.
disable-reference-pool = []

# Optimizes PyObject to Vec conversion and so on.
Expand Down
10 changes: 10 additions & 0 deletions guide/src/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,16 @@ This feature is a backwards-compatibility feature to allow continued use of the

This feature and the APIs it enables is expected to be removed in a future PyO3 version.

### `py-clone`

This feature was introduced to ease migration. It was found that delayed reference counts cannot be made sound and hence `Clon`ing an instance of `Py<T>` must panic without the GIL being held. To avoid migrations introducing new panics without warning, the `Clone` implementation itself is now gated behind this feature.

### `disable-reference-pool`

This is a performance-oriented feature which disabled the global reference pool and the assocaited overhead for the crossing the Python-Rust boundary. However, if enabled, `Drop`ping an instance of `Py<T>` without the GIL being held will abort the process.

Since this feature does not really have additive semantics, it should only be enabled in leaf crates, i.e. in a crate producing a Python extension or embedding the Python interpreter.

### `macros`

This feature enables a dependency on the `pyo3-macros` crate, which provides the procedural macros portion of PyO3's API:
Expand Down
2 changes: 1 addition & 1 deletion guide/src/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ PyO3 uses global mutable state to keep track of deferred reference count updates

This functionality can be avoided by adding the `disable-reference-pool` feature. This removes the global reference pool and the associated costs completely. However, it does _not_ remove the `Drop` implementation for `Py<T>` which is necessary to interoperate with existing Rust code written without PyO3-based code in mind. To stay compatible with the wider Rust ecosystem in these cases, we keep the implementation but abort when `Drop` is called without the GIL being held.

This limitation is important to keep in mind when this setting is used, especially when embedding Python code into a Rust application as it is quite easy to accidentally drop a `Py<T>` returned from `Python::with_gil` without making sure to re-acquire the GIL beforehand. For example, the following code
This limitation is important to keep in mind when this setting is used, especially when embedding Python code into a Rust application as it is quite easy to accidentally drop a `Py<T>` (or types containing it like `PyErr`, `PyBackedStr` or `PyBackedBytes`) returned from `Python::with_gil` without making sure to re-acquire the GIL beforehand. For example, the following code

```rust,ignore
# use pyo3::prelude::*;
Expand Down
14 changes: 11 additions & 3 deletions src/instance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1815,8 +1815,9 @@ where
}

/// If the GIL is held this increments `self`'s reference count.
/// Otherwise this registers the [`Py`]`<T>` instance to have its reference count
/// incremented the next time PyO3 acquires the GIL.
/// Otherwise, it will panic.
///
/// Only available if the `py-clone` feature is enabled.
#[cfg(feature = "py-clone")]
impl<T> Clone for Py<T> {
#[track_caller]
Expand All @@ -1828,7 +1829,14 @@ impl<T> Clone for Py<T> {
}
}

/// Dropping a `Py` instance decrements the reference count on the object by 1.
/// Dropping a `Py` instance decrements the reference count
/// on the object by one if the GIL is held.
///
/// Otherwise and by default, this registers the underlying pointer to have its reference count
/// decremented the next time PyO3 acquires the GIL.
///
/// However, if the `disable-reference-pool` feature is enabled,
/// it will abort the process.
impl<T> Drop for Py<T> {
#[track_caller]
fn drop(&mut self) {
Expand Down

0 comments on commit 9e43e2b

Please sign in to comment.