Skip to content

Commit

Permalink
Add support for generating non-abi3 python import libraries for Windows
Browse files Browse the repository at this point in the history
targets
  • Loading branch information
messense authored and davidhewitt committed May 15, 2022
1 parent a025e8e commit 29b8731
Show file tree
Hide file tree
Showing 8 changed files with 37 additions and 23 deletions.
4 changes: 2 additions & 2 deletions Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,9 +193,9 @@ Some of the functionality of `pyo3-build-config`:
`PYO3_CROSS_PYTHON_IMPLEMENTATION`) or system files.
When cross compiling extension modules it is often possible to make it work without any
additional user input.
- When an experimental feature `generate-abi3-import-lib` is enabled, the `pyo3-ffi` build script can
- When an experimental feature `generate-import-lib` is enabled, the `pyo3-ffi` build script can
generate `python3.dll` import libraries for Windows targets automatically via an external
[`python3-dll-a`] crate. This enables the users to cross compile abi3 extensions for Windows without
[`python3-dll-a`] crate. This enables the users to cross compile Python extensions for Windows without
having to install any Windows Python libraries.

<!-- External Links -->
Expand Down
4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,9 @@ abi3-py39 = ["abi3-py310", "pyo3-build-config/abi3-py39", "pyo3-ffi/abi3-py39"]
abi3-py310 = ["abi3", "pyo3-build-config/abi3-py310", "pyo3-ffi/abi3-py310"]

# Automatically generates `python3.dll` import libraries for Windows targets.
generate-abi3-import-lib = ["pyo3-ffi/generate-abi3-import-lib"]
generate-import-lib = ["pyo3-ffi/generate-import-lib"]
# Deprecated, replaced by `generate-import-lib`
generate-abi3-import-lib = ["generate-import-lib"]

# Changes `Python::with_gil` and `Python::acquire_gil` to automatically initialize the
# Python interpreter if needed.
Expand Down
2 changes: 1 addition & 1 deletion examples/maturin-starter/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,6 @@ crate-type = ["cdylib"]
pyo3 = { path = "../../", features = ["extension-module"] }

[features]
abi3 = ["pyo3/abi3-py37", "pyo3/generate-abi3-import-lib"]
abi3 = ["pyo3/abi3-py37", "pyo3/generate-import-lib"]

[workspace]
14 changes: 7 additions & 7 deletions guide/src/building_and_distribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ PyO3 will still attempt to compile `abi3` extension modules after displaying a w
On Unix-like systems this works unconditionally; on Windows you must also set the `RUSTFLAGS` environment variable
to contain `-L native=/path/to/python/libs` so that the linker can find `python3.lib`.

If the `python3.dll` import library is not available, an experimental `generate-abi3-import-lib` crate
If the `python3.dll` import library is not available, an experimental `generate-import-lib` crate
feature may be enabled, and the required library will be created and used by PyO3 automatically.

*Note*: MSVC targets require LLVM binutils (`llvm-dlltool`) to be available in `PATH` for
Expand Down Expand Up @@ -242,12 +242,12 @@ When cross-compiling, PyO3's build script cannot execute the target Python inter
* `PYO3_CROSS_PYTHON_VERSION`: Major and minor version (e.g. 3.9) of the target Python installation. This variable is only needed if PyO3 cannot determine the version to target from `abi3-py3*` features, or if `PYO3_CROSS_LIB_DIR` is not set, or if there are multiple versions of Python present in `PYO3_CROSS_LIB_DIR`.
* `PYO3_CROSS_PYTHON_IMPLEMENTATION`: Python implementation name ("CPython" or "PyPy") of the target Python installation. CPython is assumed by default when this variable is not set, unless `PYO3_CROSS_LIB_DIR` is set for a Unix-like target and PyO3 can get the interpreter configuration from `_sysconfigdata*.py`.

An experimental `pyo3` crate feature `generate-abi3-import-lib` enables the user to cross-compile
"abi3" extension modules for Windows targets without setting the `PYO3_CROSS_LIB_DIR` environment
An experimental `pyo3` crate feature `generate-import-lib` enables the user to cross-compile
extension modules for Windows targets without setting the `PYO3_CROSS_LIB_DIR` environment
variable or providing any Windows Python library files. It uses an external [`python3-dll-a`] crate
to generate import libraries for the Stable ABI Python DLL for MinGW-w64 and MSVC compile targets.
*Note*: MSVC targets require LLVM binutils to be available on the host system.
More specifically, `python3-dll-a` requires `llvm-dlltool` executable to be present in `PATH` when
to generate import libraries for the Python DLL for MinGW-w64 and MSVC compile targets.
*Note*: MSVC targets require LLVM binutils or MSVC build tools to be available on the host system.
More specifically, `python3-dll-a` requires `llvm-dlltool` or `lib.exe` executable to be present in `PATH` when
targeting `*-pc-windows-msvc`.

An example might look like the following (assuming your target's sysroot is at `/home/pyo3/cross/sysroot` and that your target is `armv7`):
Expand Down Expand Up @@ -277,7 +277,7 @@ cargo build --target x86_64-pc-windows-gnu
Any of the `abi3-py3*` features can be enabled instead of setting `PYO3_CROSS_PYTHON_VERSION` in the above examples.

`PYO3_CROSS_LIB_DIR` can often be omitted when cross compiling extension modules for Unix and macOS targets,
or when cross compiling "abi3" extension modules for Windows and the experimental `generate-abi3-import-lib`
or when cross compiling extension modules for Windows and the experimental `generate-import-lib`
crate feature is enabled.

The following resources may also be useful for cross-compiling:
Expand Down
6 changes: 3 additions & 3 deletions guide/src/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,12 @@ These features are extensions of the `abi3` feature to specify the exact minimum

See the [building and distribution](building_and_distribution.md#minimum-python-version-for-abi3) section for further detail.

### `generate-abi3-import-lib`
### `generate-import-lib`

This experimental feature is used to generate import libraries for the Stable ABI Python DLL
This experimental feature is used to generate import libraries for Python DLL
for MinGW-w64 and MSVC (cross-)compile targets.

Enabling it allows to (cross-)compile `abi3` extension modules to any Windows targets
Enabling it allows to (cross-)compile extension modules to any Windows targets
without having to install the Windows Python distribution files for the target.

See the [building and distribution](building_and_distribution.md#building-abi3-extensions-without-a-python-interpreter)
Expand Down
12 changes: 9 additions & 3 deletions pyo3-build-config/src/impl_.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1409,8 +1409,9 @@ fn default_cross_compile(cross_compile_config: &CrossCompileConfig) -> Result<In

// Auto generate python3.dll import libraries for Windows targets.
#[cfg(feature = "python3-dll-a")]
if abi3 && lib_dir.is_none() {
lib_dir = self::import_lib::generate_abi3_import_lib(&cross_compile_config.target)?;
if lib_dir.is_none() {
let py_version = if abi3 { None } else { Some(version) };
lib_dir = self::import_lib::generate_import_lib(&cross_compile_config.target, py_version)?;
}

Ok(InterpreterConfig {
Expand Down Expand Up @@ -1723,7 +1724,12 @@ pub fn make_interpreter_config() -> Result<InterpreterConfig> {
// Auto generate python3.dll import libraries for Windows targets.
#[cfg(feature = "python3-dll-a")]
{
interpreter_config.lib_dir = self::import_lib::generate_abi3_import_lib(&host)?;
let py_version = if interpreter_config.abi3 {
None
} else {
Some(interpreter_config.version)
};
interpreter_config.lib_dir = self::import_lib::generate_import_lib(&host, py_version)?;
}

Ok(interpreter_config)
Expand Down
14 changes: 9 additions & 5 deletions pyo3-build-config/src/import_lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,24 @@ use python3_dll_a::ImportLibraryGenerator;

use crate::errors::{Context, Result};

use super::{Architecture, OperatingSystem, Triple};
use super::{Architecture, OperatingSystem, PythonVersion, Triple};

/// Generates the `python3.dll` import library for Windows targets.
/// Generates the `python3.dll` or `pythonXY.dll` import library for Windows targets.
///
/// Places the generated import library into the build script output directory
/// and returns the full library directory path.
///
/// Does nothing if the target OS is not Windows.
pub(super) fn generate_abi3_import_lib(target: &Triple) -> Result<Option<String>> {
pub(super) fn generate_import_lib(
target: &Triple,
py_version: Option<PythonVersion>,
) -> Result<Option<String>> {
if target.operating_system != OperatingSystem::Windows {
return Ok(None);
}

let out_dir = env::var_os("OUT_DIR")
.expect("generate_abi3_import_lib() must be called from a build script");
let out_dir =
env::var_os("OUT_DIR").expect("generate_import_lib() must be called from a build script");

// Put the newly created import library into the build script output directory.
let mut out_lib_dir = PathBuf::from(out_dir);
Expand All @@ -37,6 +40,7 @@ pub(super) fn generate_abi3_import_lib(target: &Triple) -> Result<Option<String>
let env = target.environment.to_string();

ImportLibraryGenerator::new(&arch, &env)
.version(py_version.map(|v| (v.major, v.minor)))
.generate(&out_lib_dir)
.context("failed to generate python3.dll import library")?;

Expand Down
4 changes: 3 additions & 1 deletion pyo3-ffi/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ abi3-py39 = ["abi3-py310", "pyo3-build-config/abi3-py39"]
abi3-py310 = ["abi3", "pyo3-build-config/abi3-py310"]

# Automatically generates `python3.dll` import libraries for Windows targets.
generate-abi3-import-lib = ["pyo3-build-config/python3-dll-a"]
generate-import-lib = ["pyo3-build-config/python3-dll-a"]
# Deprecated, replaced by `generate-import-lib`
generate-abi3-import-lib = ["generate-import-lib"]


[build-dependencies]
Expand Down

0 comments on commit 29b8731

Please sign in to comment.