Python can't find DLL when linking external C library (gtstreamer) on Windows

[marcpabst]

Bug Description

Hey, I really like maturin/PyO3 and I've been using it for a project linking gstreamer. I'm now testing my package on Windows (was using MacOS before) and I'm running into a problem with Python unable to find DLLs.

A simple example repo can be found here: https://github.com/marcpabst/webcam-py

Importing the created python package (maturin develop) works fine on MacOS but fails on Windows with ImportError: DLL load failed while importing webcam_py: The specified module could not be found.. It works when I manually copy the gstreamer DLLs to tge site-packages folder, but that seems to break internal linking within gstreamer.

Any pointers?

Your maturin version (maturin --version)

1.4.0

Your Python version (python -V)

Python

Your pip version (pip -V)

3.8.0

What bindings you're using

pyo3

Does cargo build work?

• Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

• Yes

Steps to Reproduce

1. Run the provided repo with maturin develop
2. Try to import webcam_py
3. 3. Observe the ImportError: DLL load failed while importing webcam_py: The specified module could not be found.

[marcpabst]

Looks like Python is not automatically searching $PATH from v3.8 onwards, it works if you use os.add_dll_directory(). Should maturin/PyO3 handle this?

[messense]

How should maturin handle this? What's the right argument to pass to os.add_dll_directory()?

[marcpabst]

Good queston. os.add_dll_directory() needs to be called with each directory to look for DLLs in for. Could we maybe have an option to just add everything in PATH to os.add_dll_directory() when importing? The alternative is some form of bundling, but that can cause license issues.

I'm still not sure why it just works on MacOS.

for context: https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew

[messense]

IMHO, for security it should be left for the library/user to call os.add_dll_directory(), doing in in maturin/pyo3 unconditionally seems worrisome.

[marcpabst]

What are the security implications you're worried about? IMHO this is the standard for system-wide dynamic DLLs and most Python modules that present bindings to C libs will search PATH. I generally think that it's highly unintuitive to require a library user to call os.add_dll_directory() before importing things. You can't even have your library handle this (by letting the user specify a search path), because linked libraries will always be loaded upon imports (I think).

So there needs to be some sort of way to either tell maturin or poy3 to add PATH, or to inject custom python code before the module DLL is loaded. I'm not suggesting this should happen unconditionally for every module, but for some modules this will just be required.

[messense]

What are the security implications you're worried about?

The doc says:

DLL dependencies for extension modules and DLLs loaded with ctypes on Windows are now resolved more securely.

If there isn't any security implication, you'd want python to handle it automatically, right?

Anyway, I suggest you use a mixed Python/Rust project layout and add os.add_dll_directory() call before importing the Rust extension module.

[marcpabst]

Thats probbaly the way to go, thanks! Maybe worth documenting as the standard error when trying to import on Windows is not very helpful.