Improve collection of QtWebEngineProcess helper and ensure Qt is relocatable #8315
Conversation
... from Qt provided by linux distributions When collecting `QtWebEngineProcess` helper on linux, always force the relatve target collection directory to be `libexec` directory under the Qt directory within PyQt/PySide package directory. This ensures that the helper is collected in PyPI-wheel-compliant location even when collecting Qt that is provided by linux distribution; for example, the correct location for PyQt5 is `_MEIPASS/PyQt5/Qt5/libexec`, as opposed to auto-computed `_MEIPASS/PyQt5/Qt5/lib64/qt5/libexec` due to Fedora Qt5 package installing the helper under `/usr/lib64/qt5/libexec` (with Qt `PrefixPath` being `/usr` and `LibraryExecutablesPath` being `/usr/lib64/qt5/libexec`).
... from Anaconda PySide2 package on Windows When collecting `QtWebEngineProcess` helper from `PySide2`/`PySide6` on Windows, always force the target collection directory to be the PySide package directory (i.e., the relative target collection directory should be `.`). This ensures that the helper is collected in PyPI-wheel-compliant location, even if the package is provided by Anaconda, where the Qt `PrefixPath` is for example `C:/Users/<user>/miniconda3/envs/<env-name>/Library` and the corresponding `LibraryExecutablesPath` is `C:/Users/<user>/miniconda3/envs/<env-name>/Library/bin`. Thus, the auto-computed relative target collection directory would be `bin`, but we need to match the PyPI wheel layout, where the `LibraryExecutablesPath` is the same as the `PrefixPath` (i.e., the PySide package directory).
Have the run-time hooks for `PySide2`, `PySide6`, `PyQt5`, and `PyQt6` check for the presence of embedded `:/qt/etc/qt.conf` resource, and if not present, inject their own version. This aims to ensure that the bundled Qt is always relocatable. All Qt bindings packages, when packaged as PyPI wheels, inject embedded `qt.conf` file as Qt embedded resource during their initialization, and use it to set the `PrefixPath` to the top-level Qt directory within the package's directory. This enures that all paths in `QLibraryInfo` are properly set, and is essential for making the package and its bundled Qt relocatable. Qt bindings packaged by other providers (for example, Anaconda packages on all OSes, or packages provided by linux distributions) do not seem to use this mechanism when running under unfrozen python. However, when collected into PyInstaller-frozen application, `PyQt5` and `PyQt6` packages, even if originally not packaged as PyPI wheels, seem to enable the embedded `qt.conf` injection, making themselves relocatable. On the other hand, `PySide2` package does not, which makes it non-relocatable. For example, when using Fedora-packaged `PySide2` in a frozen application, the `PrefixPath` remains set to `/usr`, and thus a `QtWebEngine`-enabled frozen application ends up using system-installed `QtWebEngineProcess` helper in `/usr/lib64/qt5/libexec`, if available (and failing to start if it is unavailable). In the case of Anaconda-packaged `PySide2`, the `PrefixPath` path ends up being `/`, and the collected `QtWebEngineProcess` fails to be discovered. Therefore, if the embedded `qt.conf` resource is not provided by the package itself, we attempt to provide our own, to ensure that the package is relocatable, and that paths in `QLibraryInfo` are properly set (i.e., that they match the relocatable PyPI wheel layout, which we are trying to reconstruct in our Qt collection approach).
rokm
force-pushed
the
nonstandard-qtwebengine-locations
branch
from
1b5c91c
to
331be47
Compare
In some of our tests, we seem to collect only the top-level Qt bindings package (for example, `PyQt5`), without any of its submodules. Consequently, the `QtCore` submodule is unavailable. Have the newly-introduced `create_embedded_qt_conf()` helper in `_pyi_rth_utils.qt` gracefully handle such cases (i.e., exit on `ImportError`, as without `QtCore`, we cannot check for presence or absence of the embedded `:/qt/etc/qt.conf` resource).
There was a problem hiding this comment.
I was a lot happier with the custom Qt distributions aren't supported stance. Debian can't even package a simple pure Python package without relocating site-packages to a Python ABI invariant dist-packages using a monkeypatched setuptools, smothering it in pre-install/post-install scripts to deal with the bytecode then occasionally converting the *.dist-info directory into a single file for no discernible reason – I don't like the idea of supporting whatever they might choose to do to Qt.
Fix/extend collection of NSS shared libraries when collecting `QtWebEngine` on Linux. The existing implementation assumes that the said shared libraries are located in a `nss` sub-directory located next to the `libnss3.so` shared library. However, we need to collect those libraries even if they are not in a separate directory, but rather located next to the `libnss3.so`. This is, for example, the case on Debian bookworm: https://packages.debian.org/bookworm/amd64/libnss3/filelist whereas previous versions had separate `nss` directory: https://packages.debian.org/bullseye/amd64/libnss3/filelist
Extend the mechanism for collection of `.hmac` files from pyinstaller#8288 to `.chk` files that are used by NSS libraries.
So out of morbid curiosity, I gave this a try with Debian Bullseye and Bookworm. With changes made in this PR, a With Bookworm-packaged When looking for those NSS libraries, I noticed that they have accompanying |
|
Well now I'm tempted to throw some other arbitrary platform at you and see what you flush out. |
Starting with PyInstaller 6.5.0, it imports Qt bindings early, due to this change: pyinstaller/pyinstaller#8315 We warn about this in order to avoid unintentional early Qt imports in qutebrowser. However, in the case of using PyInstaller, we just suppress the warning now, as it's not us to blame. See #8123
Improve collection of
QtWebEngineProcesshelper in two specific cases:In both cases, we need to override the auto-computed relative destination path to match the standard PyPI wheel layout, which our Qt collection approach is trying to reconstruct.
Fixes #8307.
When trying to test the above changes, it turned out that
PySide2, collected either from Fedora RPMs or from Anaconda (on Windows, Linux, or macOS), does not inject the embeddedqt.confresource to make the Qt relocatable, and consequently the paths returned byQLibraryInfoare incorrect (and theQtWebEngineProcesshelper fails to be discovered, although it is collected into correct location). Curiously enough, the Fedora- or Anaconda-installedPyQtpackages (both 5 and 6) do not seem to inject the embeddedqt.confwhen located in their original standard library directory, but they do once they are collected into PyInstaller-frozen application. So it looks like that they have dynamic switch, whereas withPySidepackages, this switch is static. (Sidenote: PyPI wheels of all four bindings always seem to inject the embeddedqt.conf, because they always need to be fully relocatable).So now our run-time hooks for all four Qt bindings check for the presence of embedded
qt.confresource, and on the off chance that it is not present, inject their own version, in an attempt to ensure that the collected Qt is always relocatable and that paths inQLibraryInfoare correct (that they point to the collected Qt directories, and that the layout matches the layout of PyPI wheels that our Qt collection approach is trying to reconstruct).