FileNotFoundError: [WinError 2] The system cannot find the file specified: ... fastparquet.libs

[davetapley]

Description of the issue

⚠️ Only on Windows ⚠️

When attempting pandas.read_parquet with fastparquet installed:

FileNotFoundError: [WinError 2] The system cannot find the file specified: ... fastparquet.libs

Context information (for bug reports)

• Output of pyinstaller --version: 5.10.1
• Version of Python: 3.10.9
• Platform:
  • Windows Server 2022 Version 10.0.20348 Build 20348 ❌
  • Windows 10 10.0.19044 Build 19044 ❌
  • Ubuntu 20.04 x64 ✅
  • macOS 10.15.7 (19H2026) ✅
• How you installed Python: pyenv-win, pyenv

Make sure everything is packaged correctly

• start with clean installation
• use the latest development version
• Run your frozen program from a command window (shell) — instead of double-clicking on it
• Package your program in --onedir mode
• Package without UPX, say: use the option --noupx or set upx=False in your .spec-file
• Repackage you application in verbose/debug mode. For this, pass the option --debug to pyi-makespec or pyinstaller or use EXE(..., debug=1, ...) in your .spec file.

A minimal example program which shows the error

pandas == 1.5.2
fastparquet == 2023.2.0

# repro.py
from pandas import read_parquet

df = read_parquet("https://github.com/Teradata/kylo/raw/master/samples/sample-data/parquet/userdata1.parquet")
print(len(df))

$ pyinstaller repro.py --hidden-import fastparquet

Stacktrace / full error message

On Windows:

>  .\dist\repro\repro.exe
Traceback (most recent call last):
  File "repro.py", line 3, in <module>
    df = read_parquet("https://github.com/Teradata/kylo/raw/master/samples/sample-data/parquet/userdata1.parquet")
  File "pandas\io\parquet.py", line 501, in read_parquet
  File "pandas\io\parquet.py", line 48, in get_engine
  File "pandas\io\parquet.py", line 266, in __init__
  File "pandas\compat\_optional.py", line 141, in import_optional_dependency
  File "importlib\__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "fastparquet\__init__.py", line 19, in <module>
  File "fastparquet\__init__.py", line 10, in _delvewheel_init_patch_0_0_9
  File "os.py", line 1118, in add_dll_directory
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\Users\\dave\\pyinstaller-fastparquet\\dist\\repro\\fastparquet.libs'
[6312] Failed to execute script 'repro' due to unhandled exception!```

On Ubuntu / macOS:

$  .\dist\repro\repro
1000

[davetapley]

I assume similar / same as:

• Regression in scipy 1.9.1 -> 1.9.2 pyinstaller#7168

[rokm]

I assume similar / same as:

* [Regression in scipy 1.9.1 -> 1.9.2 pyinstaller/pyinstaller#7168](https://github.com/pyinstaller/pyinstaller/issues/7168)

Indeed, looks like we'll need a hook for fast_parquet to handle the external libs directory.

[rokm]

Moving to the contrib hooks repository...

[davetapley]

Haha, I almost opened it here, but picked the main repo because pyinstaller/pyinstaller#7168 got moved there 😆

@rokm is this something I (zero knowledge of PyInstaller internals) could do a PR for, or will someone else handle? 🙏🏻

[rokm]

I'll prepare a PR with a hook (and more importantly, a test), unless someone else beats me to it...