Adding datafiles of a third party module in hooks.

[VakarisZ]

Hello. I'm working on a module, which uses a third-party module without pyinstaller hooks.
So, I'm trying to
import policyuniverse but data.json is missing.
I know I can add datafiles to the .spec file, but since I'm not packaging the module itself, there's no spec file. Is there a way to include datas of another module, so that my module users wouldn't have to alter their spec files?

I've tried adding a hook hook-policyuniverse.py:

datas = collect_data_files('policyuniverse')

But this seems to do nothing.

[bwoodsend]

Where did you put the hook? It needs to be in or added to PyInstaller's hook search path. Also you need to build with the --clean option whenever you change a hook.

[VakarisZ]

I have similar setup to the hooksample application and I'm trying to test my module by running (test_venv) C:\Users\Vakaris\Desktop\ScoutSuite>pytest ./ScoutSuite/__pyinstaller. I'm not sure how pyinstaller is supposed to pick up hooks, but I presume it's supposed to look into the setup.cfg file.

My dir structure (relevant files marked green):

setup.cfg:

[options.entry_points]
pyinstaller40 =
  hook-dirs = ScoutSuite.__pyinstaller:get_hook_dirs
  tests     = ScoutSuite.__pyinstaller:get_PyInstaller_tests

__pyinstaller.__init__.py:

def get_hook_dirs():
    return [os.path.dirname(__file__)]

def get_PyInstaller_tests():
    return [os.path.dirname(__file__)]

test_scoutsuite_packaging.py:

def test_pyinstalller_scoutsuite(tmp_path):
    app_name = "ScoutSuiteApp"
    workpath = tmp_path / "build"
    distpath = tmp_path / "dist"
    app = tmp_path / (app_name + ".py")
    app.write_text("\n".join(TEST_CODE))
    args = [
        # Place all generated files in ``tmp_path``.
        '--workpath', str(workpath),
        '--distpath', str(distpath),
        '--specpath', str(tmp_path),
        '--clean',
        str(app),
    ]
    pyi_main.run(args)
    subprocess.run([str(distpath / app_name / app_name)], check=True)

Failing import statement:

from policyuniverse.expander_minimizer import get_actions_from_statement, _expand_wildcard_action

Traceback:

<...>
File "c:\users\vakaris\desktop\test_venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "ScoutSuite\core\conditions.py", line 7, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "c:\users\vakaris\desktop\test_venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\policyuniverse\__init__.py", line 21, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Vakaris\\AppData\\Local\\Temp\\pytest-of-Vakaris\\pytest-16\\test_pyinstalller_scoutsuite0\\dist\\ScoutSuiteApp\\poli
cyuniverse\\data.json'

[bwoodsend]

Hmm, looks like you've done it all right. There should be an INFO: Loading module hook 'hook-ScoutSuite.py' from ... in the build log if that hook is being noticed. Can you see it in there? Either way, it'll be easier to just put TEST_CODE in a .py file and PyInstaller it rather than to do it through pytest.

[VakarisZ]

I'm new to python packaging in general and missed this documentation, which elaborates on how to launch the tests etc. Seems like I solved my issues for now. Thanks a lot!