Issue in python 3.6 in the _MAKEFILE and _CONFIG_H compare/pass prefixes handling

[poteznyKrolik]

pyinstaller/PyInstaller/utils/hooks/__init__.py

Line 793 in 5d4bbbe

possible_prefixes.sort(key=lambda p: len(p), reverse=True)

I had an issue getting py installer to run

data-team/common/dag_env_check/dag_env_check onDATA-1480[$!?] via ·dag_env_check_installer via 🐍3.6.8  took 5s 
✘1 ➜ PyInstaller --onefile dag_env_check 
52 INFO: PyInstaller: 3.5
52 INFO: Python: 3.6.8
60 INFO: Platform: Darwin-18.7.0-x86_64-i386-64bit
60 INFO: wrote /Users/martinzajac/projects/team/data-team/common/dag_env_check/dag_env_check/dag_env_check.spec
64 INFO: UPX is not available.
66 INFO: Extending PYTHONPATH with paths
['/Users/martinzajac/projects/team/data-team/common/dag_env_check',
 '/Users/martinzajac/projects/team/data-team/common/dag_env_check/dag_env_check']
66 INFO: checking Analysis
66 INFO: Building Analysis because Analysis-00.toc is non existent
66 INFO: Initializing module dependency graph...
67 INFO: Initializing module graph hooks...
69 INFO: Analyzing base_library.zip ...
2705 INFO: running Analysis Analysis-00.toc
2711 INFO: Caching module hooks...
2715 INFO: Analyzing /Users/martinzajac/projects/team/data-team/common/dag_env_check/dag_env_check/dag_env_check
2966 INFO: Processing pre-find module path hook   distutils
3815 INFO: Processing pre-find module path hook   site
3815 INFO: site: retargeting to fake-dir '/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/fake-modules'
4568 INFO: Loading module hooks...
4569 INFO: Loading module hook "hook-lib2to3.py"...
4570 INFO: Loading module hook "hook-encodings.py"...
4633 INFO: Loading module hook "hook-sysconfig.py"...
/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer
/Users/martinzajac/.pyenv/versions/3.6.8/
/Users/martinzajac/.pyenv/versions/3.6.8/Python.framework/Versions/3.6
/Users/martinzajac/.pyenv/versions/3.6.8/
Traceback (most recent call last):
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/bin/PyInstaller", line 11, in <module>
    sys.exit(run())
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/__main__.py", line 111, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/__main__.py", line 63, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 844, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 791, in build
    exec(code, spec_namespace)
  File "/Users/martinzajac/projects/team/data-team/common/dag_env_check/dag_env_check/dag_env_check.spec", line 17, in <module>
    noarchive=False)
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 243, in __init__
    self.__postinit__()
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/building/datastruct.py", line 158, in __postinit__
    self.assemble()
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 502, in assemble
    module_hook.post_graph()
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/building/imphook.py", line 410, in post_graph
    self._load_hook_module()
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/building/imphook.py", line 377, in _load_hook_module
    self.hook_module_name, self.hook_filename)
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/compat.py", line 793, in importlib_load_source
    return mod_loader.load_module()
  File "<frozen importlib._bootstrap_external>", line 399, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 823, in load_module
  File "<frozen importlib._bootstrap_external>", line 682, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/hooks/hook-sysconfig.py", line 30, in <module>
    datas = [(_CONFIG_H, relpath_to_config_or_make(_CONFIG_H))]
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/utils/hooks/__init__.py", line 808, in relpath_to_config_or_make
    prefix = _find_prefix(filename)
  File "/Users/martinzajac/.pyenv/versions/3.6.8/envs/dag_env_check_installer/lib/python3.6/site-packages/PyInstaller/utils/hooks/__init__.py", line 797, in _find_prefix
    return possible_prefixes[0]
IndexError: list index out of range

The following allowed me to build:

def _find_prefix(filename):
    """
    In virtualenv, _CONFIG_H and _MAKEFILE may have same or different
    prefixes, depending on the version of virtualenv.
    Try to find the correct one, which is assumed to be the longest one.
    """
    if not is_venv:
        return sys.prefix
    filename = os.path.abspath(filename)
    prefixes = [os.path.abspath(sys.prefix), base_prefix]
    possible_prefixes = []
    for prefix in prefixes:
        possible_prefixes.append(prefix)
    possible_prefixes.sort(key=lambda p: len(p), reverse=False)
    return possible_prefixes[0]

[JordanSlaman]

I also experienced this issue and the above code fixed it for me!
Thanks!

[htgoebel]

@poteznyKrolik @JordanSlaman What about providing a pull-request, so others will ot be trapped by the same error again?

[JordanSlaman]

The logic that was breaking in master for me seemed to be resolved by using a shorter path to python.

I'm not sure enough of what the logic should be, but I think the issue here is with the length assumption.

[JordanSlaman]

I'd be weary to make a PR of someone else's bandaid without any understanding or testing of the behaviour.

[mcouthon]

The fix indeed works, and like @JordanSlaman I don't understand why.