Trouble Running Bootloader to Set Up PyInstaller with MinGW

[robert-7]

Brief Goal

I'd like to use PyInstaller to create an executable file for my Python program that uses some C++ and Cython back-end code that was setup / installed using MinGW.

Problem

I'm not sure where I'm stuck with the setup. But I've documented everything I've tried and everything I see.

More Details / Scenario

I'm using virtual machine running Windows 10 (64 bit OS) with Python 2.7.12. I managed to set up my project, Gambit, a program written in C++ and includes some Cython code to allow me to make Python calls. It was built using MinGW. Moreover, I wrote a Python script, gen_tree_manila_poker.py that works with Gambit. Therefore, what I'd like to do is to create a build (a Windows executable). Now, I installed PyInstaller 3.1.1 from archive (I forgot why, but that's what I have).

Attempt 1 (Try to Run PyInstaller Directly without Running Bootloader)

Now, when I run python setup.py install it installs PyInstaller in C:\Python27\Lib\site-packages\PyInstaller-3.1.1-py2.7.egg. Now, when I try to run pyinstaller gen_tree_manila_poker.py, below is the output (which indicates it's using Micrsoft Visual C++ 9.0):

Click to see output for running `pyinstaller gen_tree_manila_poker.py`

$ pyinstaller gen_tree_manila_poker.py 250 INFO: PyInstaller: 3.1.1 250 INFO: Python: 2.7.12 265 INFO: Platform: Windows-10-10.0.14393 343 INFO: wrote c:\Users\Robert\code\github\gambit\src\python\gambit\tests\test_games\personal_test_games\gen_tree_manila_poker.spec 359 INFO: UPX is not available. 608 INFO: Extending PYTHONPATH with paths ['c:\\Users\\Robert\\code\\github\\gambit\\src\\python\\gambit\\tests\\test_games\\personal_test_games', 'c:\\Users\\Robert\\code\\github\\gambit\\src\\python\\gambit\\tests\\test_games\\personal_test_games'] 625 INFO: checking Analysis 641 INFO: Building Analysis because out00-Analysis.toc is non existent 641 INFO: Initializing module dependency graph... 655 INFO: Initializing module graph hooks... 1203 INFO: running Analysis out00-Analysis.toc 1233 INFO: Adding Microsoft.VC90.CRT to dependent assemblies of final executable required by c:\Python27\python.exe 1828 INFO: Found C:\Windows\WinSxS\Manifests\x86_policy.9.0.microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.1_none_8550c6b5d18a9128.manifest 1843 INFO: Found C:\Windows\WinSxS\Manifests\x86_policy.9.0.microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.9247_none_f47e3bf8f656f083.manifest 2406 INFO: Searching for assembly x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.30729.9247_none ... 2500 INFO: Found manifest C:\Windows\WinSxS\Manifests\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.9247_none_5090cb78bcba4a35.manifest 2546 INFO: Searching for file msvcr90.dll 2577 INFO: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.9247_none_5090cb78bcba4a35\msvcr90.dll 2577 INFO: Searching for file msvcp90.dll 2608 INFO: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.9247_none_5090cb78bcba4a35\msvcp90.dll 2608 INFO: Searching for file msvcm90.dll 2608 INFO: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.9247_none_5090cb78bcba4a35\msvcm90.dll 3141 INFO: Found C:\Windows\WinSxS\Manifests\x86_policy.9.0.microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.1_none_8550c6b5d18a9128.manifest 3141 INFO: Found C:\Windows\WinSxS\Manifests\x86_policy.9.0.microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.9247_none_f47e3bf8f656f083.manifest 3155 INFO: Adding redirect Microsoft.VC90.CRT version (9, 0, 21022, 8) -> (9, 0, 30729, 9247) 3531 INFO: Analyzing c:\Users\Robert\code\github\gambit\src\python\gambit\tests\test_games\personal_test_games\gen_tree_manila_poker.py 10375 INFO: Processing pre-find module path hook distutils 14707 INFO: Processing pre-safe import module hook _xmlplus 34593 INFO: Looking for import hooks ... 34657 INFO: Processing hook hook-distutils.py 34657 INFO: Processing hook hook-sysconfig.py 34671 INFO: Processing hook hook-xml.py 35144 INFO: Processing hook hook-lxml.etree.py 35215 INFO: Processing hook hook-httplib.py 35223 INFO: Processing hook hook-pygments.py 44004 INFO: Processing hook hook-_tkinter.py 44800 INFO: checking Tree 44809 INFO: Building Tree because out00-Tree.toc is non existent 44809 INFO: Building Tree out00-Tree.toc 46116 INFO: checking Tree 46132 INFO: Building Tree because out01-Tree.toc is non existent 46136 INFO: Building Tree out01-Tree.toc 46312 INFO: Processing hook hook-pkg_resources.py 46327 INFO: Processing hook hook-encodings.py 56030 INFO: checking Tree 56046 INFO: Building Tree because out02-Tree.toc is non existent 56046 INFO: Building Tree out02-Tree.toc 57847 INFO: checking Tree 57855 INFO: Building Tree because out03-Tree.toc is non existent 57875 INFO: Building Tree out03-Tree.toc 57957 INFO: Looking for ctypes DLLs 58082 INFO: Analyzing run-time hooks ... 58152 INFO: Including run-time hook 'pyi_rth_pkgres.py' 58190 INFO: Including run-time hook 'pyi_rth__tkinter.py' 58323 INFO: Looking for dynamic libraries 60027 INFO: Looking for eggs 60034 INFO: Using Python library C:\Windows\system32\python27.dll 60038 INFO: Found binding redirects: [BindingRedirect(name=u'Microsoft.VC90.CRT', language=None, arch=u'x86', oldVersion=(9, 0, 21022, 8), newVersion=(9, 0, 30729, 9247), publicKeyToken=u'1fc8b3b9a1e18e3b')] 60175 INFO: Warnings written to c:\Users\Robert\code\github\gambit\src\python\gambit\tests\test_games\personal_test_games\build\gen_tree_manila_poker\warngen_tree_manila_poker.txt 62188 INFO: checking PYZ 62188 INFO: Building PYZ because out00-PYZ.toc is non existent 62188 INFO: Building PYZ (ZlibArchive) c:\Users\Robert\code\github\gambit\src\python\gambit\tests\test_games\personal_test_games\build\gen_tree_manila_poker\out00-PYZ.pyz 67359 INFO: checking PKG 67375 INFO: Building PKG because out00-PKG.toc is non existent 67383 INFO: Building PKG (CArchive) out00-PKG.pkg 67679 INFO: Bootloader c:\Python27\lib\site-packages\pyinstaller-3.1.1-py2.7.egg\PyInstaller\bootloader\Windows-32bit\run.exe 67691 INFO: checking EXE 67697 INFO: Building EXE because out00-EXE.toc is non existent 67707 INFO: Building EXE from out00-EXE.toc 67738 INFO: Appending archive to EXE c:\Users\Robert\code\github\gambit\src\python\gambit\tests\test_games\personal_test_games\build\gen_tree_manila_poker\gen_tree_manila_poker.exe 68379 INFO: checking COLLECT 68388 INFO: Building COLLECT because out00-COLLECT.toc is non existent 68398 INFO: Building COLLECT out00-COLLECT.toc 68988 INFO: Redirecting Microsoft.VC90.CRT version (9, 0, 21022, 8) -> (9, 0, 30729, 9247)

When I run the executable I get a 'six' is required error which I "solved" by jumping down to downgrading setuptools to 19.2 using pip install setuptools==19.2. Then I tried it again and see I'm mssing DLLs

Click to see output running the .exe

$ gen_tree_manila_poker.exe > output.txt Traceback (most recent call last): File "", line 13, in File "c:\python27\lib\site-packages\PyInstaller-3.1.1-py2.7.egg\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module exec(bytecode, module.__dict__) File "build\bdist.win32\egg\gambit\__init__.py", line 23, in File "c:\python27\lib\site-packages\PyInstaller-3.1.1-py2.7.egg\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module exec(bytecode, module.__dict__) File "build\bdist.win32\egg\gambit\lib\libgambit.py", line 7, in File "build\bdist.win32\egg\gambit\lib\libgambit.py", line 6, in __bootstrap__ ImportError: DLL load failed: The specified module could not be found. gen_tree_manila_poker returned -1

So at this point, I'm not sure what DLLs I'm missing... Help???

Attempt 2 (Realize I Might need to run the bootloader)

So, I'm trying to build with MinGW and now MSVC 9.0. So, in the /bootloader directory, I would need to run python ./waf distclean all while ensuring MinGW is on the path (done with C:/MinGW/bin). (QUESTION: MINGW EVEN SUPPORTED BECAUSE PYINSTALLER LINKS REFERENCES MINGW-W64). So, I ran python ./waf distclean all and I got a gcc.exe: error: unrecognized command line option '-municode' error (log below):

Click to see output running `python ./waf distclean all`

$ python ./waf distclean all 'distclean' finished successfully (0.031s) 'all' finished successfully (0.000s) 'distclean' finished successfully (0.000s) Setting top to : c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader Setting out to : c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build Platform : Windows-32bit detected Checking for 'msvc' (C compiler) : not found Checking for 'gcc' (C compiler) : C:\MinGW\bin\gcc.exe Checking for program 'windres' : C:\MinGW\bin\windres.exe Checking for flags -m32 : yes Checking for program 'strip' : C:\MinGW\bin\strip.exe Checking for library user32 : yes Checking for library comctl32 : yes Checking for library kernel32 : yes Checking for library ws2_32 : yes Checking for function unsetenv : not found 'configure' finished successfully (3.304s) Waf: Entering directory `c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug' [ 1/23] Compiling zlib\adler32.c gcc.exe: error: unrecognized command line option '-municode'

Waf: Leaving directory `c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug'
Build failed
-> task in 'zlib' failed (exit status 1):
{task 46035344: c adler32.c -> adler32.c.1.o}
['C:\MinGW\bin\gcc.exe', '-m32', '-O2', '-Wdeclaration-after-statement', '-Wimplicit-function-declaration', '-Werror', '-mms-bitfields', '-municode', '-Ic:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug\zlib', '-Ic:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\zlib', '-DWIN32', '-DLAUNCH_DEBUG', '-DNDEBUG', '..\..\zlib\adler32.c', '-c', '-o', 'c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug\zlib\adler32.c.1.o']

A friend recommended just commenting out some of commands in wscript that were adding parameters causing it to fail (in this case, the -municode parameter). So I commented out:

            # Define UNICODE and _UNICODE for wchar_t WinAPI
            ctx.env.append_value('CFLAGS', '-municode')

            # Use Unicode entry point wmain/wWinMain
            ctx.env.append_value('LINKFLAGS', '-municode')

Rerunning gave me an cc1.exe: all warnings being treated as errors error

Click to see output running `python ./waf distclean all`

$ python ./waf distclean all 'distclean' finished successfully (0.020s) 'all' finished successfully (0.000s) 'distclean' finished successfully (0.004s) Setting top to : c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader Setting out to : c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build Platform : Windows-32bit detected Checking for 'msvc' (C compiler) : not found Checking for 'gcc' (C compiler) : C:\MinGW\bin\gcc.exe Checking for program 'windres' : C:\MinGW\bin\windres.exe Checking for flags -m32 : yes Checking for program 'strip' : C:\MinGW\bin\strip.exe Checking for library user32 : yes Checking for library comctl32 : yes Checking for library kernel32 : yes Checking for library ws2_32 : yes Checking for function unsetenv : not found 'configure' finished successfully (2.544s) Waf: Entering directory `c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug' [ 1/23] Compiling zlib\adler32.c [ 2/23] Compiling zlib\crc32.c [ 3/23] Compiling zlib\inffast.c [ 4/23] Compiling zlib\inflate.c [ 5/23] Compiling zlib\inftrees.c [ 6/23] Compiling zlib\ioapi.c [ 7/23] Compiling zlib\unzip.c [ 8/23] Compiling zlib\zutil.c [ 9/23] Linking build\debug\libstatic_zlib.a [10/23] Compiling src\main.c [11/23] Compiling src\pyi_archive.c [12/23] Compiling src\pyi_global.c [13/23] Compiling src\pyi_launch.c [14/23] Compiling src\pyi_main.c ..\..\src\pyi_main.c: In function 'pyi_main': ..\..\src\pyi_main.c:117:9: error: implicit declaration of function 'SetDllDirectory' [-Werror=implicit-function-declaration] SetDllDirectory(dllpath_w); ^ cc1.exe: all warnings being treated as errors

Waf: Leaving directory `c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug'
Build failed
-> task in 'run_d' failed (exit status 1):
{task 50959280: c pyi_main.c -> pyi_main.c.2.o}
['C:\MinGW\bin\gcc.exe', '-m32', '-O2', '-Wdeclaration-after-statement', '-Wimplicit-function-declaration', '-Werror', '-mms-bitfields', '-Ic:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug\src', '-Ic:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\src', '-Ic:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug\windows', '-Ic:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\windows', '-Ic:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug\zlib', '-Ic:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\zlib', '-DWIN32', '-DLAUNCH_DEBUG', '-DNDEBUG', '..\..\src\pyi_main.c', '-c', '-o', 'c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug\src\pyi_main.c.2.o']

So I commented out:

ctx.env.append_value('CFLAGS', '-Werror')

And I run it one more time only to get an undefined reference to WinMain@16 error, seen below:

Click to see output running `python ./waf distclean all`

$ python ./waf distclean all 'distclean' finished successfully (0.031s) 'all' finished successfully (0.000s) 'distclean' finished successfully (0.000s) Setting top to : c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader Setting out to : c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build Platform : Windows-32bit detected Checking for 'msvc' (C compiler) : not found Checking for 'gcc' (C compiler) : C:\MinGW\bin\gcc.exe Checking for program 'windres' : C:\MinGW\bin\windres.exe Checking for flags -m32 : yes Checking for program 'strip' : C:\MinGW\bin\strip.exe Checking for library user32 : yes Checking for library comctl32 : yes Checking for library kernel32 : yes Checking for library ws2_32 : yes Checking for function unsetenv : not found 'configure' finished successfully (3.296s) Waf: Entering directory `c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug' [ 1/23] Compiling zlib\adler32.c [ 2/23] Compiling zlib\crc32.c [ 3/23] Compiling zlib\inffast.c [ 4/23] Compiling zlib\inflate.c [ 5/23] Compiling zlib\inftrees.c [ 6/23] Compiling zlib\ioapi.c [ 7/23] Compiling zlib\unzip.c [ 8/23] Compiling zlib\zutil.c [ 9/23] Linking build\debug\libstatic_zlib.a [10/23] Compiling src\main.c [11/23] Compiling src\pyi_archive.c [12/23] Compiling src\pyi_global.c [13/23] Compiling src\pyi_launch.c [14/23] Compiling src\pyi_main.c ..\..\src\pyi_main.c: In function 'pyi_main': ..\..\src\pyi_main.c:117:9: warning: implicit declaration of function 'SetDllDirectory' [-Wimplicit-function-declaration] SetDllDirectory(dllpath_w); ^

[15/23] Compiling src\pyi_path.c
[16/23] Compiling src\pyi_python.c
[17/23] Compiling src\pyi_python27_compat.c
[18/23] Compiling src\pyi_pythonlib.c
[19/23] Compiling src\pyi_utils.c
[20/23] Compiling src\pyi_win32_utils.c
[21/23] Compiling windows\run.rc
[22/23] Linking build\debug\run_d.exe
c:/mingw/bin/../lib/gcc/mingw32/5.3.0/../../../libmingw32.a(main.o):(.text.startup+0xa0): undefined reference to WinMain@16' src\pyi_main.c.2.o:pyi_main.c:(.text+0x12b): undefined reference to SetDllDirectory'
src\pyi_main.c.2.o:pyi_main.c:(.text+0x40a): undefined reference to `SetDllDirectory'
collect2.exe: error: ld returned 1 exit status

Waf: Leaving directory `c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug'
Build failed
-> task in 'run_d' failed (exit status 1):
{task 46249488: cprogram main.c.2.o,pyi_archive.c.2.o,pyi_global.c.2.o,pyi_launch.c.2.o,pyi_main.c.2.o,pyi_path.c.2.o,pyi_python.c.2.o,pyi_python27_compat.c.2.o,pyi_pythonlib.c.2.o,pyi_utils.c.2.o,pyi_win32_utils.c.2.o,run.rc.o -> run_d.exe}
['C:\MinGW\bin\gcc.exe', '-Wl,--enable-auto-import', '-m32', '-Wl,--large-address-aware', '-lmingw32', '-mwindows', 'src\main.c.2.o', 'src\pyi_archive.c.2.o', 'src\pyi_global.c.2.o', 'src\pyi_launch.c.2.o', 'src\pyi_main.c.2.o', 'src\pyi_path.c.2.o', 'src\pyi_python.c.2.o', 'src\pyi_python27_compat.c.2.o', 'src\pyi_pythonlib.c.2.o', 'src\pyi_utils.c.2.o', 'src\pyi_win32_utils.c.2.o', 'windows\run.rc.o', '-o', 'c:\Users\Robert\code\github\PyInstaller-3.1.1\PyInstaller-3.1.1\bootloader\build\debug\run_d.exe', '-Wl,-Bstatic', '-L.', '-lstatic_zlib', '-Wl,-Bdynamic', '-luser32', '-lcomctl32', '-lkernel32', '-lws2_32']

Anyways, at this point, I've tried fixing this by adding ctx.env.append_value('LINKFLAGS', '-lmingw32') somewhere in the wscript, but it's safe to say I'm stuck, exhausted all ideas (and hope), and just need help.

What I Need

Help understanding and getting past this / fixing problems. Please comment if I forgot to add something or need to explain something!

[htgoebel]

Your first atempt is the correct one. If you get errors for missing DLLs not all required files are packages. Please have a look at Make sure everything is packaged correctly for tips to find out what is missing.

For a more detailed understanding of PyInstaller I suggest reading the manual.

[htgoebel]

New new on this since 8 months, I assume you solve the problem. Otherwise I suggest reading the chapter about building the bootloader in the manual, it has been update a lot recently.