Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pyinstaller crashes when packaging application that uses pyodbc #2492

Closed
nurieady opened this issue Mar 9, 2017 · 2 comments
Closed

Pyinstaller crashes when packaging application that uses pyodbc #2492

nurieady opened this issue Mar 9, 2017 · 2 comments

Comments

@nurieady
Copy link

nurieady commented Mar 9, 2017

Pyinstaller crashes when packaging an application that imports pyodbc. I created a very simple "Hello World" application that simply imports pyodbc...

import pyodbc
print("hello")

Removing the pyodbc import statement allows pyinstaller to run without issue. Below is the log from the crash...

C:\Users\Owner\Seafile\Documents\RSO\SRP\Database_Utils>pyinstaller --log-level DEBUG test.py
176 INFO: PyInstaller: 3.3.dev0+8b491de
176 INFO: Python: 3.4.4
177 INFO: Platform: Windows-10-10.0.14393
178 INFO: wrote C:\Users\Owner\Seafile\Documents\RSO\SRP\Database_Utils\test.spec
179 DEBUG: Testing for UPX ...
181 INFO: UPX is not available.
184 DEBUG: script: C:\Users\Owner\Seafile\Documents\RSO\SRP\Database_Utils\test.py
185 INFO: Extending PYTHONPATH with paths
['C:\\Users\\Owner\\Seafile\\Documents\\RSO\\SRP\\Database_Utils',
 'C:\\Users\\Owner\\Seafile\\Documents\\RSO\\SRP\\Database_Utils']
185 INFO: checking Analysis
186 INFO: Building Analysis because out00-Analysis.toc is non existent
186 INFO: Initializing module dependency graph...
196 INFO: Initializing module graph hooks...
202 INFO: Analyzing base_library.zip ...
203 DEBUG: Collecting submodules for collections
361 DEBUG: collect_submodules - Found submodules: {'collections', 'collections.__main__', 'collections.abc'}
362 DEBUG: Collecting submodules for encodings
514 DEBUG: collect_submodules - Found submodules: {'encodings.cp037', 'encodings.hp_roman8', 'encodings.iso8859_4', 'encodings.mac_romanian', 'encodings.utf_8', 'encodings.undefined', 'encodings.cp424', 'encodings.utf_32_le', 'encodings.hex_codec', 'encodings.euc_jisx0213', 'encodings.johab', 'encodings.cp858', 'encodings.mac_iceland', 'encodings.base64_codec', 'encodings.cp1257', 'encodings.koi8_r', 'encodings.gb18030', 'encodings.cp875', 'encodings.cp437', 'encodings.bz2_codec', 'encodings.cp869', 'encodings.zlib_codec', 'encodings.shift_jisx0213', 'encodings.koi8_u', 'encodings.cp1125', 'encodings.punycode', 'encodings.cp852', 'encodings.latin_1', 'encodings.quopri_codec', 'encodings.cp865', 'encodings.big5', 'encodings.cp864', 'encodings.iso2022_jp_2004', 'encodings.iso8859_15', 'encodings.cp1254', 'encodings.euc_jis_2004', 'encodings.big5hkscs', 'encodings.aliases', 'encodings.iso8859_13', 'encodings.ptcp154', 'encodings.iso8859_3', 'encodings.shift_jis', 'encodings.cp949', 'encodings.iso2022_jp', 'encodings.iso8859_6', 'encodings.iso8859_7', 'encodings.utf_32', 'encodings.cp856', 'encodings.utf_7', 'encodings.cp950', 'encodings.cp720', 'encodings.iso8859_8', 'encodings.iso8859_11', 'encodings.cp862', 'encodings.iso2022_jp_1', 'encodings.cp857', 'encodings.gb2312', 'encodings.cp861', 'encodings.cp1026', 'encodings.cp737', 'encodings.cp932', 'encodings.utf_32_be', 'encodings.iso2022_jp_2', 'encodings.iso2022_jp_ext', 'encodings.cp1253', 'encodings.iso8859_10', 'encodings.mac_latin2', 'encodings.mac_croatian', 'encodings.mac_arabic', 'encodings.ascii', 'encodings.mac_turkish', 'encodings.cp1255', 'encodings.rot_13', 'encodings.cp775', 'encodings.unicode_escape', 'encodings.palmos', 'encodings.iso8859_2', 'encodings.cp1252', 'encodings.raw_unicode_escape', 'encodings', 'encodings.cp866', 'encodings.unicode_internal', 'encodings.euc_jp', 'encodings.mac_farsi', 'encodings.cp1258', 'encodings.euc_kr', 'encodings.cp855', 'encodings.tis_620', 'encodings.mac_greek', 'encodings.iso8859_1', 'encodings.cp1140', 'encodings.cp850', 'encodings.cp863', 'encodings.iso2022_jp_3', 'encodings.cp1256', 'encodings.charmap', 'encodings.cp874', 'encodings.idna', 'encodings.mac_centeuro', 'encodings.cp500', 'encodings.utf_16', 'encodings.iso2022_kr', 'encodings.uu_codec', 'encodings.utf_8_sig', 'encodings.iso8859_16', 'encodings.iso8859_9', 'encodings.mbcs', 'encodings.hz', 'encodings.mac_cyrillic', 'encodings.gbk', 'encodings.cp1250', 'encodings.cp1006', 'encodings.cp273', 'encodings.cp65001', 'encodings.utf_16_be', 'encodings.shift_jis_2004', 'encodings.cp860', 'encodings.iso8859_14', 'encodings.cp1251', 'encodings.iso8859_5', 'encodings.mac_roman', 'encodings.utf_16_le'}
3300 INFO: Processing pre-find module path hook   distutils
5462 DEBUG: Adding python files to base_library.zip
5514 DEBUG: Hidden import: codecs
5515 DEBUG: Hidden import 'codecs' already found
5516 INFO: running Analysis out00-Analysis.toc
5517 DEBUG: Analyzing C:\Python34\python.exe
5542 DEBUG: Adding python34.dll dependency of python.exe from C:\WINDOWS\system32\python34.dll
5549 DEBUG: Skipping KERNEL32.dll dependency of python.exe
5553 DEBUG: Adding MSVCR100.dll dependency of python.exe from C:\WINDOWS\system32\msvcr100.dll
5553 DEBUG: Analyzing C:\WINDOWS\system32\python34.dll
5577 DEBUG: Skipping ADVAPI32.dll dependency of python34.dll
5582 DEBUG: Skipping SHELL32.dll dependency of python34.dll
5586 DEBUG: Skipping USER32.dll dependency of python34.dll
5591 DEBUG: Skipping KERNEL32.dll dependency of python34.dll
5595 DEBUG: Analyzing C:\WINDOWS\system32\msvcr100.dll
5629 DEBUG: Skipping KERNEL32.dll dependency of msvcr100.dll
5632 INFO: Caching module hooks...
5642 INFO: Analyzing C:\Users\Owner\Seafile\Documents\RSO\SRP\Database_Utils\test.py
5648 INFO: Loading module hooks...
5649 INFO: Loading module hook "hook-distutils.py"...
5652 INFO: Loading module hook "hook-pydoc.py"...
5653 INFO: Loading module hook "hook-encodings.py"...
5654 DEBUG: Collecting submodules for encodings
5805 DEBUG: collect_submodules - Found submodules: {'encodings.cp037', 'encodings.hp_roman8', 'encodings.iso8859_4', 'encodings.mac_romanian', 'encodings.utf_8', 'encodings.undefined', 'encodings.cp424', 'encodings.utf_32_le', 'encodings.hex_codec', 'encodings.euc_jisx0213', 'encodings.johab', 'encodings.cp858', 'encodings.mac_iceland', 'encodings.base64_codec', 'encodings.cp1257', 'encodings.koi8_r', 'encodings.gb18030', 'encodings.cp875', 'encodings.cp437', 'encodings.bz2_codec', 'encodings.cp869', 'encodings.zlib_codec', 'encodings.shift_jisx0213', 'encodings.koi8_u', 'encodings.cp1125', 'encodings.punycode', 'encodings.cp852', 'encodings.latin_1', 'encodings.quopri_codec', 'encodings.cp865', 'encodings.big5', 'encodings.cp864', 'encodings.iso2022_jp_2004', 'encodings.iso8859_15', 'encodings.cp1254', 'encodings.euc_jis_2004', 'encodings.big5hkscs', 'encodings.aliases', 'encodings.iso8859_13', 'encodings.ptcp154', 'encodings.iso8859_3', 'encodings.shift_jis', 'encodings.cp949', 'encodings.iso2022_jp', 'encodings.iso8859_6', 'encodings.iso8859_7', 'encodings.utf_32', 'encodings.cp856', 'encodings.utf_7', 'encodings.cp950', 'encodings.cp720', 'encodings.iso8859_8', 'encodings.iso8859_11', 'encodings.cp862', 'encodings.iso2022_jp_1', 'encodings.cp857', 'encodings.gb2312', 'encodings.cp861', 'encodings.cp1026', 'encodings.cp737', 'encodings.cp932', 'encodings.utf_32_be', 'encodings.iso2022_jp_2', 'encodings.iso2022_jp_ext', 'encodings.cp1253', 'encodings.iso8859_10', 'encodings.mac_latin2', 'encodings.mac_croatian', 'encodings.mac_arabic', 'encodings.ascii', 'encodings.mac_turkish', 'encodings.cp1255', 'encodings.rot_13', 'encodings.cp775', 'encodings.unicode_escape', 'encodings.palmos', 'encodings.iso8859_2', 'encodings.cp1252', 'encodings.raw_unicode_escape', 'encodings', 'encodings.cp866', 'encodings.unicode_internal', 'encodings.euc_jp', 'encodings.mac_farsi', 'encodings.cp1258', 'encodings.euc_kr', 'encodings.cp855', 'encodings.tis_620', 'encodings.mac_greek', 'encodings.iso8859_1', 'encodings.cp1140', 'encodings.cp850', 'encodings.cp863', 'encodings.iso2022_jp_3', 'encodings.cp1256', 'encodings.charmap', 'encodings.cp874', 'encodings.idna', 'encodings.mac_centeuro', 'encodings.cp500', 'encodings.utf_16', 'encodings.iso2022_kr', 'encodings.uu_codec', 'encodings.utf_8_sig', 'encodings.iso8859_16', 'encodings.iso8859_9', 'encodings.mbcs', 'encodings.hz', 'encodings.mac_cyrillic', 'encodings.gbk', 'encodings.cp1250', 'encodings.cp1006', 'encodings.cp273', 'encodings.cp65001', 'encodings.utf_16_be', 'encodings.shift_jis_2004', 'encodings.cp860', 'encodings.iso8859_14', 'encodings.cp1251', 'encodings.iso8859_5', 'encodings.mac_roman', 'encodings.utf_16_le'}
5816 INFO: Loading module hook "hook-pyodbc.py"...
6320 INFO: Loading module hook "hook-xml.py"...
6732 DEBUG: Looking for egg data files...
6732 INFO: Looking for ctypes DLLs
6734 DEBUG: Scanning pyodbc for shared libraries or dlls
Traceback (most recent call last):
  File "C:\Python34\Scripts\pyinstaller-script.py", line 9, in <module>
    load_entry_point('PyInstaller==3.3.dev0+8b491de', 'console_scripts', 'pyinstaller')()
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\__main__.py", line 87, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\__main__.py", line 45, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\building\build_main.py", line 788, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\building\build_main.py", line 734, in build
    exec(text, spec_namespace)
  File "<string>", line 16, in <module>
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\building\build_main.py", line 212, in __init__
    self.__postinit__()
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\building\datastruct.py", line 161, in __postinit__
    self.assemble()
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\building\build_main.py", line 505, in assemble
    ctypes_binaries = scan_code_for_ctypes(co)
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\depend\utils.py", line 104, in scan_code_for_ctypes
    __recursivly_scan_code_objects_for_ctypes(co, binaries)
  File "C:\Python34\lib\site-packages\pyinstaller-3.3.dev0+8b491de-py3.4.egg\PyInstaller\depend\utils.py", line 130, in __recursivly_scan_code_objects_for_ctypes
    instructions = modulegraph.get_instructions(co)
  File "C:\Python34\lib\dis.py", line 221, in get_instructions
    co = _get_code_object(x)
  File "C:\Python34\lib\dis.py", line 112, in _get_code_object
    type(x).__name__)
TypeError: don't know how to disassemble NoneType objects
@htgoebel
Copy link
Member

htgoebel commented Mar 9, 2017

Hm, this should not happen: It looks like PyInstaller tries to analyse pyodbc's byte-code, which does not exist since this module is a C-extension. PyInstaller tries to detect the modules imported by C-extensions, and maybe pyodbc (indirectly) imports ctypes - which would explain why pyodbc is recogniced as "imports ctypes". But in PyiModuleGraph.get_co_using_ctypes() the extensions ought to be filtered out.

I can't reproduce this here since I don't have Windows.

Can you please

  • start PyInstaller under debugger control python -m pdb ./pyinstaller.py ...
  • set a breakpoint break PyInstaller/depend/analysis.py:535
  • start running with c (continue)
  • when the breakpoint is reached, print referers and try debug, why pyodbc is not filtered out, esp. what it pyobdc's type(node).__name__?

@nurieady
Copy link
Author

nurieady commented Mar 9, 2017 via email

@htgoebel htgoebel self-assigned this Mar 10, 2017
@htgoebel htgoebel added this to the PyInstaller 3.3 milestone Mar 10, 2017
htgoebel added a commit to htgoebel/pyinstaller that referenced this issue Mar 10, 2017
… to ctypes.

Includes test-case.

The fix (see two commits back, 17a49a9)
raised another issue: ctype imports from the main script were ignored,
since "Script" is not part of "PURE_PYTHON_MODULE_TYPES".
htgoebel added a commit to htgoebel/pyinstaller that referenced this issue Mar 11, 2017
In CI tests are run with --debug, which will make the graph and xref
files to be written- For this the filenames need to be defined :-(
htgoebel added a commit to htgoebel/pyinstaller that referenced this issue Mar 11, 2017
… to ctypes.

Includes test-case.

The fix (see two commits back, 17a49a9)
raised another issue: ctype imports from the main script were ignored,
since "Script" is not part of "PURE_PYTHON_MODULE_TYPES".
htgoebel added a commit to htgoebel/pyinstaller that referenced this issue Mar 11, 2017
In CI tests are run with --debug, which will make the graph and xref
files to be written- For this the filenames need to be defined :-(
htgoebel pushed a commit that referenced this issue Mar 13, 2017
Yet another CONF value that needs to be defined.

Also see 55a3936.
Closes #2499, closes #2498.
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants