New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing `warnings` module in base_library.zip #3397

Closed
mnencia opened this Issue Mar 12, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@mnencia
Contributor

mnencia commented Mar 12, 2018

When you use the -W option to python interpreter some of the modules present in base_library.zip requires the warnings module, so every time you execute a --onefile bundle you obtain the following error during the initial load phase.

'import warnings' failed; traceback:
ModuleNotFoundError: No module named 'warnings'

Here is a minimal test case

# testwarnings.py
import sys

if sys.warnoptions:
    for wopt in sys.warnoptions:
        print(wopt)
# testwarnings.spec
# -*- mode: python -*-

block_cipher = None


a = Analysis(['testwarnings.py'],
             pathex=['/Users/mnencia/tmp/x/src'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          [
              ('W ignore:some_message', None, 'OPTION'),
              ('v', None, 'OPTION'),
          ],
          a.binaries,
          a.zipfiles,
          a.datas,
          name='testwarnings',
          debug=False,
          strip=False,
          upx=True,
          runtime_tmpdir=None,
          console=True )

and here is the build log

$ pyinstaller testwarnings.spec
91 INFO: PyInstaller: 3.3.1
92 INFO: Python: 3.6.4
102 INFO: Platform: Darwin-17.4.0-x86_64-i386-64bit
105 INFO: UPX is not available.
108 INFO: Extending PYTHONPATH with paths
['/Users/mnencia/tmp/x/src', '/Users/mnencia/tmp/x/src']
108 INFO: checking Analysis
108 INFO: Building Analysis because out00-Analysis.toc is non existent
108 INFO: Initializing module dependency graph...
110 INFO: Initializing module graph hooks...
114 INFO: Analyzing base_library.zip ...
4133 INFO: running Analysis out00-Analysis.toc
4150 INFO: Caching module hooks...
4155 INFO: Analyzing testwarnings.py
4160 INFO: Loading module hooks...
4160 INFO: Loading module hook "hook-encodings.py"...
4275 INFO: Loading module hook "hook-xml.py"...
4611 INFO: Loading module hook "hook-pydoc.py"...
4628 INFO: Looking for ctypes DLLs
4628 INFO: Analyzing run-time hooks ...
4640 INFO: Looking for dynamic libraries
4753 INFO: Looking for eggs
4754 INFO: Using Python library /usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/Python
4756 INFO: Warnings written to /Users/mnencia/tmp/x/src/build/testwarnings/warntestwarnings.txt
4800 INFO: Graph cross-reference written to /Users/mnencia/tmp/x/src/build/testwarnings/xref-testwarnings.html
4810 INFO: checking PYZ
4810 INFO: Building PYZ because out00-PYZ.toc is non existent
4810 INFO: Building PYZ (ZlibArchive) /Users/mnencia/tmp/x/src/build/testwarnings/out00-PYZ.pyz
5183 INFO: Building PYZ (ZlibArchive) /Users/mnencia/tmp/x/src/build/testwarnings/out00-PYZ.pyz completed successfully.
5192 INFO: checking PKG
5193 INFO: Building PKG because out00-PKG.toc is non existent
5193 INFO: Building PKG (CArchive) out00-PKG.pkg
7425 INFO: Building PKG (CArchive) out00-PKG.pkg completed successfully.
7428 INFO: Bootloader /Users/mnencia/tmp/x/lib/python3.6/site-packages/PyInstaller/bootloader/Darwin-64bit/run
7428 INFO: checking EXE
7428 INFO: Building EXE because out00-EXE.toc is non existent
7428 INFO: Building EXE from out00-EXE.toc
7428 INFO: Appending archive to EXE /Users/mnencia/tmp/x/src/dist/testwarnings
7437 INFO: Fixing EXE for code signing /Users/mnencia/tmp/x/src/dist/testwarnings
7445 INFO: Building EXE from out00-EXE.toc completed successfully.

and the execution result (with -v python option)

import _frozen_importlib # frozen
import _imp # builtin
import sys # builtin
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
import _thread # previously loaded ('_thread')
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import _weakref # previously loaded ('_weakref')
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
# installing zipimport hook
import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'>
# installed zipimport hook
# zipimport: found 148 names in '/var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import codecs # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/codecs.pyc
import encodings.aliases # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/encodings/aliases.pyc
import encodings # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/encodings/__init__.pyc
import encodings.utf_8 # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/encodings/utf_8.pyc
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
import encodings.latin_1 # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/encodings/latin_1.pyc
import _weakrefset # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/_weakrefset.pyc
import abc # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/abc.pyc
import io # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/io.pyc
import '_locale' # <class '_frozen_importlib.BuiltinImporter'>
import _bootlocale # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/_bootlocale.pyc
'import warnings' failed; traceback:
ignore:some_message
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'warnings'
# extension module '_struct' loaded from '/var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/_struct.so'
# extension module '_struct' executed from '/var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/_struct.so'
import '_struct' # <_frozen_importlib_external.ExtensionFileLoader object at 0x102b96eb8>
# extension module 'zlib' loaded from '/var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/zlib.so'
# extension module 'zlib' executed from '/var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/zlib.so'
import 'zlib' # <_frozen_importlib_external.ExtensionFileLoader object at 0x102b9ae80>
# PyInstaller: FrozenImporter(/Users/mnencia/tmp/x/src/dist/testwarnings?3486102)
import os # PyInstaller PYZ
import 'errno' # <class '_frozen_importlib.BuiltinImporter'>
import stat # PyInstaller PYZ
import '_stat' # <class '_frozen_importlib.BuiltinImporter'>
import 'stat' # <pyimod03_importers.FrozenImporter object at 0x102bb4240>
import posixpath # PyInstaller PYZ
import genericpath # PyInstaller PYZ
import 'genericpath' # <pyimod03_importers.FrozenImporter object at 0x102bb4240>
import 'posixpath' # <pyimod03_importers.FrozenImporter object at 0x102bb4240>
# _collections_abc not found in PYZ
import _collections_abc # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/_collections_abc.pyc
import 'os' # <pyimod03_importers.FrozenImporter object at 0x102bb4240>
import warnings # PyInstaller PYZ
# re not found in PYZ
import enum # PyInstaller PYZ
# types not found in PYZ
# functools not found in PYZ
import '_functools' # <class '_frozen_importlib.BuiltinImporter'>
# collections not found in PYZ
# operator not found in PYZ
import '_operator' # <class '_frozen_importlib.BuiltinImporter'>
import operator # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/operator.pyc
# keyword not found in PYZ
import keyword # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/keyword.pyc
# heapq not found in PYZ
# _heapq not found in PYZ
# extension module '_heapq' loaded from '/var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/_heapq.so'
# extension module '_heapq' executed from '/var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/_heapq.so'
import '_heapq' # <_frozen_importlib_external.ExtensionFileLoader object at 0x1030c7e48>
import heapq # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/heapq.pyc
import 'itertools' # <class '_frozen_importlib.BuiltinImporter'>
# reprlib not found in PYZ
import reprlib # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/reprlib.pyc
import '_collections' # <class '_frozen_importlib.BuiltinImporter'>
import collections # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/collections/__init__.pyc
# weakref not found in PYZ
import weakref # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/weakref.pyc
import functools # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/functools.pyc
# collections.abc not found in PYZ
import collections.abc # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/collections/abc.pyc
import types # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/types.pyc
import 'enum' # <pyimod03_importers.FrozenImporter object at 0x102bb4240>
# sre_compile not found in PYZ
import '_sre' # <class '_frozen_importlib.BuiltinImporter'>
# sre_parse not found in PYZ
# sre_constants not found in PYZ
import sre_constants # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/sre_constants.pyc
import sre_parse # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/sre_parse.pyc
import sre_compile # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/sre_compile.pyc
# copyreg not found in PYZ
import copyreg # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/copyreg.pyc
import re # loaded from Zip /var/folders/tg/yy01xlp964s0wwyn_16fbsfm0000gn/T/_MEIjn2gLA/base_library.zip/re.pyc
import 'warnings' # <pyimod03_importers.FrozenImporter object at 0x102bb4240>
# ctypes not found in PYZ
# ctypes not found in PYZ
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.__interactivehook__
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup[2] removing builtins
# cleanup[2] removing sys
# cleanup[2] removing _frozen_importlib
# cleanup[2] removing _imp
# cleanup[2] removing _warnings
# cleanup[2] removing _thread
# cleanup[2] removing _weakref
# cleanup[2] removing _frozen_importlib_external
# cleanup[2] removing _io
# cleanup[2] removing marshal
# cleanup[2] removing posix
# cleanup[2] removing zipimport
# cleanup[2] removing encodings
# cleanup[2] removing codecs
# cleanup[2] removing _codecs
# cleanup[2] removing encodings.aliases
# cleanup[2] removing encodings.utf_8
# cleanup[2] removing _signal
# cleanup[2] removing __main__
# destroy __main__
# cleanup[2] removing encodings.latin_1
# cleanup[2] removing io
# destroy io
# cleanup[2] removing abc
# cleanup[2] removing _weakrefset
# destroy _weakrefset
# cleanup[2] removing _bootlocale
# destroy _bootlocale
# cleanup[2] removing _locale
# cleanup[2] removing struct
# cleanup[2] removing _struct
# cleanup[2] removing pyimod01_os_path
# cleanup[2] removing pyimod02_archive
# cleanup[2] removing zlib
# cleanup[2] removing pyimod03_importers
# cleanup[2] removing os
# cleanup[2] removing errno
# cleanup[2] removing stat
# cleanup[2] removing _stat
# cleanup[2] removing posixpath
# cleanup[2] removing genericpath
# cleanup[2] removing os.path
# cleanup[2] removing _collections_abc
# cleanup[2] removing warnings
# cleanup[2] removing re
# destroy re
# cleanup[2] removing enum
# cleanup[2] removing types
# destroy types
# cleanup[2] removing functools
# cleanup[2] removing _functools
# cleanup[2] removing collections
# cleanup[2] removing operator
# destroy operator
# cleanup[2] removing _operator
# cleanup[2] removing keyword
# destroy keyword
# cleanup[2] removing heapq
# cleanup[2] removing _heapq
# cleanup[2] removing itertools
# cleanup[2] removing reprlib
# destroy reprlib
# cleanup[2] removing _collections
# cleanup[2] removing weakref
# destroy weakref
# cleanup[2] removing collections.abc
# cleanup[2] removing sre_compile
# cleanup[2] removing _sre
# cleanup[2] removing sre_parse
# cleanup[2] removing sre_constants
# destroy sre_constants
# cleanup[2] removing copyreg
# destroy zipimport
# destroy _signal
# destroy encodings
# destroy warnings
# destroy abc
# destroy errno
# destroy posixpath
# destroy _stat
# destroy genericpath
# destroy os
# destroy stat
# destroy enum
# destroy sre_compile
# destroy _locale
# destroy copyreg
# destroy sre_parse
# destroy _sre
# destroy functools
# destroy _functools
# destroy _collections_abc
# destroy heapq
# destroy collections.abc
# destroy _operator
# destroy _heapq
# destroy _collections
# destroy collections
# destroy itertools
# cleanup[3] wiping _frozen_importlib
# destroy _frozen_importlib_external
# cleanup[3] wiping _imp
# cleanup[3] wiping _warnings
# cleanup[3] wiping _thread
# cleanup[3] wiping _weakref
# cleanup[3] wiping _io
# cleanup[3] wiping marshal
# cleanup[3] wiping posix
# cleanup[3] wiping codecs
# cleanup[3] wiping _codecs
# cleanup[3] wiping encodings.aliases
# cleanup[3] wiping encodings.utf_8
# cleanup[3] wiping encodings.latin_1
# cleanup[3] wiping struct
# cleanup[3] wiping _struct
# destroy _struct
# cleanup[3] wiping pyimod01_os_path
# cleanup[3] wiping pyimod02_archive
# cleanup[3] wiping zlib
# destroy zlib
# cleanup[3] wiping pyimod03_importers
# cleanup[3] wiping sys
# cleanup[3] wiping builtins
# destroy _imp
# destroy io
# destroy _warnings
# destroy posix
# destroy _thread
# destroy _weakref
# destroy _frozen_importlib

If I add the following code on top of the spec file it solves the issue and the error disappears:

# fix for missing `warnings` module during base_library.zip loading
from PyInstaller.compat import PY3_BASE_MODULES
PY3_BASE_MODULES.update({
    'warnings',
})
@htgoebel

This comment has been minimized.

Member

htgoebel commented Mar 12, 2018

Thanks for this detailed bug report. I verified this also happens with Python 3.4.

We would appreciate if you'd submit a pull-request for this. See our Development Guide on how to create pull-requests for PyInstaller. Thanks.

mnencia added a commit to mnencia/pyinstaller that referenced this issue Mar 12, 2018

Add missing warnings module in base_library.zip
When you use the -W option to python interpreter some of the modules
present in base_library.zip requires the warnings module, raising an
error. The issue is solved by adding it to the PY3_BASE_MODULES set.

Closes: pyinstaller#3397
@mnencia

This comment has been minimized.

Contributor

mnencia commented Mar 12, 2018

Pull request submitted

htgoebel added a commit that referenced this issue Mar 13, 2018

compat: Add module 'warnings' to base_library.zip.
When you use the -W option to python interpreter some of the modules
present in base_library.zip requires the warnings module, raising an
error. The issue is solved by adding it to the PY3_BASE_MODULES set.

Closes: #3397

@htgoebel htgoebel added this to the PyInstaller 3.4 milestone Sep 2, 2018

cowo78 pushed a commit to cowo78/pyinstaller that referenced this issue Dec 7, 2018

compat: Add module 'warnings' to base_library.zip.
When you use the -W option to python interpreter some of the modules
present in base_library.zip requires the warnings module, raising an
error. The issue is solved by adding it to the PY3_BASE_MODULES set.

Closes: pyinstaller#3397
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment