"import configparser" broke load ConfigParser

[ombschervister]

I’ve found an apparent bug.
Downloaded winpython 2.7.10.104, installed with pip the following libraries:

pip install pyqode.core
pip install pyqode.qt

Then if I try to use collect_submodules(‘PyQt4’) in the spec-file,
ConfigParser import is not working

Error Messages:
In the master branch:

Traceback (most recent call last):
  File "<string>", line 5, in <module>
ImportError: No module named ConfigParser
run returned -1

In the develop branch:

Traceback(most recent call last):
    File "(string)", line 1, in <module>
NameError: name 'c' is not defined
pyiboot01_bootstrap returned -1

Here the code:
spec-file:

# -*- mode: python -*-
block_cipher = None
# collect_submodules('PyQt4') broke ConfigParser
from PyInstaller.utils.hooks import collect_submodules

a = Analysis(['run.py'],
             pathex=[],
             binaries=None,
             datas=None,
             hiddenimports=collect_submodules('PyQt4'),
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='run',
          debug=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               name='run')

python-file

# coding=utf-8
import sys
from PyQt4 import QtGui
import rc
import ConfigParser


class Widget(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)

        png_label = QtGui.QLabel()
        svg_label = QtGui.QLabel()

        v_layout = QtGui.QVBoxLayout(self)
        v_layout.addWidget(QtGui.QLabel("png:"))
        v_layout.addWidget(png_label)
        v_layout.addWidget(QtGui.QLabel("svg:"))
        v_layout.addWidget(svg_label)

        png_label.setPixmap(QtGui.QPixmap(":/example1.png").scaled(200, 200))
        svg_label.setPixmap(QtGui.QPixmap(":/example2.svg").scaled(200, 200))


app = QtGui.QApplication(sys.argv)
w = Widget()
w.show()
sys.exit(app.exec_())

[htgoebel]

If your program already imports PyQt4, why do you need hiddenimports=collect_submodules('PyQt4')? This should be handled by the PyQt4.Hook already.

[ombschervister]

QThread (in conjunction with the subprocess) did not work in my program without calling of collect_submodules('PyQt4'). But it was a few months ago. I will check if they have started to work in the current state of the branch.

[ombschervister]

On this commit the bug is appeared (found with git bisect):

21e25faa05e2ed7483038cf339d386b4ffe00aa7 is the first bad commit
commit 21e25faa05e2ed7483038cf339d386b4ffe00aa7
Author: David Vierra <codewarrior@hawaii.rr.com>
Date:   Sun Nov 22 03:19:19 2015 -1000

TOC: Don't normcase the names in the TOC itself, only the names in the unique-filename set.

:040000 040000 cb1677f95f5d8fe0a48e5548130a6a2e2a263fbb 21e2087983ea52f2a35a7a69c510cd485e7bd9b2 M                PyInstaller

[htgoebel]

Some debugging ideas:

• try a minimal example: import PyQT4, ConfigParser should show the error, too
• try switching both names
• when using the development head you need to compile the bootloader first ATM

[ombschervister]

Ok, I tried minimal example. Error "No module named ConfigParser" raises in any order of imports.

The source code for the test and logs of pyinstaller work and results of the created EXE-file:
https://github.com/ombschervister/pyinstaller_collect_test

[ombschervister]

And even

import ConfigParser
print "success"

does not work when add collect_submodules('PyQt4') to spec-file

[tjstum]

I think the PyQt4 bit is a bit of a red herring. I tried the example program test.py:

import configparser

def main():
    print "success!"

if __name__ == "__main__":
    main()

and that produces the same failure when packaged just with pyinstaller --clean -y test.py:

Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import configparser
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py",
 line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\configparser\__init__.py", line 5, in <module>
ImportError: No module named ConfigParser
Failed to execute script test

I do also see this during the build:

8315 WARNING: Attempted to add Python module twice with different upper/lowercases: ConfigParser
8315 WARNING: Attempted to add Python module twice with different upper/lowercases: ConfigParser

This might be a Windows issue... I was unable to reproduce the minimal example on OS X. ConfigParser and configparser are two different modules (the latter is a backport from Python 3).

[htgoebel]

So you have configparser installed, and, while importing ConfigParser, configparser will be imported by PyInstaller? Huh, this is frightening.

Please try to find out, why this is happening. I assume that for some reason configparser is imported by some other module.

• Please try in a virtaul env where only configparser is installed.
• Please run PyInstaller with --debug. In the build directory you will then find a .html file listing all imports. Try to figur out, what imports configparser-

[ombschervister]

For point "Please try in a virtaul env where only configparser is installed."

I looked at the configparser.

It is a simlpe folder with init.py file:

from __future__ import absolute_import
import sys

if sys.version_info[0] < 3:
    from ConfigParser import *
    try:
        from ConfigParser import (_Chainmap, Error, InterpolationMissingOptionError)
    except ImportError:
        pass
else:
    raise ImportError('This package should not be accessible on Python 3. '
                      'Either you are trying to run from the python-future src folder '
                      'or your installation of python-future is corrupted.')

And any call of configparser in frozen code does not work (raise exception about no module ConfigParser).
For example:

import configparser
print "success"

import configparser
import ConfigParser
print "success"

import ConfigParser
import configparser
print "success"

Simple

import ConfigParser

is work success

So if you import 'ConfigParser' and 'configparser', only 'configparser' will be imported. But 'configparser' use 'ConfigParser', so exception will be raised.

[tjstum]

I did some more investigation, and this is likely an environmental problem. It also appears to be the same issue reported in PythonCharmers/python-future#118

I had, on my machine, a folder named configparser that is how @ombschervister described it. I separately had, in site-packages, a configparser.py module. I think that's the root of the issue.

I was able to get this to work (in that the minimal example built and ran) by just removing the configparser folder. That folder is provided by python-future (see above issue).

I'm not entirely sure what the best way for PyInstaller to detect, workaround or fix this issue. You'd need some sort of conditional hook where "if python-future is installed and configparser is imported, also include ConfigParser and maybe fix some case issues if you're on Windows."

It's possible for authors to work around this by installing the 3.5.0b2 version of configparser and changing the import to be from backports import configparser. I also tested that, and that also makes the test case work.

[ombschervister]

Maybe it is not Windows problem only, but the problem for equal lowercased names of folders and files for any operation system?
For example:

from lib import test as file_test
from LIB import test as folder_test

file_test()
folder_test()

print "success"

lib.py:

def test():
    print "file"

folder LIB with init.py:

def test():
    print "folder"

Maybe so Pyinstaller need to add '' to end of package names in the unique-filename set and does not hook configparser?

[Punkside]

I had a similar case so I tried to use the add_alias_module() function as a pre_safe_import_module hook and it seems to work around this issue.

pre_safe_import_module/hook-configparser.py:

from PyInstaller.compat import is_py2

if is_py2:
    def pre_safe_import_module(psim_api):
        psim_api.add_alias_module('ConfigParser', 'configparser')

The duplicate upper/lowercases warning still remain though.

[stephenrauch]

Changes for #1935 (specifically a680f74) likely resolves the upper/lower warning problem and may well have resolved the rest of this issue. To test you can copy the file referenced by a680f74 into the install directory.

[Punkside]

I did not see those changes and you are right, this correct this issue altogether.

[htgoebel]

Duplicate of #1935