Skip to content

nose install fails w/ traceback using python 3.3.0 #637

Closed
maveryKearney opened this Issue Mar 1, 2013 · 5 comments

2 participants

@maveryKearney

Contents: ( I know ... scary that a bug report requires a table of contents)

  • Overview
  • code snippet
  • Traceback

Overview

nose installation fails via either "easy_install nose" and "python setup.py install"
on python 3.3.0.

"TypeError: can't use a string pattern on a bytes-like object"

See full traceback at end,

I am running on OS windows 7. I installed Python 3.3.0 via the windows msi install.
I have added only the latest distribution and virtualenv packages in addition and
w/o problems. For comparison purposes I followed the same process with python 2.7.
I also experienced the problem with a compiled version of python 3.3.0
Am I leaving information out? It is hard to believe I am the first one
to install nose with python 3.3.0

Both nose 1.2.1 and nose development version 1.3.0 (nose-devs-nose-0ac507f) exhibit the problem for me.

The problematic code is in setup.py. The pertinent snippet(see below) starts at line setup.py:43. The snippet includes a patch which makes the problem go away. I suspect it might not be valid. I think I have experienced the infinite loop problem described in the code commenton python 2.7 and 3.3.0.

The code opens "script_name" and looks for a bit of text to replace. re expects text, but some of files passed to this code are .exe files and "contents" is in undecoded bytes. I tried decoding the bytes. UTF-8? I guessed wrong. Though I could have made a mistake.
Perhaps I'll get smarter later.

When nose is installed on python3.3.0 4 files are scanned, and the code fails on .exe.
The "solution" is to ignore .exe files.
nosetests-script.py
nosetests.exe
nosetests-3.3-script.py
nosetests-3.3.exe

When nose is installed on python2.7.0 passes in slightly different set of files but doesn't
fail on anything. Is the encoding known? I didn't run across it while fussing with this.
nosetests.exe
nosetests.exe.manifest
nosetests-2.7-script.py
nosetests-2.7.exe
nosetests-2.7.exe.manifest

SO I have a questionable work around for now. The tests that come with 1.2.1 nose don't seem to work for me on either 2.7 or 3.3. Perhaps I am not selecting the proper options test options. I see some reporting of test instability in the bug reports.

Code snippet:

# This is required by multiprocess plugin; on Windows, if
# the launch script is not import-safe, spawned processes
# will re-run it, resulting in an infinite loop.
if sys.platform == 'win32':
    import re
    from setuptools.command.easy_install import easy_install

    def wrap_write_script(self, script_name, contents, *arg, **kwarg):
#       import pdb; pdb.set_trace()
        (unused,tmpExt)= os.path.splitext(script_name)    #py33
        if tmpExt!='.py':                                 #py33
            return                                        #py33
        bad_text = re.compile(
            "\n"
            "sys.exit\(\n"
            "   load_entry_point\(([^\)]+)\)\(\)\n"
            "\)\n")
        good_text = (
            "\n"
            "if __name__ == '__main__':\n"
            "    sys.exit(\n"
            r"        load_entry_point(\1)()\n"
            "    )\n"
            )
        contents = bad_text.sub(good_text, contents)

Traceback

 c:\opt\ZODB33>easy_install nose
easy_install nose
Searching for nose
Reading http://pypi.python.org/simple/nose/
Reading http://somethingaboutorange.com/mrl/projects/nose/
Reading http://readthedocs.org/docs/nose/
Best match: nose 1.2.1
Downloading http://pypi.python.org/packages/source/n/nose/nose-1.2.1.tar.gz#md5=735e3f1ce8b07e70ee1b742a8a53585a
Processing nose-1.2.1.tar.gz
Writing c:\users\mkearney\appdata\local\temp\easy_install-i2myhy\nose-1.2.1\setup.cfg
Running nose-1.2.1\setup.py -q bdist_egg --dist-dir c:\users\mkearney\appdata\local\temp\easy_install-i2myhy\nose-1.2.1\egg-dist-tmp-k2t8yc
no previously-included directories found matching 'doc\.build'
Adding nose 1.2.1 to easy-install.pth file
Installing nosetests-3.3-script.py script to c:\opt\py33\Scripts
Traceback (most recent call last):
  File "c:\opt\py33\Scripts\easy_install-script.py", line 9, in <module>
    load_entry_point('distribute==0.6.31', 'console_scripts', 'easy_install')()
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 1932, in main
    with_ei_usage(lambda:
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 1913, in with_ei_usage
    return f()
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 1936, in <lambda>
    distclass=DistributionWithoutHelpCommands, **kw
  File "c:\Python33\Lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "c:\Python33\Lib\distutils\dist.py", line 917, in run_commands
    self.run_command(cmd)
  File "c:\Python33\Lib\distutils\dist.py", line 936, in run_command
    cmd_obj.run()
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 358, in run
    self.easy_install(spec, not self.no_deps)
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 598, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 630, in install_item
    self.process_distribution(spec, dist, deps)
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 659, in process_distribution
    self.install_egg_scripts(dist)
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 529, in install_egg_scripts
    self.install_wrapper_scripts(dist)
  File "c:\opt\py33\lib\site-packages\distribute-0.6.31-py3.3.egg\setuptools\command\easy_install.py", line 734, in install_wrapper_scripts
    self.write_script(*args)
  File "setup.py", line 63, in wrap_write_script
TypeError: can't use a string pattern on a bytes-like object
@jszakmeister jszakmeister was assigned Mar 1, 2013
@jszakmeister

I'll take a look at this over the weekend.

@jszakmeister

nose 1.2.1 is incompatible with Python 3.3. I just tried the tip of nose and it installs just fine for me:

:: PYTHONPATH=$(pwd)/dumb easy_install -d dumb nose-1.3.0.tar.gz
Creating /Volumes/ScratchRaid/jszakmeister-scratch/projects/nose/dist/dumb/site.py
Processing nose-1.3.0.tar.gz
Writing /var/folders/9W/9WK-dXFfH2eMq+dEVqJZg++++TI/-Tmp-/easy_install-uqtw2h/nose-1.3.0/setup.cfg
Running nose-1.3.0/setup.py -q bdist_egg --dist-dir /var/folders/9W/9WK-dXFfH2eMq+dEVqJZg++++TI/-Tmp-/easy_install-uqtw2h/nose-1.3.0/egg-dist-tmp-czq4qj
no previously-included directories found matching 'doc/.build'
Adding nose 1.3.0 to easy-install.pth file
Installing nosetests-3.3 script to dumb
Installing nosetests script to dumb

Installed /Volumes/ScratchRaid/jszakmeister-scratch/projects/nose/dist/dumb/nose-1.3.0-py3.3.egg
Processing dependencies for nose==1.3.0
Finished processing dependencies for nose==1.3.0

Can you provide a traceback using the tip of nose?

@jszakmeister

Ah, I see... this is a Windows only thing... I'll fire up my VM and try.

@maveryKearney

Sadly True. Thanks for looking at it.
Good luck.
-m

@jszakmeister

@maveryKearney I believe you had the right idea in mind (skip rewriting the executables). I've tested this with Python 3.3 and Python 2.7 under Windows. Both install correctly now. The one thing I haven't done, and I believe this maybe true for most everyone, is run nose with multiprocess on a test suite. I did use the installed nose in Python 2.7 to run the nose's own unit test suite. And I did try passing --processes=2 and running the test suite too. Both passed.

@jpellerin does #638 look acceptable to you?

@jszakmeister jszakmeister added a commit that closed this issue Mar 3, 2013
@jszakmeister jszakmeister Fix #637: nose install fails w/ traceback using python 3.3.0
To cope with multiprocessing, nose attempts to rewrite some scripts.  It
turns out that some of the "scripts" are actually executables.  Under
Python 2.x, we'd just perform the substitution on the executable too,
but it never matched anything, and therefore, never replaced any bytes.
Under Python 3, that tactic no longer works because the scripts come as
a string, while the executables come as bytes.  We shouldn't try to
rewrite an executable anyways, so let's just skip them.
b03fec5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.