Regression: Setuptools 3.7 installation fails with TypeError under pbr

[ghost]

Originally reported by: alex_gaynor (Bitbucket: alex_gaynor, GitHub: Unknown)

---

Complete reproducer:

$ mktmpenv
New python executable in /Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/bin/python
Installing setuptools, pip...pip install -Udone.
[14:24:47] (tempenv-4b7d34306d0a) ~/p/openstack-requirements (master)
$ pip install -U setuptools
Unpacking /Users/alex_gaynor/.pip/wheels/setuptools-4.0-py2.py3-none-any.whl
Installing collected packages: setuptools
  Found existing installation: setuptools 3.6
    Uninstalling setuptools:
      Successfully uninstalled setuptools
Successfully installed setuptools
Cleaning up...
[14:24:52] (tempenv-4b7d34306d0a) ~/p/openstack-requirements (master)
$ pip install --no-use-wheel python-daemon
Downloading/unpacking python-daemon
  Using download cache from /Users/alex_gaynor/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpython-daemon%2Fpython-daemon-1.6.tar.gz
  Running setup.py (path:/Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/build/python-daemon/setup.py) egg_info for package python-daemon
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/build/python-daemon/setup.py", line 62, in <module>
        u"Topic :: Software Development :: Libraries :: Python Modules",
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
        dist.run_commands()
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "<string>", line 14, in replacement_run
      File "/Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 368, in write_requirements
        _write_requirements(data, dist.install_requires)
      File "/Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 363, in _write_requirements
        stream.writelines(lines)
    TypeError: 'unicode' does not have the buffer interface
    Complete output from command python setup.py egg_info:
    running egg_info

creating pip-egg-info/python_daemon.egg-info

Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/build/python-daemon/setup.py", line 62, in <module>

    u"Topic :: Software Development :: Libraries :: Python Modules",

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup

    dist.run_commands()

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands

    self.run_command(cmd)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command

    cmd_obj.run()

  File "<string>", line 14, in replacement_run

  File "/Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 368, in write_requirements

    _write_requirements(data, dist.install_requires)

  File "/Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 363, in _write_requirements

    stream.writelines(lines)

TypeError: 'unicode' does not have the buffer interface

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /Users/alex_gaynor/.virtualenvs/tempenv-4b7d34306d0a/build/python-daemon
Storing debug log for failure in /Users/alex_gaynor/.pip/pip.log

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/213

[ghost]

Original comment by alex_gaynor (Bitbucket: alex_gaynor, GitHub: Unknown):

---

It appears that 3.7 was the release which introduced this regression.

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

I've thusfar been unable to replicate the failure. I've installed several packages (pytest, including py, sqlalchemy, and others) without failure (using easy_install against sdists). I suspect the regression may be related to pip.

python-daemon fails to run on Windows and on Python 3. Is there another package that is maybe a little less opinionated on which the failure can be repeated?

[ghost]

Original comment by therve (Bitbucket: therve, GitHub: therve):

---

The bug is specific to python 2 though.

The following patch seems the fix the issue for me: http://paste.openstack.org/show/82402/

[ghost]

Original comment by divius (Bitbucket: divius, GitHub: divius):

---

I can also reproduce this bug with "pbr" package.

[ghost]

Original comment by Ivoz (Bitbucket: Ivoz, GitHub: Ivoz):

---

It's confusing to me why at line 367 here we want a text buffer on python 3 but specifically a bytes buffer on python 2. Why not just text buffer (io.StringIO) on both?

The current code definitely fails on being passed unicode install requirements.

[ghost]

Original comment by lothiraldan (Bitbucket: lothiraldan, GitHub: lothiraldan):

---

I can reproduce this bug with "pbr" package too, I downloaded it, run python setup.py egg_info and it crash.

[ghost]

Original comment by jamezpolley (Bitbucket: jamezpolley, GitHub: jamezpolley):

---

As well as python-daemon and pbr, I can reproduce this with virtualenvwrapper

[ghost]

Original comment by pavelrad (Bitbucket: pavelrad, GitHub: pavelrad):

---

Also had this issue after upgrading to Python 2.7.7 on OS X with Homebrew. Brew automatically installs setuptools 4.0 and after that I was unable to install virtualenvwrapper. Downgrade to setuptools 3.6 solved this issue.

[ghost]

Original comment by kiall (Bitbucket: kiall, GitHub: kiall):

---

virtualenvwrapper depends on pbr, so it's likely that again rather than a new bug :)

Any of the OpenStack packages will show this error - e.g. python-glanceclient, python-novaclient, python-keystoneclient, etc etc

[ghost]

Original comment by pavelrad (Bitbucket: pavelrad, GitHub: pavelrad):

---

I don't know what is pbr and I don't see it in pip list after virtualenvwrapper had been installed. Are you sure that the problem is caused by that package?

[ghost]

Original comment by Ivoz (Bitbucket: Ivoz, GitHub: Ivoz):

---

@pavelrad it depends on it at build time, but not at run time

[ghost]

Original comment by kiall (Bitbucket: kiall, GitHub: kiall):

---

Yes, it's a install time dep: https://bitbucket.org/dhellmann/virtualenvwrapper/src/58c2459e5b27e3876b99842f323222349d1b6ed1/setup.py?at=master

[ghost]

Original comment by pavelrad (Bitbucket: pavelrad, GitHub: pavelrad):

---

I see :)

[ghost]

Original comment by Ivoz (Bitbucket: Ivoz, GitHub: Ivoz):

---

This seems like an idiomatic patch to me:

--- a/setuptools/command/egg_info.py    Sun Jun 01 22:03:32 2014 +0100
+++ b/setuptools/command/egg_info.py    Mon Jun 02 20:30:46 2014 +1000
@@ -5,13 +5,12 @@
 import os
 import re
 import sys
-import io

 from setuptools import Command
 import distutils.errors
 from distutils import log
 from setuptools.command.sdist import sdist
-from setuptools.compat import basestring, PY3, unicode
+from setuptools.compat import basestring, PY3, unicode, StringIO
 from setuptools import svn_utils
 from distutils.util import convert_path
 from distutils.filelist import FileList as _FileList
@@ -364,7 +363,7 @@

 def write_requirements(cmd, basename, filename):
     dist = cmd.distribution
-    data = io.StringIO() if PY3 else io.BytesIO()
+    data = StringIO()
     _write_requirements(data, dist.install_requires)
     extras_require = dist.extras_require or {}
     for extra in sorted(extras_require):

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

I believe this problem to be fixed in 3.7.1 and 4.0.1. Please report if you find that's not the case.

[ghost]

Original comment by Ivoz (Bitbucket: Ivoz, GitHub: Ivoz):

---

@jaraco is there a particular reason this was applied to 3.7 and not 3.8 (say, if someone wants to continue using the latest 3.X setuptools)?

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

@Ivoz I applied the fix to 3.7, as that's where the regression occurred. I then merged that with the latest tip (4.0), assuming there would be little demand for 3.x support. Now that there's a report against 4.0, it's probably worth making a 3.8.1 release with this fix.

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Merge with 3.7.1 (Ref #213)

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

3.8.1 now is released with this fix.