index.txt, news.txt and __init__.py are UTF-8 (see #761)

[kmike]

this is needed to use "LC_ALL=C" in py32 tox testing. see #761 for details.

[kmike]

It seems that the build fails because unicode metadata is not supported by distutils under Python 2.x. In my own packages I usually encode metadata to utf8 under Python 2.x (that's an encoding http://pypi.python.org/pypi uses). Should pip do the same?

[qwcode]

the failure is only in 2.5. encode the long_description back down to utf-8, and you'll get a clean travis build I think.

long_description = (parts[0] + long_description + parts[2] +
"\n\n" + read("docs", "news.txt")).encode('utf-8')

[kmike]

I believe binary long_description would be incorrect in Python 3.x

[qwcode]

ok, it fails in python3.
what do you mean exactly by "In my own packages I usually encode metadata to utf8 under Python 2.x"
a conditional in setup.py?

[kmike]

Yes, I think there should be a conditional in pip's setup.py.

utf8-encoded long_description is not without issues though: for example, setup.py register fails with it because distutils tries to check the markup using docutils and this fails with UnicodeDecodeError ( http://bugs.python.org/issue13114 ). I don't think this would be an issue for end-users of pip.

[hltbra]

Should we close this pull request because of pypa/virtualenv#201 (comment) and commit 1656ac3?

[kmike]

Yes, decoding from utf8 is pointless given that files are ascii now; also, decoding from utf8 was not enough to fix non-ascii issues.