New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeEncodeError in archive_util #710

Open
jaraco opened this Issue Aug 2, 2016 · 3 comments

Comments

Projects
None yet
3 participants
@jaraco
Member

jaraco commented Aug 2, 2016

After fixing #709, a new error is revealed in the LC_ALL=C tests in CI.

______________________________ test_unicode_files ______________________________

tarfile_with_unicode = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/unicode-pkg-1.0.tar.gz'

tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_unicode_files0')

    def test_unicode_files(tarfile_with_unicode, tmpdir):

        target = tmpdir / 'out'

>       archive_util.unpack_archive(tarfile_with_unicode, six.text_type(target))

setuptools/tests/test_archive_util.py:55: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

setuptools/archive_util.py:53: in unpack_archive

    driver(filename, extract_dir, progress_filter)

setuptools/archive_util.py:169: in unpack_tarfile

    tarobj._extract_member(member, final_dst)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2108: in _extract_member

    self.makefile(tarinfo, targetpath)

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tarfile.TarFile object at 0x7f86ff6c9a20>

tarinfo = <TarInfo './._testim\xe4ge.png' at 0x7f86ffefd8e0>

targetpath = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/out/./._testim\xe4ge.png'

    def makefile(self, tarinfo, targetpath):

        """Make a file called targetpath.

            """

        source = self.fileobj

        source.seek(tarinfo.offset_data)

>       with bltn_open(targetpath, "wb") as target:

E       UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 65: ordinal not in range(128)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2148: UnicodeEncodeError

The issue lies in the "Ugh" code in archive_util.py.

jaraco added a commit that referenced this issue Aug 2, 2016

@jaraco

This comment has been minimized.

Member

jaraco commented Aug 2, 2016

That didn't fix it. I'm giving up for now. If anyone else would like to take a stab at it, please feel free.

@pejorativefox

This comment has been minimized.

pejorativefox commented Aug 2, 2016

Can confirm this, as it breaks many things upstream like zc.buildout. Clamping the setuptools version to 25.1.1 reverts to old behavior. To be honest though I can't really see anything in the diff that would do this........ Thanks for taking a stab at it @jaraco

sileht added a commit to sileht/setuptools that referenced this issue Aug 2, 2016

Revert "Ensure that tmpdir is unicode. Fixes pypa#704."
This reverts commit 8579495.

As we can see on pypa#709,
this breaks many things (easy_install C extensions, all py3.5 tests,
run with LANG=C).

So instead of fixing in a hurry all new bugs due to this, I propose to
revert this commit until all downsides of this change have been
investigated.

Related bug: pypa#709
Related bug: pypa#710
Related bug: pypa#712

jaraco added a commit that referenced this issue Aug 2, 2016

jaraco added a commit that referenced this issue Aug 2, 2016

jaraco added a commit that referenced this issue Dec 18, 2016

@pganssle

This comment has been minimized.

Member

pganssle commented Oct 23, 2018

@jaraco Is this fixed by 6c6f4ca, or was that just a temporary workaround?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment