Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeEncodeError in archive_util #710

Open
jaraco opened this issue Aug 2, 2016 · 5 comments
Open

UnicodeEncodeError in archive_util #710

jaraco opened this issue Aug 2, 2016 · 5 comments
Labels
bug help wanted Needs Investigation Issues which are likely in scope but need investigation to figure out the cause

Comments

@jaraco
Copy link
Member

jaraco commented Aug 2, 2016

After fixing #709, a new error is revealed in the LC_ALL=C tests in CI.

______________________________ test_unicode_files ______________________________

tarfile_with_unicode = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/unicode-pkg-1.0.tar.gz'

tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_unicode_files0')

    def test_unicode_files(tarfile_with_unicode, tmpdir):

        target = tmpdir / 'out'

>       archive_util.unpack_archive(tarfile_with_unicode, six.text_type(target))

setuptools/tests/test_archive_util.py:55: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

setuptools/archive_util.py:53: in unpack_archive

    driver(filename, extract_dir, progress_filter)

setuptools/archive_util.py:169: in unpack_tarfile

    tarobj._extract_member(member, final_dst)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2108: in _extract_member

    self.makefile(tarinfo, targetpath)

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tarfile.TarFile object at 0x7f86ff6c9a20>

tarinfo = <TarInfo './._testim\xe4ge.png' at 0x7f86ffefd8e0>

targetpath = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/out/./._testim\xe4ge.png'

    def makefile(self, tarinfo, targetpath):

        """Make a file called targetpath.

            """

        source = self.fileobj

        source.seek(tarinfo.offset_data)

>       with bltn_open(targetpath, "wb") as target:

E       UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 65: ordinal not in range(128)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2148: UnicodeEncodeError

The issue lies in the "Ugh" code in archive_util.py.

@jaraco
Copy link
Member Author

jaraco commented Aug 2, 2016

That didn't fix it. I'm giving up for now. If anyone else would like to take a stab at it, please feel free.

@pejorativefox
Copy link

Can confirm this, as it breaks many things upstream like zc.buildout. Clamping the setuptools version to 25.1.1 reverts to old behavior. To be honest though I can't really see anything in the diff that would do this........ Thanks for taking a stab at it @jaraco

sileht added a commit to sileht/setuptools that referenced this issue Aug 2, 2016
This reverts commit 8579495.

As we can see on pypa#709,
this breaks many things (easy_install C extensions, all py3.5 tests,
run with LANG=C).

So instead of fixing in a hurry all new bugs due to this, I propose to
revert this commit until all downsides of this change have been
investigated.

Related bug: pypa#709
Related bug: pypa#710
Related bug: pypa#712
jaraco added a commit that referenced this issue Aug 2, 2016
jaraco added a commit that referenced this issue Aug 2, 2016
jaraco added a commit that referenced this issue Dec 18, 2016
@pganssle pganssle added the Needs Triage Issues that need to be evaluated for severity and status. label Oct 19, 2018
@pganssle
Copy link
Member

@jaraco Is this fixed by 6c6f4ca, or was that just a temporary workaround?

@pganssle pganssle added bug help wanted Needs Investigation Issues which are likely in scope but need investigation to figure out the cause and removed Needs Triage Issues that need to be evaluated for severity and status. labels Oct 23, 2018
@mgorny
Copy link
Contributor

mgorny commented Apr 4, 2020

Unless I'm doing something wrong, this is now XPASS-ing for me on 46.1.3, with PyPy3 7.3.0, CPython 3.{6.10,7.7,8.2}.

@mgorny
Copy link
Contributor

mgorny commented Apr 4, 2020

Sorry, this → setuptools/tests/test_archive_util.py::test_unicode_files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug help wanted Needs Investigation Issues which are likely in scope but need investigation to figure out the cause
Projects
None yet
Development

No branches or pull requests

4 participants