Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeEncodeError in archive_util #710

Open
jaraco opened this issue Aug 2, 2016 · 5 comments
Open

UnicodeEncodeError in archive_util #710

jaraco opened this issue Aug 2, 2016 · 5 comments

Comments

@jaraco
Copy link
Member

@jaraco jaraco commented Aug 2, 2016

After fixing #709, a new error is revealed in the LC_ALL=C tests in CI.

______________________________ test_unicode_files ______________________________

tarfile_with_unicode = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/unicode-pkg-1.0.tar.gz'

tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_unicode_files0')

    def test_unicode_files(tarfile_with_unicode, tmpdir):

        target = tmpdir / 'out'

>       archive_util.unpack_archive(tarfile_with_unicode, six.text_type(target))

setuptools/tests/test_archive_util.py:55: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

setuptools/archive_util.py:53: in unpack_archive

    driver(filename, extract_dir, progress_filter)

setuptools/archive_util.py:169: in unpack_tarfile

    tarobj._extract_member(member, final_dst)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2108: in _extract_member

    self.makefile(tarinfo, targetpath)

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tarfile.TarFile object at 0x7f86ff6c9a20>

tarinfo = <TarInfo './._testim\xe4ge.png' at 0x7f86ffefd8e0>

targetpath = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/out/./._testim\xe4ge.png'

    def makefile(self, tarinfo, targetpath):

        """Make a file called targetpath.

            """

        source = self.fileobj

        source.seek(tarinfo.offset_data)

>       with bltn_open(targetpath, "wb") as target:

E       UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 65: ordinal not in range(128)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2148: UnicodeEncodeError

The issue lies in the "Ugh" code in archive_util.py.

jaraco added a commit that referenced this issue Aug 2, 2016
@jaraco
Copy link
Member Author

@jaraco jaraco commented Aug 2, 2016

That didn't fix it. I'm giving up for now. If anyone else would like to take a stab at it, please feel free.

@pejorativefox
Copy link

@pejorativefox pejorativefox commented Aug 2, 2016

Can confirm this, as it breaks many things upstream like zc.buildout. Clamping the setuptools version to 25.1.1 reverts to old behavior. To be honest though I can't really see anything in the diff that would do this........ Thanks for taking a stab at it @jaraco

sileht added a commit to sileht/setuptools that referenced this issue Aug 2, 2016
This reverts commit 8579495.

As we can see on pypa#709,
this breaks many things (easy_install C extensions, all py3.5 tests,
run with LANG=C).

So instead of fixing in a hurry all new bugs due to this, I propose to
revert this commit until all downsides of this change have been
investigated.

Related bug: pypa#709
Related bug: pypa#710
Related bug: pypa#712
jaraco added a commit that referenced this issue Aug 2, 2016
jaraco added a commit that referenced this issue Aug 2, 2016
jaraco added a commit that referenced this issue Dec 18, 2016
…ative str as workaround for #709, #710, and #712.
@pganssle
Copy link
Member

@pganssle pganssle commented Oct 23, 2018

@jaraco Is this fixed by 6c6f4ca, or was that just a temporary workaround?

@mgorny
Copy link
Contributor

@mgorny mgorny commented Apr 4, 2020

Unless I'm doing something wrong, this is now XPASS-ing for me on 46.1.3, with PyPy3 7.3.0, CPython 3.{6.10,7.7,8.2}.

@mgorny
Copy link
Contributor

@mgorny mgorny commented Apr 4, 2020

Sorry, this → setuptools/tests/test_archive_util.py::test_unicode_files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants