Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bad magic number in 'six' #359

Closed
chrisdamato opened this issue Oct 1, 2021 · 4 comments
Closed

bad magic number in 'six' #359

chrisdamato opened this issue Oct 1, 2021 · 4 comments

Comments

@chrisdamato
Copy link

Fedora 33. Any dnf or yum command produces the same error message before the normal output of the command:

Failed loading plugin "changelog": bad magic number in 'six': b'\x03\xf3\r\n'

For example:

$ dnf info python3-six
Failed loading plugin "changelog": bad magic number in 'six': b'\x03\xf3\r\n'
Last metadata expiration check: 1:22:05 ago on Fri 01 Oct 2021 07:59:45 AM EDT.
Installed Packages
Name         : python3-six
Version      : 1.15.0
Release      : 2.fc33
Architecture : noarch
Size         : 104 k
Source       : python-six-1.15.0-2.fc33.src.rpm
Repository   : @System
From repo    : fedora
Summary      : Python 2 and 3 compatibility utilities
URL          : https://pypi.python.org/pypi/six
License      : MIT
Description  : python-six provides simple utilities for wrapping over differences between
             : Python 2 and Python 3.
             : Python 3 version.

I tried updating the system, reinstalling six using pip and dnf, no change.

If I start python3 and import six, there is no error message. If I disable the changelog plugin to dnf (a builtin apparently) the message does not appear.

Hope I'm doing this right. Thanks for any help you are able to provide.
@atc0005
Copy link

atc0005 commented Dec 16, 2021

@chrisdamato I ran into this today. The culprit ended up being a /usr/bin/six.pyc file. This seems to have been created by the VMware Perl SDK installation process. Go figure.

Once I removed that file, the error went away.

@hroncok
Copy link
Contributor

hroncok commented Feb 23, 2022

When you run e.g. dnf or any other commend that is written in Python and installed in /usr/bin, Python adds /usr/bin as sys.path[0]. When a file called six.pyc exists in there, it is imported instead of six and the import fails.

We (Fedora) would like to start using a flag in shebang lines of Python scripts installed in /usr/bin that would prevent this undesired behavior, but such flag does not exist yet (-I does that, but it also does other things). I've asked for such flag many winters ago in https://discuss.python.org/t/python-flag-envvar-not-to-put-current-directory-to-sys-path-but-dont-ignore-pythonpath/4235

hroncok added a commit to hroncok/dnf that referenced this issue Feb 23, 2022
@hroncok
Remove /usr/bin from sys.path to avoid accidentally importing garbage
dbefa7e 
See https://bugzilla.redhat.com/show_bug.cgi?id=2057340
and benjaminp/six#359

dnf should never import Python modules from /usr/bin but users can
have files in there that look like Python modules and Python will
try to import them and fail.

Consider a tool that is *not* written in Python and is called "copy.pyc".
Naturally, it resides in /usr/bin/copy.pyc and dnf fails:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: bad magic number in 'copy': b'...'

Similarly, a tool actually written in Python, called "copy.py" might have
as well own /usr/bin/copy.py and dnf fails as well:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: cannot import name 'deepcopy' from 'copy' (/usr/bin/copy.py)

Either problem can happen for a variety of names.
We better not let that happen.

A more general solution that would prevent Python doing this entirely
does not exists yet, see https://discuss.python.org/t/4235

Hence, proposing this to dnf, which is a critical piece of the system.
@hroncok hroncok mentioned this issue Feb 23, 2022
Merged
@hroncok
Copy link
Contributor

hroncok commented Feb 23, 2022

I've opened rpm-software-management/dnf#1815 for dnf specifically, but this can happen to any Python script residing in a directory like /usr/bin.

hroncok added a commit to hroncok/dnf that referenced this issue Feb 23, 2022
@hroncok
Remove /usr/bin from sys.path to avoid accidentally importing garbage
dcdeb96 
See https://bugzilla.redhat.com/show_bug.cgi?id=2057340
and benjaminp/six#359

dnf should never import Python modules from /usr/bin but users can
have files in there that look like Python modules and Python will
try to import them and fail.

Consider a tool that is *not* written in Python and is called "copy.pyc".
Naturally, it resides in /usr/bin/copy.pyc and dnf fails:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: bad magic number in 'copy': b'...'

Similarly, a tool actually written in Python, called "copy.py"
might as well own /usr/bin/copy.py and dnf fails as well:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: cannot import name 'deepcopy' from 'copy' (/usr/bin/copy.py)

Either problem can happen for a variety of names.
We better not let that happen.

A more general solution that would prevent Python doing this entirely
does not exists yet, see https://discuss.python.org/t/4235

Hence, proposing this to dnf, which is a critical piece of the system.
@hroncok
Copy link
Contributor

hroncok commented Feb 23, 2022 •
edited
Loading

I've also opened https://discuss.python.org/t/should-console-scripts-entry-points-exclude-the-scripts-directory-from-sys-path/13896 for console_scripts entry points (which dnf is not)

lukash pushed a commit to rpm-software-management/dnf that referenced this issue Feb 25, 2022
@hroncok @lukash
Remove /usr/bin from sys.path to avoid accidentally importing garbage
c68038f 
See https://bugzilla.redhat.com/show_bug.cgi?id=2057340
and benjaminp/six#359

dnf should never import Python modules from /usr/bin but users can
have files in there that look like Python modules and Python will
try to import them and fail.

Consider a tool that is *not* written in Python and is called "copy.pyc".
Naturally, it resides in /usr/bin/copy.pyc and dnf fails:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: bad magic number in 'copy': b'...'

Similarly, a tool actually written in Python, called "copy.py"
might as well own /usr/bin/copy.py and dnf fails as well:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: cannot import name 'deepcopy' from 'copy' (/usr/bin/copy.py)

Either problem can happen for a variety of names.
We better not let that happen.

A more general solution that would prevent Python doing this entirely
does not exists yet, see https://discuss.python.org/t/4235

Hence, proposing this to dnf, which is a critical piece of the system.
@vstinner vstinner mentioned this issue May 5, 2022
Closed
m-blaha pushed a commit to rpm-software-management/dnf that referenced this issue Sep 22, 2023
@hroncok @m-blaha
Remove /usr/bin from sys.path to avoid accidentally importing garbage
591341f 
See https://bugzilla.redhat.com/show_bug.cgi?id=2057340
and benjaminp/six#359

dnf should never import Python modules from /usr/bin but users can
have files in there that look like Python modules and Python will
try to import them and fail.

Consider a tool that is *not* written in Python and is called "copy.pyc".
Naturally, it resides in /usr/bin/copy.pyc and dnf fails:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: bad magic number in 'copy': b'...'

Similarly, a tool actually written in Python, called "copy.py"
might as well own /usr/bin/copy.py and dnf fails as well:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: cannot import name 'deepcopy' from 'copy' (/usr/bin/copy.py)

Either problem can happen for a variety of names.
We better not let that happen.

A more general solution that would prevent Python doing this entirely
does not exists yet, see https://discuss.python.org/t/4235

Hence, proposing this to dnf, which is a critical piece of the system.
m-blaha pushed a commit to rpm-software-management/dnf that referenced this issue Sep 22, 2023
@hroncok @m-blaha
RHEL-1245: Remove /usr/bin from sys.path to avoid accidentally import…
29f4df4 
…ing garbage

See https://bugzilla.redhat.com/show_bug.cgi?id=2057340
and benjaminp/six#359

dnf should never import Python modules from /usr/bin but users can
have files in there that look like Python modules and Python will
try to import them and fail.

Consider a tool that is *not* written in Python and is called "copy.pyc".
Naturally, it resides in /usr/bin/copy.pyc and dnf fails:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: bad magic number in 'copy': b'...'

Similarly, a tool actually written in Python, called "copy.py"
might as well own /usr/bin/copy.py and dnf fails as well:

    Traceback (most recent call last):
      File "/usr/bin/dnf", line 57, in <module>
        from dnf.cli import main
      File "/usr/lib/python3.10/site-packages/dnf/__init__.py", line 30, in <module>
        import dnf.base
      File "/usr/lib/python3.10/site-packages/dnf/base.py", line 31, in <module>
        from copy import deepcopy
    ImportError: cannot import name 'deepcopy' from 'copy' (/usr/bin/copy.py)

Either problem can happen for a variety of names.
We better not let that happen.

A more general solution that would prevent Python doing this entirely
does not exists yet, see https://discuss.python.org/t/4235

Hence, proposing this to dnf, which is a critical piece of the system.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@benjaminp @chrisdamato @hroncok @atc0005