Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyCryptodome as replacement for PyCrypto #52674

Closed
xsc27 opened this issue Apr 24, 2019 · 9 comments
Closed

PyCryptodome as replacement for PyCrypto #52674

xsc27 opened this issue Apr 24, 2019 · 9 comments

Comments

@xsc27
Copy link

@xsc27 xsc27 commented Apr 24, 2019

Description of Issue/Question

PyCrypto is unmaintained. And sometimes fails to build. There several issues open about PyCrypto failing on Python 3, but this is the longest thread, pycrypto/pycrypto#218. And definitely does not build with Pypy3.

PyCryptodome is a fork and (mostly) API compatible replacement, see https://pycryptodome.readthedocs.io/en/latest/src/vs_pycrypto.html.

Adapting Salt for PyCryptodome is a faster near-term solution rather than implementing #30287 or making the crypto backend modular (#44923 (comment))

Versions Report

At least salt 2019.2.0 (Fluorine) and earlier.

@dmurphy18
Copy link
Contributor

@dmurphy18 dmurphy18 commented Apr 24, 2019

@CMeza99 Thanks for your suggestion but but there is already the ability to utilize PyCryptodome instead of PyCrypto (on Redhat family) for those who desire. Currently SaltStack makes available a version of PyCryptodomex (Redhat family) and M2Crypto (all Linux platforms), and there is a picking order as to which package is used as follows:

PyCrypto - basic level
PyCryptodomx - preferred over PyCrypto if installed
M2Crypto - preferred over PyCryptodome and PyCrypto if installed

This pecking order is present on both the 2018.3 and 2019.2 branches.
Support for #30287 if and when, is probably going to occur in Python 3 given the short runway that Python 2.7 has.

Hope this clarifies the issue for you, that PyCyrptodome and M2Crypto can already be used as alternatives to PyCrypto. If this is sufficient information, please consider closing this issue unless there is something else I can help clarify about this for you.

@xsc27
Copy link
Author

@xsc27 xsc27 commented Apr 24, 2019

Thanks @dmurphy18. So I suppose my issue is that pycrpto installs as a dependency regardless if one of the preferred libraries are installed.

M2Crypto==0.32.0
pip==19.1
salt==2019.2.0
  - Jinja2 [required: Any, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - MarkupSafe [required: Any, installed: 1.1.1]
  - msgpack [required: >=0.5,!=0.5.5, installed: 0.6.1]
  - pycrypto [required: >=2.6.1, installed: 2.6.1]

I did some digging and it looks like PyCrypto is set as a dependency for ZeroMQ. Is this is necessary? https://github.com/saltstack/salt/blob/develop/requirements/zeromq.txt#L3

@dmurphy18
Copy link
Contributor

@dmurphy18 dmurphy18 commented Apr 24, 2019

@CMeza99 Actually Salt depends on python-crypto in packaged form (commonly PyCrypto) as a hard depend, and has done for years. It is unlikely to be removed from Python 2.7 packages at this point.

Need to cleanup those requirement files when using pip

root@c7master:~# yum deplist salt
Loaded plugins: fastestmirror, langpacks
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
Loading mirror speeds from cached hostfile

  • base: mirrors.xmission.com
  • epel: mirrors.xmission.com
  • extras: mirrors.xmission.com
  • os_pkgs_repo: mirrors.xmission.com
  • updates: mirrors.xmission.com
    package: salt.noarch 2019.2.0-1.el7
    dependency: /bin/sh
    provider: bash.x86_64 4.2.46-31.el7
    dependency: /usr/bin/env
    provider: coreutils.x86_64 8.22-23.el7
    dependency: /usr/bin/python
    provider: python.x86_64 2.7.5-76.el7
    dependency: PyYAML
    provider: PyYAML.x86_64 3.11-1.el7
    dependency: pciutils
    provider: pciutils.x86_64 3.5.1-3.el7
    dependency: python(abi) = 2.7
    provider: python.x86_64 2.7.5-76.el7
    dependency: python-crypto >= 2.6.1
    provider: python2-crypto.x86_64 2.6.1-15.el7
    provider: python-crypto.x86_64 2.6.1-2.el7

When Salt moves to an all Python 3 client (no Python 2.7 built or involved) can look again at removing it, but until the issue with Salt and python-tornado > 5.0 on Python 3 is resolved, have to have Python 2.7 around to build Python 3 packages. Note that fix is currently scheduled for the Neon release.

@xsc27
Copy link
Author

@xsc27 xsc27 commented Apr 25, 2019

@dmurphy18 thank you for the info.

I will close this issue for now as there are blockers that will be addressed in an upcoming version. Though it is confusing that the issue is with Salt and python-tornado, but PyCrypto is coming as part of requirements/zeromq.txt.

@udf2457
Copy link

@udf2457 udf2457 commented Feb 13, 2020

@dmurphy18, re: your comment to @CMeza99, may I point out this quote from the PyCryptodome man page (https://pycryptodome.readthedocs.io/en/latest/src/installation.html):

One must avoid having both PyCrypto and PyCryptodome installed at the same time, as they will interfere with each other.

It is therefore in my view imperative that Salt needs to eradicate its dependence on the unsupported PyCrypto ASAP.

@dmurphy18
Copy link
Contributor

@dmurphy18 dmurphy18 commented Feb 13, 2020

@udf2457 I would very much like to drop pycrypto however I still have to support OS's such as Debian 8 (until June 30th) till they are effectively EOL and there is no pycryptodome support on Debian 8. Similarly I ran into issues with pycryptodomex in 32-bit mode on RHEL 6 which does not end of life until 30th Oct 2020. My preference would be for later versions of M2Crypto as used on Py 3 RHEL 7 & 8 as supplied and installed by SaltStack packages.

The pycrypto issue arose two days before final v3000 tag, and it was the quickest way to resolve the issue and not miss the already delayed release. This use of pycrypto was no different that what had been released in previous releases, that is, unchanged from previous releases. Efforts are been made in packaging to install either pycryptodome[x] or M2Crypto where possible for an OS in the next release, and if I could drop Debian 8 (Raspbian 8 too), RHEL 6 and Amazon Linux 1 support, it would be gone already; I have to wait for those OS's to EOL.

I would gladly accept a contribution on a backported version of pycryptodome[x] running on those platforms in order to be rid of pycrypto, note: remember Raspbian 8 is Debian 8 on ARM HL.

@udf2457
Copy link

@udf2457 udf2457 commented Feb 13, 2020

@dmurphy18 Bring on June 2020, finally the demise of Debian 8 !

M2Crypto is not available on Debian 10 for python3, only the obsolete python2:

$ sudo apt-cache search M2Crypto
m2crypto-doc - Python wrapper for the OpenSSL library (docs)
python-m2crypto - Python wrapper for the OpenSSL library (Python 2 modules)
python-m2ext - Extensions to the M2Crypto Python package

Regardign your last paragraph (contribution). I don't do python, not even in newbie spaghetti-code format. So can't help you there. Unfortunatley.

@ymasson
Copy link
Contributor

@ymasson ymasson commented Feb 18, 2020

@udf2457 python3-m2crypto is available in buster-backports.

root@845d019ba068# apt-cache search M2Crypt
m2crypto-doc - Python wrapper for the OpenSSL library (docs)
python-m2crypto - Python wrapper for the OpenSSL library (Python 2 modules)
python-m2ext - Extensions to the M2Crypto Python package
python3-m2crypto - Python wrapper for the OpenSSL library (Python 3 modules)

root@845d019ba068# apt-cache policy python3-m2crypto
python3-m2crypto:
  Installed: (none)
  Candidate: 0.31.0-6~bpo10+1
  Version table:
     0.31.0-6~bpo10+1 100
        100 http://deb.debian.org/debian buster-backports/main amd64 Packages

@udf2457
Copy link

@udf2457 udf2457 commented Feb 18, 2020

@ymasson All very well if your environment permits the use of buster-backports.

Some of us work with environments where the double-edged sword warnings posted on the buster-backports page need to be heeded:

backports are provided on an as-is basis, with risk of incompatibilities with other components in Debian stable. Use with care!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants