TypeError Excepiton during connect (probably incorrect usage of instance method)

[a1ds]

Hello.

The problem looks like a bug, but I don’t sure.

My environment:

###
uname -a
Linux motkb-dev-app 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux

###
cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.3 (Maipo)

###
yum list installed | grep python36
rh-python36-python.x86_64             3.6.3-1.el7               @asm_dev_local
rh-python36-python-devel.x86_64       3.6.3-1.el7               @asm_dev_local
rh-python36-python-libs.x86_64        3.6.3-1.el7               @asm_dev_local
rh-python36-python-pip.noarch         9.0.1-2.el7               @asm_dev_local
rh-python36-python-setuptools.noarch  36.5.0-1.el7              @asm_dev_local
rh-python36-python-tkinter.x86_64     3.6.3-1.el7               @asm_dev_local
rh-python36-python-tools.x86_64       3.6.3-1.el7               @asm_dev_local
rh-python36-python-virtualenv.noarch  15.1.0-2.el7              @asm_dev_local
rh-python36-runtime.x86_64            2.0-1.el7                 @asm_dev_local

###
python --version
Python 3.6.3

###
Python 3.6.3 (default, Oct 25 2017, 13:35:50)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux

###
pip list
Package        Version
-------------- -------
asn1crypto     0.24.0
bcrypt         3.1.4
cffi           1.11.5
cryptography   2.2
docopt         0.6.2
idna           2.6
jedi           0.11.1
paramiko       2.4.1
parso          0.1.1
pip            9.0.1
prompt-toolkit 1.0.15
ptpython       0.41
pyasn1         0.4.2
pycparser      2.18
Pygments       2.2.0
PyNaCl         1.2.1
setuptools     28.8.0
six            1.11.0
wcwidth        0.1.7
wheel          0.29.0

Steps to reproduce the problem:

steps_to_reproduce_the_problem.py

import paramiko

host = '127.0.0.1'

client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())

client.connect(hostname=host)


stdin, stdout, stderr = client.exec_command('ls -l')
data = stdout.read() + stderr.read()
print(data)
client.close()

The execution of the code above produce an error:

Unknown exception: an integer is required (got type bytes)
Traceback (most recent call last):
  File "/home/someone/adir/venv_paramiko_test/lib/python3.6/site-packages/paramiko/transport.py", line 1925, in run
    self.kex_engine.parse_next(ptype, m)
  File "/home/someone/adir/venv_paramiko_test/lib/python3.6/site-packages/paramiko/kex_ecdh_nist.py", line 47, in parse_next
    return self._parse_kexecdh_reply(m)
  File "/home/someone/adir/venv_paramiko_test/lib/python3.6/site-packages/paramiko/kex_ecdh_nist.py", line 106, in _parse_kexecdh_reply
    self.transport._set_K_H(K, self.hash_algo(hm.asbytes()).digest())
  File "/home/someone/adir/venv_paramiko_test/lib64/python3.6/hashlib.py", line 90, in inner
    return func(*args, **kwargs)
TypeError: an integer is required (got type bytes)

Traceback (most recent call last):
  File "steps_to_reproduce_the_problem.py", line 9, in <module>
    client.connect(hostname=host)
  File "/home/someone/adir/venv_paramiko_test/lib/python3.6/site-packages/paramiko/client.py", line 392, in connect
    t.start_client(timeout=timeout)
  File "/home/someone/adir/venv_paramiko_test/lib/python3.6/site-packages/paramiko/transport.py", line 545, in start_client
    raise e
  File "/home/someone/adir/venv_paramiko_test/lib/python3.6/site-packages/paramiko/transport.py", line 1925, in run
    self.kex_engine.parse_next(ptype, m)
  File "/home/someone/adir/venv_paramiko_test/lib/python3.6/site-packages/paramiko/kex_ecdh_nist.py", line 47, in parse_next
    return self._parse_kexecdh_reply(m)
  File "/home/someone/adir/venv_paramiko_test/lib/python3.6/site-packages/paramiko/kex_ecdh_nist.py", line 106, in _parse_kexecdh_reply
    self.transport._set_K_H(K, self.hash_algo(hm.asbytes()).digest())
  File "/home/someone/adir/venv_paramiko_test/lib64/python3.6/hashlib.py", line 90, in inner
    return func(*args, **kwargs)
TypeError: an integer is required (got type bytes)

Preliminary conclusions

I performed some investigation on kex_ecdh_nist.py, and found(I hope so) the reason of the error.

Here is a patch that I used to fix the issue:

21c21
<     hash_algo = sha256
---
>     hash_algo = staticmethod(sha256)
113c113
<     hash_algo = sha384
---
>     hash_algo = staticmethod(sha384)
119c119
<     hash_algo = sha512
---
>     hash_algo = staticmethod(sha512)

And some explanations: the class KexNistp256 has an attribute hash_algo, it points to the function sha256(from hashlib), and here is the code:

    ..................
    18  class KexNistp256():
    19
    20      name = "ecdh-sha2-nistp256"
    21      hash_algo = sha256
    22      curve = ec.SECP256R1()
    23
    ..................

Then, later, the call of the hash_algo:

    ..................
   104          hm.add_string(Q_S_bytes)
   105          hm.add_mpint(K)
   106          self.transport._set_K_H(K, self.hash_algo(hm.asbytes()).digest())
   107          self.transport._verify_key(K_S, sig)
   108          self.transport._activate_outbound()
    ..................

And the call self.hash_algo(hm.asbytes()) it the reason of the error.

This happened because for the instance of the class, hash_algo, is not a function, but a bind method, and a bind method, during the call of KexNistp256.hash_algo puts object that called the method(in our case it is self) as a first argument of the method, and then, puts all other arguments, so in a nutshell, final call looks like this:

sha256(self, hm.asbytes())

[ploxiln]

Makes sense. You could open a pull request.

[mr-sven]

Hi,
I run into same issue, at our side it was fixed by upgrade to python-3.6.3-3.
Nevertheless if you implement this change, you should do this in all kex_* files.

Regards Sven

[bitprophet]

Thanks for everyone who's reported/discussed/PR'd this! Couple questions re: issue / PR:

• Are there any concrete details on exactly what environment(s) give rise to this bug? Interpreter versions, SSL versions, etc? TypeError: an integer is required (got type bytes) on python3.6 (python3.5 is ok) #1187 says it has to do with one's OpenSSL, @mr-sven asserts that a change made to the interpreter/stdlib is capable of fixing it, etc. I'm assuming it's only popping up on Python 3 and with certain SSL versions, but I'd love to know for sure so we can document it better in the changelog.
• Are there reasonable tests we can write for this (eg mocking out the sha* objects so they exhibit the problematic behavior even if the platform under test doesn't natively have it)?

[a1ds]

+1 for @mr-sven comment, that the change should be made for all of kex_* except kex_gss.py

Answering @bitprophet's questions:

• I'll try to get some details close to the end of the week, but preliminary, my guess is the issue is related with a concrete implementation of hashlib/_hashlib modules(because I updated my env to version python-3.6.3-3 as @mr-sven suggested, and the problem is gone; nothing else were updated).
• Yes there are. In a abstract case, a function from hashlib module can be represented by any callble object, and the code of kex_* modules should work anyway.

For an example the test case can be the following:

1. Wrap functions from hashlib module with the decorator(example right after the list).
2. Use decorated functions instead of original ones.
3. Call of hash_algo for an instance should working.

def wrapper(function):
    @wraps(function)
    def wrapped(*args, **kwargs):
        return function(*args, **kwargs)
    return wrapped

sha256 = wrapper(hashlib.sha256)

Here is more general example that explain the issue(you will get the error with python 2 and python 3):

import hashlib
from functools import wraps


def wrapper(function):
    @wraps(function)
    def wrapped(*args, **kwargs):
        return function(*args, **kwargs)
    return wrapped


class JustAExample():
    from_lib = hashlib.sha256
    wrapped = wrapper(hashlib.sha256)
    wrapped_static = staticmethod(wrapper(hashlib.sha256))
    from_lib_static = staticmethod(hashlib.sha256)


j = JustAExample()

# will work
j.wrapped_static()
# will work
j.from_lib_static()
# may exhibit the issue or may not
j.from_lib()
# definitely exhibit the issue
j.wrapped()

[a1ds]

I compared hashlib modules from packages rh-python36-python-libs-3.6.3-1.el7.x86_64.rpm and rh-python36-python-libs-3.6.3-3.el7.x86_64.rpm, and here is the diff:

275c275
<         if 'sha3_' or 'blake2' or 'shake_' in __func_name:
---
>         if __func_name.startswith(('sha3_', 'blake2', 'shake_')):

if statement from 3.6.3-1 is always True, and all of the functions of the module will be wrapped with func = __ignore_usedforsecurity(func), and this exhibited the issue in paramiko modules.

Code snipet from hashlib of 3.6.3-1:

for __func_name in __always_supported:
    # try them all, some may not work due to the OpenSSL
    # version not supporting that algorithm.
    try:
        func = __get_hash(__func_name)
        if 'sha3_' or 'blake2' or 'shake_' in __func_name:
            func = __ignore_usedforsecurity(func)
        globals()[__func_name] = func
    except ValueError:
        import logging
        logging.exception('code for hash %s was not found.', __func_name)

Code snipet from hashlib of 3.6.3-3:

for __func_name in __always_supported:
    # try them all, some may not work due to the OpenSSL
    # version not supporting that algorithm.
    try:
        func = __get_hash(__func_name)
        if __func_name.startswith(('sha3_', 'blake2', 'shake_')):
            func = __ignore_usedforsecurity(func)
        globals()[__func_name] = func
    except ValueError:
        import logging
        logging.exception('code for hash %s was not found.', __func_name)

[ploxiln]

ah, this is from a redhat/centos/fedora custom patch

https://github.com/fedora-python/python36/blob/master/00146-hashlib-fips.patch

https://git.centos.org/blob/rpms!rh-python36-python.git/1f1fb3303e9a5ae48b09311a810bb3227cfebb0c/SOURCES!00146-hashlib-fips.patch

[ploxiln]

So, basically, a bug in redhat/centos/fedora, fixed in redhat/centos/fedora

Identified: https://bugzilla.redhat.com/show_bug.cgi?id=1497342#c12
fixed: https://git.centos.org/commitdiff/rpms!rh-python36-python.git/1f1fb3303e9a5ae48b09311a810bb3227cfebb0c