New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pyopenssl adapter under Python 3 #113

Closed
webknjaz opened this Issue Sep 3, 2018 · 10 comments

Comments

Projects
None yet
2 participants
@webknjaz
Copy link
Member

webknjaz commented Sep 3, 2018

It works under Python 2, but not under Python 3.
It's a follow-up for #6

I'm submitting a ...

  • bug report
  • feature request
  • question about the decisions made in the repository

Describe the bug. What is the current behavior?

It raises exception OpenSSL.SSL.WantReadError.

What is the motivation / use case for changing the behavior?
I want consistency in SSL.

To Reproduce

Steps to reproduce the behavior: tox -- --no-cov -s cheroot/test/test_ssl.py

Expected behavior

That it'd work.

Details

 cheroot/test/test_ssl.py::test_ssl_adapters[builtin] ✓                                                          50% █████     
WantReadError()                                                                                                               
Traceback (most recent call last):        
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1219, in communicate                                    
    req.parse_request()            
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 689, in parse_request                                   
    success = self.read_request_line()        
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 728, in read_request_line                               
    request_line = self.rfile.readline()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 277, in readline                                        
    data = self.rfile.readline(256)         
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 513, in readline                                          
    b = self.read(nreadahead())     
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)      
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1076, in peek                                           
    return self._peek_unlocked(size)                                                                                          
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked                                   
    current = self.raw.read(to_read)                                                                                           
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)                                                                                             
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)                       
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()                                                                        
OpenSSL.SSL.WantReadError
Exception in thread CP Server Thread-13:                                                                                      Traceback (most recent call last):
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1219, in communicate
    req.parse_request()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 689, in parse_request
    success = self.read_request_line()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 728, in read_request_line
    request_line = self.rfile.readline()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 277, in readline
    data = self.rfile.readline(256)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 513, in readline
    b = self.read(nreadahead())
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1076, in peek
    return self._peek_unlocked(size)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked
    current = self.raw.read(to_read)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/workers/threadpool.py", line 112, in run                                     conn.communicate()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1256, in communicate
    self._conditional_error(req, '500 Internal Server Error')
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1282, in _conditional_error                                 req.simple_response(response)                             
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1071, in simple_response
    self.conn.wfile.write(EMPTY.join(buf))
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/makefile.py", line 31, in write
    self._flush_unlocked()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/makefile.py", line 40, in _flush_unlocked
    n = self.raw.write(bytes(self._write_buf))
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 607, in write
    return self._sock.send(b)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1729, in send
    self._raise_ssl_error(self._ssl, result)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

Environment

  • Cheroot version: master (6.5.2)
  • CherryPy version: N/A
  • Python version: 3.X
  • OS: XXX
  • Browser: N/A

Additional context

@webknjaz

This comment has been minimized.

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Sep 3, 2018

@webknjaz

This comment has been minimized.

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Sep 6, 2018

@webknjaz

This comment has been minimized.

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Dec 30, 2018

Fixed.

@webknjaz webknjaz closed this Dec 30, 2018

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Dec 30, 2018

@mar10 could you please try out PyOpenSSL support now? I think I've finally fixed it.

@mar10

This comment has been minimized.

Copy link
Contributor

mar10 commented Dec 31, 2018

Hi,

gave it a quick test.
The problem was reproducible here: WsgiDAV/3.0.0a8 Cheroot/6.5.3 Python/3.7.0

(wsgidav-9PbuForg) bash-3.2$ python -m OpenSSL.debug
pyOpenSSL: 18.0.0
cryptography: 2.4.2
cffi: 1.11.5
cryptography's compiled against OpenSSL: OpenSSL 1.1.0j  20 Nov 2018
cryptography's linked OpenSSL: OpenSSL 1.1.0j  20 Nov 2018
Pythons's OpenSSL: OpenSSL 1.0.2p  14 Aug 2018
Python executable: /Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/bin/python
Python version: 3.7.0 (default, Jul 26 2018, 22:23:34) 
[Clang 9.1.0 (clang-902.0.39.2)]
Platform: darwin
2018-12-31 12:09:50.866 - INFO    : SSL / HTTPS enabled. Adapter: <class 'cheroot.ssl.pyopenssl.pyOpenSSLAdapter'>
2018-12-31 12:09:50.866 - INFO    : Running WsgiDAV/3.0.0a8 Cheroot/6.5.3 Python/3.7.0
2018-12-31 12:09:50.866 - INFO    : Serving on https://0.0.0.0:8080 ...
WantReadError()
Traceback (most recent call last):
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 1242, in communicate
    req.parse_request()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 712, in parse_request
    success = self.read_request_line()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 751, in read_request_line
    request_line = self.rfile.readline()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 300, in readline
    data = self.rfile.readline(256)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 513, in readline
    b = self.read(nreadahead())
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 1076, in peek
    return self._peek_unlocked(size)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked
    current = self.raw.read(to_read)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

(looks like issue #132)
Seems you nailed it with the current master:

(wsgidav-9PbuForg) bash-3.2$ pip install git+git://github.com/cherrypy/cheroot.git@master
[...]
2018-12-31 12:24:28.789 - INFO    : WsgiDAV/3.0.0a8 Python/3.7.0 Darwin-18.2.0-x86_64-i386-64bit
[...]
2018-12-31 12:24:28.850 - INFO    : SSL / HTTPS enabled. Adapter: <class 'cheroot.ssl.pyopenssl.pyOpenSSLAdapter'>
2018-12-31 12:24:28.851 - INFO    : Running WsgiDAV/3.0.0a8 Cheroot/6.5.4.dev43+gab92fb97 Python/3.7.0
2018-12-31 12:24:28.851 - INFO    : Serving on https://0.0.0.0:8080 ...
2018-12-31 12:24:45.333 - INFO    : 127.0.0.1 - (anonymous) - [2018-12-31 11:24:45] "GET /" elap=0.006sec -> 200 OK

Thanks for keeping up the good work and happy new year 🎉

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Dec 31, 2018

Cool thanks :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment