Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_2_KeyboardInterrupt failing on Python 3.5 #1406

Closed
ghost opened this issue Feb 6, 2016 · 7 comments
Closed

test_2_KeyboardInterrupt failing on Python 3.5 #1406

ghost opened this issue Feb 6, 2016 · 7 comments
Labels
bug major

Comments

@ghost
Copy link

ghost commented Feb 6, 2016

Originally reported by: Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco)

======================================================================
ERROR: test_2_KeyboardInterrupt (cherrypy.test.test_states.ServerStateTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jaraco/Dropbox/code/public/cherrypy/cherrypy/test/test_states.py", line 209, in test_2_KeyboardInterrupt
    self.getPage("/ctrlc")
  File "/Users/jaraco/Dropbox/code/public/cherrypy/cherrypy/test/helper.py", line 349, in getPage
    protocol)
  File "/Users/jaraco/Dropbox/code/public/cherrypy/cherrypy/test/webtest.py", line 255, in getPage
    self.HTTP_CONN, protocol or self.PROTOCOL)
  File "/Users/jaraco/Dropbox/code/public/cherrypy/cherrypy/test/webtest.py", line 558, in openURL
    conn.endheaders()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1079, in endheaders
    self._send_output(message_body)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 911, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 854, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 826, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 711, in create_connection
    raise err
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 702, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused
-------------------- >> begin captured logging << --------------------
cherrypy.error: INFO: [06/Feb/2016:08:17:31] ENGINE Bus STARTING
cherrypy.error: INFO: [06/Feb/2016:08:17:31] ENGINE Started monitor thread 'Session cleanup'.
cherrypy.error: INFO: [06/Feb/2016:08:17:31] ENGINE Started monitor thread '_TimeoutMonitor'.
cherrypy.error: INFO: [06/Feb/2016:08:17:32] ENGINE Serving on http://127.0.0.1:54583
cherrypy.error: INFO: [06/Feb/2016:08:17:32] ENGINE Bus STARTED
cherrypy.error: INFO: [06/Feb/2016:08:17:32] ENGINE Already serving on http://127.0.0.1:54583
cherrypy.access.4400260936: INFO: 127.0.0.1 - - [06/Feb/2016:08:17:32] "GET / HTTP/1.1" 200 11 "" ""
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE <Ctrl-C> hit: shutting down HTTP server
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus STOPPING
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 54583)) shut down
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Stopped thread '_TimeoutMonitor'.
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Stopped thread 'Session cleanup'.
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus STOPPED
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus EXITING
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus EXITED
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Waiting for child threads to terminate...
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus STARTING
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Started monitor thread 'Session cleanup'.
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Started monitor thread '_TimeoutMonitor'.
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Serving on http://127.0.0.1:54583
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus STARTED
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Already serving on http://127.0.0.1:54583
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE <Ctrl-C> hit: shutting down HTTP server
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus STOPPING
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 54583)) shut down
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Stopped thread '_TimeoutMonitor'.
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Stopped thread 'Session cleanup'.
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus STOPPED
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus EXITING
cherrypy.error: INFO: [06/Feb/2016:08:17:33] ENGINE Bus EXITED
--------------------- >> end captured logging << ---------------------
@ghost
Copy link
Author

ghost commented Feb 6, 2016

Original comment by Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco):

The error seems to be related to how connections are retried now. I'm not sure why a call to endheaders should trigger a reconnect. In what world is it viable to reconnect a dropped HTTP connection and resume at the endheaders stage? It seems a little naive to attempt a reconnect during any send call.

I'm tempted in the test to just be more lenient and catch that error as well.

@ghost
Copy link
Author

ghost commented Feb 6, 2016

Original comment by Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco):

I tried replicating the failure using a simple http server that disconnects on request:

import http.server


class DisconnectingHandler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        self.connection.close()


def run():
    httpd = http.server.HTTPServer(("", 8000), DisconnectingHandler)
    httpd.handle_request()


if __name__ == '__main__':
    run()

And calling it simply with the client:

import http.client
conn = http.client.HTTPConnection('localhost', 8000)
conn.putrequest('GET', '/', skip_host=True, skip_accept_encoding=True)
conn.putheader('Host', 'localhost')
conn.endheaders()

But that doesn't fail. I'm now starting to think that the monkeypatch of putrequest on the HTTPConnection in webtest may be at fault.

@ghost
Copy link
Author

ghost commented Feb 6, 2016

Original comment by Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco):

I've removed the monkeypatch, but the failure remains the same.

@ghost
Copy link
Author

ghost commented Feb 25, 2016

Original comment by Anonymous:

I'm packaging CherryPy into Fedora and I've encountered the very same test failure when building against python 3.5 (for upcoming fedora24).

Here's the buildlog: https://copr-be.cloud.fedoraproject.org/results/torsava/python3-cherrypy/fedora-rawhide-x86_64/00163068-python3-cherrypy/build.log.gz

Note that this is after I've already applied a patch made from pull request #120 that successfully solves issue #1382.

@ghost
Copy link
Author

ghost commented Mar 4, 2016

Original comment by Joel Rivera (Bitbucket: cyraxjoe, GitHub: cyraxjoe):

The issue caused because of a new exception on the http.client module, the RemoteDisconnected exception. Is specially problematic because it inherits from socket.error which since 3.3 is an alias to OSError.

I have a fix for this issue on my fork. Which I'll probably integrate the change to cherrypy/default on the weekend if no concerns or other alternatives are suggested.

@ghost
Copy link
Author

ghost commented Mar 9, 2016

Original comment by Anonymous:

@cyraxjoe With the patch from your fork, CherryPy now successfully builds against Python 3.5 in Fedora 24, thanks for your work!

@ghost
Copy link
Author

ghost commented Mar 10, 2016

Original comment by Joel Rivera (Bitbucket: cyraxjoe, GitHub: cyraxjoe):

Add new argument to openURL for the testsuite to be able to
raise subclasses of socket.error (since python3.3 OSError)
instead of retrying.

This fixes the KeyboardError problem for 3.5 by specifying that
subclasses of BadStatusLine can be raised.

The longer explanation of the fix:

From python3.5 a new exception is retuned when the connection
ends abruptly:
  http.client.RemoteDisconnected
RemoteDisconnected is a subclass of:
  (ConnectionResetError, http.client.BadStatusLine)
and ConnectionResetError is an indirect subclass of:
   OSError
From python 3.3 an up socket.error is an alias to OSError
following PEP-3151, therefore http.client.RemoteDisconnected
is considered a socket.error.

raise_subcls specifies the classes that are not going
to be considered as a socket.error for the retries.
Given that RemoteDisconnected is part BadStatusLine
we can use the same call for all py3 versions without
sideffects. python < 3.5 will raise directly BadStatusLine
which is not a subclass for socket.error/OSError.

Fixes issue #1406.

@ghost ghost added major bug labels Apr 30, 2016
@ghost ghost closed this as completed Apr 30, 2016
jaraco pushed a commit that referenced this issue Apr 30, 2016
@cyraxjoe
Add new argument to openURL for the testsuite to be able to
ecfc80d 
raise subclasses of socket.error (since python3.3 OSError)
instead of retrying.

This fixes the KeyboardError problem for 3.5 by specifying that
subclasses of BadStatusLine can be raised.

The longer explanation of the fix:

    From python3.5 a new exception is retuned when the connection
    ends abruptly:
      http.client.RemoteDisconnected
    RemoteDisconnected is a subclass of:
      (ConnectionResetError, http.client.BadStatusLine)
    and ConnectionResetError is an indirect subclass of:
       OSError
    From python 3.3 an up socket.error is an alias to OSError
    following PEP-3151, therefore http.client.RemoteDisconnected
    is considered a socket.error.

    raise_subcls specifies the classes that are not going
    to be considered as a socket.error for the retries.
    Given that RemoteDisconnected is part BadStatusLine
    we can use the same call for all py3 versions without
    sideffects. python < 3.5 will raise directly BadStatusLine
    which is not a subclass for socket.error/OSError.

Fixes issue #1406.
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug major
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
0 participants