ssl error: SSL3_WRITE_PENDING

[Irooniam]

Hello,

I'm using gevent-socketio to create websocket connections and to serve up static assets (html, css, js, etc).

Everything works fine, however if I try to reload the page very quickly, I get the error below.

Any help trying to isolate the problem would be appreciated.

Python 2.7.3
Gevent 0.13.8 (latest)
Gevent-socketio 0.3.6

x.x.x.x - - [2013-09-25 19:45:09] "GET /static/socket.io.js HTTP/1.1" 500 0 0.162437
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
result = self._run(_self.args, *_self.kwargs)
File "/usr/lib/python2.7/site-packages/gevent/server.py", line 160, in wrap_socket_and_handle
return self.handle(ssl_socket, address)
File "build/bdist.linux-x86_64/egg/socketio/server.py", line 113, in handle
handler.handle()
File "/usr/lib/python2.7/site-packages/gevent/pywsgi.py", line 180, in handle
result = self.handle_one_request()
File "build/bdist.linux-x86_64/egg/socketio/handler.py", line 185, in handle_one_request
handleResult = super(SocketIOHandler, self).handle_one_request()
File "/usr/lib/python2.7/site-packages/gevent/pywsgi.py", line 314, in handle_one_request
self.handle_one_response()
File "build/bdist.linux-x86_64/egg/socketio/handler.py", line 96, in handle_one_response
return super(SocketIOHandler, self).handle_one_response()
File "/usr/lib/python2.7/site-packages/gevent/pywsgi.py", line 455, in handle_one_response
self.write(_INTERNAL_ERROR_BODY)
File "/usr/lib/python2.7/site-packages/gevent/pywsgi.py", line 373, in write
self.socket.sendall(msg)
File "/usr/lib/python2.7/site-packages/gevent/socket.py", line 509, in sendall
data_sent += self.send(_get_memory(data, data_sent), flags)
File "/usr/lib/python2.7/site-packages/gevent/ssl.py", line 188, in send
v = self._sslobj.write(data)
SSLError: [Errno 1] _ssl.c:1242: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry
<Greenlet at 0x6228af0: <bound method SocketIOServer.wrap_socket_and_handle of <SocketIOServer at 0x2049990 fileno=32 address=0.0.0.0:7000>>(<socket at 0x6711b50 fileno=-1>, ('x.x.x.x', 46504))> failed with SSLError

[nezza]

I have the same problem with the same versions. Happens primarily when using send/sendall to send a big chunk of data.

[jamadden]

Are you still able to reproduce this on a 1.1a version?

[chibisov]

I have the same problem with gevent 1.1rc1

[jamadden]

Can you post your versions and a current traceback?

[chibisov]

Python 2.7.0

Traceback:

File "flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "docast/common/views/mixins.py", line 10, in dispatch_request
    return super(HandleAuthErrorsMixin, self).dispatch_request(*args, **kwargs)
  File "flask/views.py", line 149, in dispatch_request
    return meth(*args, **kwargs)
  File "docast/feed/views/services/base.py", line 205, in get
    file_url = self.get_file_url(social_account, feed, data, file_ext)
  File "docast/feed/views/services/base_resource.py", line 167, in get_file_url
    'html': str(response.content)
  File "docast/core/utils.py", line 385, in update_resource
    'payload': data
  File "docast/common/utils.py", line 187, in do_deformio_request
    response = method_func(**request_kwargs)
  File "requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "requests/packages/urllib3/connectionpool.py", line 353, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "python2.7/httplib.py", line 1053, in request
    self._send_request(method, url, body, headers)
  File "python2.7/httplib.py", line 1093, in _send_request
    self.endheaders(body)
  File "python2.7/httplib.py", line 1049, in endheaders
    self._send_output(message_body)
  File "python2.7/httplib.py", line 893, in _send_output
    self.send(msg)
  File "python2.7/httplib.py", line 869, in send
    self.sock.sendall(data)
  File "requests/packages/urllib3/contrib/pyopenssl.py", line 220, in sendall
    sent = self._send_until_done(data[total_sent:total_sent+SSL_WRITE_BLOCKSIZE])
  File "requests/packages/urllib3/contrib/pyopenssl.py", line 206, in _send_until_done
    return self.connection.send(data)
  File "OpenSSL/SSL.py", line 1271, in send
    self._raise_ssl_error(self._ssl, result)
  File "OpenSSL/SSL.py", line 1187, in _raise_ssl_error
    _raise_current_error()
  File "OpenSSL/_util.py", line 48, in exception_from_error_queue
    raise exception_type(errors)

OpenSSL._util in exception_from_error_queue
Error: [('SSL routines', 'ssl3_write_pending', 'bad write retry')]

Actually I've fixed my problem. I had requests[security]==2.8.1 in my requirements. When I've changed it to requests==2.8.1 it stopped failing.

[jamadden]

Agreed, I don't see a gevent library in that last traceback anywhere, only requests and pyopenssl, and pyopenssl is not monkey-patched or supported by gevent. Only the standard-library ssl module is, and using gevent non-blocking sockets with pyopenssl is unlikely to work seamlessly: unlike gevent's ssl module which (is supposed to) gracefully handle waiting, OpenSSL.SSL.Connection:send is explicitly documented as raising exceptions on things like this.

That said, gevent's ssl module doesn't explicitly handle this error either---because it's basically a programming error and a legitimate bug in either gevent or the calling application. (The bug could only be gevent's in the case of a monkey-patch call to sendall.) There's a good explanation on this stackoverflow question. It boils down to calling OpenSSL's write with different arguments after a timeout has occurred.

In gevent's case, this was possible in 0.13.6 if the socket's timeout was 0 and sendall was called. gevent's sendall calculated a new buffer for each send call, and they weren't guaranteed to be the same. This was mostly fixed in the 1.1 series as the buffers used are smaller and more likely to be sent, but I see there's still a corner case where it could crop up. I'll see if I can come up with a reproducible test and fix.

[jamadden]

This should be fixed, as far as I can determine (and minimally test). Please open a new issue if you can still produce it with master/1.1rc2 or later.