Eventlet green threads not released back to pool

Presently, the wsgi server allows persist connections. Hence even after
the response is sent to the client, it doesn't close the client socket
connection. Because of this problem, the green thread is not released
back to the pool.

In order to close the client socket connection explicitly after the
response is sent and read successfully by the client, you simply have to
set keepalive to False when you create a wsgi server.

Added a parameter to take advantage of the new(ish) eventlet socket timeout
behaviour. Allows closing idle client connections after a period of time, eg:

$ time nc localhost 8776
real    1m0.063s

Setting 'client_socket_timeout = 0' means do not timeout.

DocImpact:
Added wsgi_keep_alive option (default=True).
Added client_socket_timeout option (default=900).

SecurityImpact

Closes-Bug: #1361360
Change-Id: I3a361d6590d1800b85791f23ac1cdfd79815341b

etc/neutron.conf

@@ -201,6 +201,18 @@ lock_path = $state_path/lock
 # enabled for various plugins for compatibility.
 # rpc_workers = 0
 
+# Timeout for client connections socket operations. If an
+# incoming connection is idle for this number of seconds it
+# will be closed. A value of '0' means wait forever. (integer
+# value)
+# client_socket_timeout = 900
+
+# wsgi keepalive option. Determines if connections are allowed to be held open
+# by clients after a request is fulfilled. A value of False will ensure that
+# the socket connection will be explicitly closed once a response has been
+# sent to the client.
+# wsgi_keep_alive = True
+
 # Sets the value of TCP_KEEPIDLE in seconds to use for each server socket when
 # starting API server. Not supported on OS X.
 # tcp_keepidle = 600

neutron/wsgi.py

@@ -70,6 +70,18 @@
     cfg.StrOpt('ssl_key_file',
                help=_("Private key file to use when starting "
                       "the server securely")),
+    cfg.BoolOpt('wsgi_keep_alive',
+                default=True,
+                help=_("Determines if connections are allowed to be held "
+                     "open by clients after a request is fulfilled. A value "
+                     "of False will ensure that the socket connection will "
+                     "be explicitly closed once a response has been sent to "
+                     "the client.")),
+    cfg.IntOpt('client_socket_timeout', default=900,
+               help=_("Timeout for client connections socket operations. "
+                    "If an incoming connection is idle for this number of "
+                    "seconds it will be closed. A value of '0' means "
+                    "wait forever.")),
 ]
 
 CONF = cfg.CONF
@@ -113,6 +125,9 @@ def __init__(self, name, threads=1000):
         self.pool = eventlet.GreenPool(threads)
         self.name = name
         self._server = None
+        # A value of 0 is converted to None because None is what causes the
+        # wsgi server to wait forever.
+        self.client_socket_timeout = CONF.client_socket_timeout or None
 
     def _get_socket(self, host, port, backlog):
         bind_addr = (host, port)
@@ -241,7 +256,9 @@ def wait(self):
     def _run(self, application, socket):
         """Start a WSGI server in a new green thread."""
         eventlet.wsgi.server(socket, application, custom_pool=self.pool,
-                             log=logging.WritableLogger(LOG))
+                             log=logging.WritableLogger(LOG),
+                             keepalive=CONF.wsgi_keep_alive,
+                             socket_timeout=self.client_socket_timeout)
 
 
 class Middleware(object):