Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Added thread locals and subprocess guides.

  • Loading branch information...
commit 67d97a1118f84521947da94418e7b3dd887b8a8c 1 parent 588e36e
@sdiehl authored
Showing with 297 additions and 12 deletions.
  1. +154 −11 index.html
  2. +143 −1 tutorial.md
View
165 index.html
@@ -65,12 +65,13 @@ <h3 class="author">
<li><a href="#groups-and-pools">Groups and Pools</a></li>
<li><a href="#locks-and-semaphores">Locks and Semaphores</a></li>
<li><a href="#thread-locals">Thread Locals</a></li>
+<li><a href="#subprocess">Subprocess</a></li>
<li><a href="#actors">Actors</a></li>
</ul>
</li>
<li><a href="#real-world-applications">Real World Applications</a><ul>
<li><a href="#gevent-zeromq">Gevent ZeroMQ</a></li>
-<li><a href="#simple-telnet-servers">Simple Telnet Servers</a></li>
+<li><a href="#simple-servers">Simple Servers</a></li>
<li><a href="#wsgi-servers">WSGI Servers</a></li>
<li><a href="#streaming-servers">Streaming Servers</a></li>
<li><a href="#long-polling">Long Polling</a></li>
@@ -249,16 +250,16 @@ <h2 id="synchronous-asynchronous-execution">Synchronous &amp; Asynchronous Execu
Task 8 done
Task 9 done
Asynchronous:
-Task 0 done
-Task 6 done
Task 1 done
-Task 9 done
-Task 3 done
-Task 4 done
Task 2 done
-Task 7 done
Task 5 done
+Task 7 done
+Task 9 done
Task 8 done
+Task 0 done
+Task 3 done
+Task 4 done
+Task 6 done
</pre></code></p>
<p>In the synchronous case all the tasks are run sequentially,
which results in the main programming <em>blocking</em> (
@@ -895,11 +896,11 @@ <h2 id="groups-and-pools">Groups and Pools</h2>
<p></code>
<pre><code class="python">
Size of group 3
-Hello from Greenlet 34519152
+Hello from Greenlet 41900624
Size of group 3
-Hello from Greenlet 34518512
+Hello from Greenlet 41900944
Size of group 3
-Hello from Greenlet 34519472
+Hello from Greenlet 41899024
Ordered
('task', 0)
('task', 1)
@@ -1011,6 +1012,148 @@ <h2 id="locks-and-semaphores">Locks and Semaphores</h2>
ensure that resources are only in use at one time in the context
of a program.</p>
<h2 id="thread-locals">Thread Locals</h2>
+<p>Gevnet also allows you to specify data which is local the
+greenlet context. Internally this is implemented as a global
+lookup which addresses a private namespace keyed by the
+greenlet's <code>getcurrent()</code> value.</p>
+<pre><code class="python">
+import gevent
+from gevent.local import local
+
+stash = local()
+
+def f1():
+ stash.x = 1
+ print(stash.x)
+
+def f2():
+ stash.y = 2
+ print(stash.y)
+
+ try:
+ stash.x
+ except AttributeError:
+ print("x is not local to f2")
+
+g1 = gevent.spawn(f1)
+g2 = gevent.spawn(f2)
+
+gevent.joinall([g1, g2])
+</pre>
+
+<p></code>
+<pre><code class="python">
+1
+2
+x is not local to f2
+</pre></code></p>
+<p>Many web framework thats integrate with gevent store HTTP session
+objects inside of gevent thread locals. For example using the
+Werkzeug utility library and its proxy object we can create
+Flask style request objects.</p>
+<pre>
+<code class="python">from gevent.local import local
+from werkzeug.local import LocalProxy
+from werkzeug.wrappers import Request
+from contextlib import contextmanager
+
+from gevent.wsgi import WSGIServer
+
+_requests = local()
+request = LocalProxy(lambda: _requests.request)
+
+@contextmanager
+def sessionmanager(environ):
+ _requests.request = Request(environ)
+ yield
+ _requests.request = None
+
+def logic():
+ return "Hello " + request.remote_addr
+
+def application(environ, start_response):
+ status = '200 OK'
+
+ with sessionmanager(environ):
+ body = logic()
+
+ headers = [
+ ('Content-Type', 'text/html')
+ ]
+
+ start_response(status, headers)
+ return [body]
+
+WSGIServer(('', 8000), application).serve_forever()
+
+<code>
+</pre>
+
+<p>Flask's system is more a bit sophisticated than this example, but the
+idea of using thread locals as local session storage is nontheless the
+same.</p>
+<h2 id="subprocess">Subprocess</h2>
+<p>As of Gevent 1.0, support has been added for cooperative waiting
+on subprocess.</p>
+<pre>
+<code class="python">import gevent
+from gevent.subprocess import Popen, PIPE
+
+# Uses a green pipe which is cooperative
+sub = Popen(['uname'], stdout=PIPE)
+read_output = gevent.spawn(sub.stdout.read)
+
+output = read_output.join()
+print(output.value)
+<code>
+</pre>
+
+<pre>
+<code class="python">Linux
+<code>
+</pre>
+
+<p>Many people also want to use gevent and multiprocessing together. This
+can be done as most multiprocessing objects expose the underlying file
+descriptors.</p>
+<pre><code class="python">
+import gevent
+from multiprocessing import Process, Pipe
+from gevent.socket import wait_read, wait_write
+
+# To Process
+a, b = Pipe()
+
+# From Process
+c, d = Pipe()
+
+def relay():
+ for i in xrange(10):
+ msg = b.recv()
+ c.send(msg + " in " + str(i))
+
+def put_msg():
+ for i in xrange(10):
+ wait_write(a.fileno())
+ a.send('hi')
+
+def get_msg():
+ for i in xrange(10):
+ wait_read(d.fileno())
+ print(d.recv())
+
+if __name__ == '__main__':
+ proc = Process(target=relay)
+ proc.start()
+
+ g1 = gevent.spawn(get_msg)
+ g2 = gevent.spawn(put_msg)
+ gevent.joinall([g1, g2], timeout=1)
+</pre>
+
+<p></code>
+<pre><code class="python">
+</pre></code></p>
<h2 id="actors">Actors</h2>
<p>The actor model is a higher level concurrency model popularized
by the language Erlang. In short the main idea is that you have a
@@ -1146,7 +1289,7 @@ <h2 id="gevent-zeromq">Gevent ZeroMQ</h2>
Switched to Server for 9
Switched to Client for 9
</pre></code></p>
-<h2 id="simple-telnet-servers">Simple Telnet Servers</h2>
+<h2 id="simple-servers">Simple Servers</h2>
<pre>
<code class="python">
# On Unix: Access with ``$ nc 127.0.0.1 5000``
View
144 tutorial.md
@@ -833,6 +833,148 @@ of a program.
## Thread Locals
+Gevnet also allows you to specify data which is local the
+greenlet context. Internally this is implemented as a global
+lookup which addresses a private namespace keyed by the
+greenlet's ``getcurrent()`` value.
+
+[[[cog
+import gevent
+from gevent.local import local
+
+stash = local()
+
+def f1():
+ stash.x = 1
+ print(stash.x)
+
+def f2():
+ stash.y = 2
+ print(stash.y)
+
+ try:
+ stash.x
+ except AttributeError:
+ print("x is not local to f2")
+
+g1 = gevent.spawn(f1)
+g2 = gevent.spawn(f2)
+
+gevent.joinall([g1, g2])
+]]]
+[[[end]]]
+
+Many web framework thats integrate with gevent store HTTP session
+objects inside of gevent thread locals. For example using the
+Werkzeug utility library and its proxy object we can create
+Flask style request objects.
+
+<pre>
+<code class="python">from gevent.local import local
+from werkzeug.local import LocalProxy
+from werkzeug.wrappers import Request
+from contextlib import contextmanager
+
+from gevent.wsgi import WSGIServer
+
+_requests = local()
+request = LocalProxy(lambda: _requests.request)
+
+@contextmanager
+def sessionmanager(environ):
+ _requests.request = Request(environ)
+ yield
+ _requests.request = None
+
+def logic():
+ return "Hello " + request.remote_addr
+
+def application(environ, start_response):
+ status = '200 OK'
+
+ with sessionmanager(environ):
+ body = logic()
+
+ headers = [
+ ('Content-Type', 'text/html')
+ ]
+
+ start_response(status, headers)
+ return [body]
+
+WSGIServer(('', 8000), application).serve_forever()
+
+
+<code>
+</pre>
+
+Flask's system is more a bit sophisticated than this example, but the
+idea of using thread locals as local session storage is nontheless the
+same.
+
+## Subprocess
+
+As of Gevent 1.0, support has been added for cooperative waiting
+on subprocess.
+
+<pre>
+<code class="python">import gevent
+from gevent.subprocess import Popen, PIPE
+
+# Uses a green pipe which is cooperative
+sub = Popen(['uname'], stdout=PIPE)
+read_output = gevent.spawn(sub.stdout.read)
+
+output = read_output.join()
+print(output.value)
+<code>
+</pre>
+
+<pre>
+<code class="python">Linux
+<code>
+</pre>
+
+Many people also want to use gevent and multiprocessing together. This
+can be done as most multiprocessing objects expose the underlying file
+descriptors.
+
+[[[cog
+import gevent
+from multiprocessing import Process, Pipe
+from gevent.socket import wait_read, wait_write
+
+# To Process
+a, b = Pipe()
+
+# From Process
+c, d = Pipe()
+
+def relay():
+ for i in xrange(10):
+ msg = b.recv()
+ c.send(msg + " in " + str(i))
+
+def put_msg():
+ for i in xrange(10):
+ wait_write(a.fileno())
+ a.send('hi')
+
+def get_msg():
+ for i in xrange(10):
+ wait_read(d.fileno())
+ print(d.recv())
+
+if __name__ == '__main__':
+ proc = Process(target=relay)
+ proc.start()
+
+ g1 = gevent.spawn(get_msg)
+ g2 = gevent.spawn(put_msg)
+ gevent.joinall([g1, g2], timeout=1)
+]]]
+[[[end]]]
+
## Actors
The actor model is a higher level concurrency model popularized
@@ -957,7 +1099,7 @@ gevent.joinall([publisher, client])
]]]
[[[end]]]
-## Simple Telnet Servers
+## Simple Servers
<pre>
<code class="python">
Please sign in to comment.
Something went wrong with that request. Please try again.