Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 107 lines (82 sloc) 3.979 kb
39938c4 @nvie Initial bulk of the drainers package.
authored
1 ===================================================
2 drainers - Event-based draining of process output
3 ===================================================
4
7facdb2 @nvie Add stillmaintained status.
authored
5 .. image:: http://stillmaintained.com/nvie/python-drainers.png
6
467fcd2 @nvie Remove Sphinx-related reference definitions.
authored
7 drainers is an abstraction around `subprocess.Popen` to read and control
abd5594 @nvie Add usage documentation.
authored
8 process output event-wise. It also allows you to abort running processes
9 either gracefully or forcefully without having to directly interact with the
10 processes or threads themself.
39938c4 @nvie Initial bulk of the drainers package.
authored
11
12 Overview
13 ========
14
abd5594 @nvie Add usage documentation.
authored
15 Defining a process
16 ------------------
467fcd2 @nvie Remove Sphinx-related reference definitions.
authored
17 A `Drainer` is a factory and controller wrapper around
18 `subprocess.Popen` and therefore takes all of the (optional) parameters
19 that `subprocess.Popen`'s initializer takes. For example, the minimal
20 `Drainer` takes a command array::
39938c4 @nvie Initial bulk of the drainers package.
authored
21
abd5594 @nvie Add usage documentation.
authored
22 from drainers import Drainer
23
24 def ignore_event(line, is_err):
25 pass
26
27 my_drainer = Drainer(['ls', '-la'], read_event_cb=ignore_event)
5cb3120 @nvie Starting helps.
authored
28 my_drainer.start()
abd5594 @nvie Add usage documentation.
authored
29
30 But, extra arguments are allowed, too::
31
32 my_drainer = Drainer(['echo', '$JAVA_HOME'], shell=True, bufsize=64,
33 read_event_cb=ignore_event)
5cb3120 @nvie Starting helps.
authored
34 my_drainer.start()
abd5594 @nvie Add usage documentation.
authored
35
467fcd2 @nvie Remove Sphinx-related reference definitions.
authored
36 The only two arguments to `Drainer` that are reserved are
37 `stdout` and `stderr`. `Drainer` requires them to be
38 `subprocess.PIPE` explicitly, and sets them for you accordingly.
abd5594 @nvie Add usage documentation.
authored
39
40 Defining a callback
41 -------------------
467fcd2 @nvie Remove Sphinx-related reference definitions.
authored
42 `Drainer`'s strength lies in the fact that each line that is read from the
43 process' standard output or standard error streams leads to a callback
abd5594 @nvie Add usage documentation.
authored
44 function being invoked. This allows you to process virtually any process'
45 output, as long as it's line-based.
46
47 The callback function can be specified using the `read_event_cb` parameter to
48 the constructor, as seen in the example above. It is mandatory. The callback
49 function specified needs to have a specific signature::
50
51 def my_callback(line, is_err):
52 ...
53
467fcd2 @nvie Remove Sphinx-related reference definitions.
authored
54 It should take two parameters: `line` (a string) and `is_err` (a boolean).
55 The latter indicates that the line is read from the standard error stream.
56 There is nothing more to it. It does not need to return anything: it's return
57 value will be ignored. Your callback may be a class method, too, like in the
abd5594 @nvie Add usage documentation.
authored
58 following example. Notice that in those cases, you pass `foo.my_method` as
59 the value for the `read_event_cb` parameter::
60
61 class MyClass(object):
62
63 def my_method(self, line, is_err):
64 ...
65
66 foo = MyClass()
67 my_drainer = Drainer(['ls'], read_event_cb=foo.my_method)
5cb3120 @nvie Starting helps.
authored
68 my_drainer.start()
abd5594 @nvie Add usage documentation.
authored
69
70 The granularity currently is a single line. If you want to read predefined
1d65916 @nvie Updated README and example to mention BufferedDrainer.
authored
71 chunks (lines) of data, use `BufferedDrainer` instead. See
72 examples/buffer_results.py for an example.
abd5594 @nvie Add usage documentation.
authored
73
74 Aborting processes
75 ------------------
467fcd2 @nvie Remove Sphinx-related reference definitions.
authored
76 `Drainer` allows you to abort a running process in the middle of execution,
77 forcefully sending the process a `terminate()` message (Python equivalent of a
78 Unix `SIGTERM` message) when a certain condition arises. By default, the
79 process will never be terminated abnormally. To specify termination criteria,
80 implement a callback function that takes no parameters and returns `True` if
81 abortion is desired and `False` otherwise. For example, for a long running
82 process you might want to terminate it if the disk is getting (almost) full.
83 But checking how much space is free can be a lengthy operation, so you might
84 want to do it only sparingly::
abd5594 @nvie Add usage documentation.
authored
85
86 def out_of_diskspace():
87 left = handytools.check_disk_free()
88 total = handytools.check_disk_total()
89 return (left / total) < 0.03
90
91 # The following drainer executes the cruncher and checks whether the disk
92 # is (almost) full every 5 seconds. It aborts if free disk space runs
93 # under 3%.
94 my_drainer = Drainer(['/bin/crunch', 'inputfile', 'outputfile'],
95 read_event_cb=ignore_event,
96 should_abort=out_of_diskspace,
97 check_interval=5.0)
98 exitcode = my_drainer.start()
99
100 The example is pretty self-explaining. You can check the exitcode to see the
101 result of the process.
39938c4 @nvie Initial bulk of the drainers package.
authored
102
103
abd5594 @nvie Add usage documentation.
authored
104 More examples
105 =============
467fcd2 @nvie Remove Sphinx-related reference definitions.
authored
106 See the `examples` directory for more detailed examples.
Something went wrong with that request. Please try again.