Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 171 lines (120 sloc) 4.71 kB
d06c6af @schmichael Boring is better
authored
1 =====
2 About
3 =====
01c6bb6 @schmichael Hey it's a readme!
authored
4
1b424db @schmichael Update docs a bit
authored
5 Mmstats is a way to expose and read diagnostic values and metrics for
6 applications.
01c6bb6 @schmichael Hey it's a readme!
authored
7
18d370a @schmichael Cleanup flask example and associated docs
authored
8 Think of mmstats as /proc for your application and the readers as procps
9 utilities.
29f974e @schmichael Compare my half baked barely working lib to venerable unix tools
authored
10
1b424db @schmichael Update docs a bit
authored
11 This project is a Python implementation, but compatible implementations can be
12 made in any language (see Goals).
13
01c6bb6 @schmichael Hey it's a readme!
authored
14 -----
15 Goals
16 -----
17
29f974e @schmichael Compare my half baked barely working lib to venerable unix tools
authored
18 * Separate publishing/writing from consuming/reading tools
01c6bb6 @schmichael Hey it's a readme!
authored
19 * Platform/language independent (a Java writer can be read by a Python tool)
20 * Predictable performance impact for writers via:
7bdb31b @schmichael I suck at rst
authored
21
22 * No locks (1 writer per thread)
18d370a @schmichael Cleanup flask example and associated docs
authored
23 * No syscalls (after instantiation)
7bdb31b @schmichael I suck at rst
authored
24 * All in userspace
25 * Reading has no impact on writers
26
01c6bb6 @schmichael Hey it's a readme!
authored
27 * Optional persistent (writer can sync anytime)
28 * 1-way (Publish/consume only; mmstats are not management extensions)
29
30 =====
31 Usage
32 =====
33
e795b06 @schmichael Can't resize mmap after initializing ctypes; determine total size first
authored
34 ------------
35 Requirements
36 ------------
37
38 CPython 2.6 or 2.7 (Windows is untested)
39
eaa6688 @schmichael Update README to reflect pypy support
authored
40 PyPy (only tested in 1.7, should be faster in 1.8)
e795b06 @schmichael Can't resize mmap after initializing ctypes; determine total size first
authored
41
ad62d11 @schmichael Add setup.py and add usage notes to readme
authored
42 -----
43 Using
44 -----
45
46 1. ``python setup.py install`` # Or copy mmstats.py into your project
47 2. ``import mmstats``
48 3. Create a subclass of mmstats.MmStats like:
49
50 ::
51
52 class WebStats(mmstats.MmStats):
1b424db @schmichael Update docs a bit
authored
53 status2xx = mmstats.CounterField(label='status.2XX')
54 status3xx = mmstats.CounterField(label='status.3XX')
55 status4xx = mmstats.CounterField(label='status.4XX')
56 status5xx = mmstats.CounterField(label='status.5XX')
57 last_hit = mmstats.DoubleField(label='timers.last_hit')
ad62d11 @schmichael Add setup.py and add usage notes to readme
authored
58
59 4. Instantiate it once per thread/process:
60
61 ::
62
63 webstats = WebStats(label_prefix='web.stats.')
64
65 5. Record some data:
66
67 ::
68
69 if response.status_code == 200:
1b424db @schmichael Update docs a bit
authored
70 webstats.status2xx.inc()
71
72 webstats.last_hit = time.time()
ad62d11 @schmichael Add setup.py and add usage notes to readme
authored
73
18d370a @schmichael Cleanup flask example and associated docs
authored
74 6. Run ``slurpstats`` to read it
75 7. Run ``mmash`` to create a web interface for stats
76 8. Run ``pollstats -p web.stats.status 2XX,3XX,4XX,5XX /tmp/mmstats-*`` for a
77 vmstat/dstat like view.
5477c1b @schmichael Mention cleanstats in the readme
authored
78 9. Did a process die unexpectedly and leave around a stale mmstat file?
79 ``cleanstats /path/to/mmstat/files`` will check to see which files are stale
80 and remove them.
ad62d11 @schmichael Add setup.py and add usage notes to readme
authored
81
dfb7dfe @schmichael Add basic development instructions
authored
82 -----------
83 Development
84 -----------
85
86 It's easiest to develop mmstats within a virtualenv:
87
43a7553 @schmichael Fix the dev instructions
authored
88 ::
89
90 $ git clone git://github.com/schmichael/mmstats.git
91 $ cd mmstats
92 $ virtualenv .
93 $ source bin/activate
94 $ python setup.py develop
95 $ ./run_flask_example # This starts up a sample web app
623e97d @schmichael Flesh out the development example a bit more
authored
96 $ curl http://localhost:5001/
97 $ curl http://localhost:5001/500
98 $ curl http://localhost:5001/status
99 $ # If you have ab installed:
100 $ ab -n 50 -c 10 http://localhost:5001/
101
102 Now to view the stats run the following in a new terminal:
103
104 ::
105
106 $ # To get a raw view of the data:
18d370a @schmichael Cleanup flask example and associated docs
authored
107 $ slurpstats mmstats-*
43a7553 @schmichael Fix the dev instructions
authored
108 $ # Or start up the web interface:
623e97d @schmichael Flesh out the development example a bit more
authored
109 $ mmash
18d370a @schmichael Cleanup flask example and associated docs
authored
110 $ # Run pollstats while ab is running:
111 $ pollstats -p flask.example. ok,bad,working mmstats-*
dfb7dfe @schmichael Add basic development instructions
authored
112
18d370a @schmichael Cleanup flask example and associated docs
authored
113 To cleanup stray mmstats files: ``rm mmstats-flask-*``
e5c9385 @schmichael Add cleanup command to docs
authored
114
dfb7dfe @schmichael Add basic development instructions
authored
115 The web interface will automatically reload when you change source files.
116
117 Put static files into static/ and template files into templates/
118
119 TODO: Factor mmash out into it's own project (with a meaningful name?)
120
121 --------
122 Testing
123 --------
01c6bb6 @schmichael Hey it's a readme!
authored
124
c85fefd @schmichael let's go shopping
authored
125 #. Run your favorite Python test runner (py.test or nosetests)
b41b903 @schmichael s/slurptests/slurpstats
authored
126 #. Run slurpstats.py
c85fefd @schmichael let's go shopping
authored
127 #. Clean /tmp/mmstats-* files up
309684e @schmichael rst is hard
authored
128
a6dfa3e @schmichael Begin documenting data structures
authored
129 ===============
130 Data Structures
131 ===============
132
133 There are two types of data structures so far in mmstats:
134
135 #. buffered
136 #. unbuffered
137
138 Buffered structures use multiple buffers for handling values which cannot be
139 written atomically.
140
141 Unbuffered structures have ff in the write buffer field.
142
143 --------
144 Buffered
145 --------
146
cc12387 @schmichael Update README to reflect new type size field
authored
147 +----------------+------------+---------------+------------+------------------+-------------+
148 | ``label size`` | ``label`` | ``type size`` | ``type`` | ``write buffer`` | ``buffers`` |
149 +================+============+===============+============+==================+=============+
150 | ``ushort`` | ``char[]`` | ``ushort`` | ``char[]`` | ``byte`` | ``varies`` |
151 +----------------+------------+---------------+------------+------------------+-------------+
a6dfa3e @schmichael Begin documenting data structures
authored
152
153 The buffers field length = sizeof(type) * buffers.
154
155 The current write buffer is referenced by: write_buffer * sizeof(type)
156
157 TODO: field for total number of buffers?
158
159 ----------
160 Unbuffered
161 ----------
162
cc12387 @schmichael Update README to reflect new type size field
authored
163
164 +----------------+------------+---------------+------------+------------------+-------------+
165 | ``label size`` | ``label`` | ``type size`` | ``type`` | ``write buffer`` | ``value`` |
166 +================+============+===============+============+==================+=============+
167 | ``ushort`` | ``char[]`` | ``ushort`` | ``char[]`` | ``ff`` | ``varies`` |
168 +----------------+------------+---------------+------------+------------------+-------------+
a6dfa3e @schmichael Begin documenting data structures
authored
169
170 The value field length = sizeof(type).
Something went wrong with that request. Please try again.