Permalink
Browse files

Merge branch 'schmichael'

  • Loading branch information...
dcolish committed Sep 29, 2011
2 parents cd8154c + 3319d24 commit 034a7f5c71531b2625d940d8a6aaf1a18c9b8701
Showing with 350 additions and 122 deletions.
  1. +7 −0 AUTHORS
  2. +15 −14 README.rst
  3. +50 −0 TODO.rst
  4. +6 −6 examples/basic.py
  5. +3 −3 examples/basic_flask.py
  6. +147 −66 mmstats.py
  7. +4 −0 run_flask_example
  8. +1 −1 setup.py
  9. +48 −0 tests/test_mmap.py
  10. +34 −9 tests/test_mmstats.py
  11. +35 −23 tests/test_types.py
View
@@ -0,0 +1,7 @@
Developers:
Dan Colish <dcolish@gmail.com>
Michael Schurter <m@schmichael.com> (creator)
Additional help:
Niall Kelly (reviewer)
Adam Lowry (reviewer)
View
@@ -47,10 +47,10 @@ Using
::
class WebStats(mmstats.MmStats):
status2xx = mmstats.UIntStat(label='status.2XX')
status3xx = mmstats.UIntStat(label='status.3XX')
status4xx = mmstats.UIntStat(label='status.4XX')
status5xx = mmstats.UIntStat(label='status.5XX')
status2xx = mmstats.UIntField(label='status.2XX')
status3xx = mmstats.UIntField(label='status.3XX')
status4xx = mmstats.UIntField(label='status.4XX')
status5xx = mmstats.UIntField(label='status.5XX')
4. Instantiate it once per thread/process:
@@ -94,11 +94,11 @@ Unbuffered structures have ff in the write buffer field.
Buffered
--------
+----------------+----------------------+----------+------------------+-------------+
| ``label size`` | ``label`` | ``type`` | ``write buffer`` | ``buffers`` |
+================+======================+==========+==================+=============+
| ``ushort`` | ``label size chars`` | ``char`` | ``byte`` | ``varies`` |
+----------------+----------------------+----------+------------------+-------------+
+----------------+------------+---------------+------------+------------------+-------------+
| ``label size`` | ``label`` | ``type size`` | ``type`` | ``write buffer`` | ``buffers`` |
+================+============+===============+============+==================+=============+
| ``ushort`` | ``char[]`` | ``ushort`` | ``char[]`` | ``byte`` | ``varies`` |
+----------------+------------+---------------+------------+------------------+-------------+
The buffers field length = sizeof(type) * buffers.
@@ -110,10 +110,11 @@ TODO: field for total number of buffers?
Unbuffered
----------
+----------------+----------------------+----------+------------------+-------------+
| ``label size`` | ``label`` | ``type`` | ``write buffer`` | ``value`` |
+================+======================+==========+==================+=============+
| ``ushort`` | ``label size chars`` | ``char`` | ``ff`` | ``varies`` |
+----------------+----------------------+----------+------------------+-------------+
+----------------+------------+---------------+------------+------------------+-------------+
| ``label size`` | ``label`` | ``type size`` | ``type`` | ``write buffer`` | ``value`` |
+================+============+===============+============+==================+=============+
| ``ushort`` | ``char[]`` | ``ushort`` | ``char[]`` | ``ff`` | ``varies`` |
+----------------+------------+---------------+------------+------------------+-------------+
The value field length = sizeof(type).
View
@@ -3,13 +3,63 @@ TODO
====
* Add every simple type (strings, ints, floats, bools, etc)
* Add timer field/contextmanager
* Add memory usage field/contextmanager
* Vary filename based on class name
* Add API to dynamically add fields to MmStat classes
* Add alternative procedural writer API (vs existing declarative models)
* Test severity of race conditions
* Test performance
==============
Scrapped Ideas
==============
---------------------------------------------------------------
Compounds Fields where 1 Writer Field = Many Mmap/Reader Fields
---------------------------------------------------------------
This seemed like a honking great idea at first. Compound fields would look just
like a mini-MmStat model:
::
class SamplingCounterField(CompoundField):
"""Records increments per ms every N increments"""
counter = CounterField()
per_ms = UInt64Field()
class _Counter(object):
"""Implement counter/rate-sampling logic here"""
def __get__(self, inst, owner):
if inst is None:
return self
return inst._fields[self.key]._counter_instance
The blocker is that there's no way to atomically update all of the compound
fields. The only way to accomplish this is for compound fields to appear as a
single double buffered field with each component field as a type in the type
signature:
::
class SamplingCounterField(DoubleBufferedField):
initial = (
CounterField.initial,
UInt64Field.initial,
)
buffer_type = (
CounterField.buffer_type,
UInt64Field.buffer_type,
)
type_signature = (
CounterField.type_signature + UInt64Field.type_signature
)
Obviously an actual implementation should remove the redundant references to
the component types.
------------------------
Metadata metaprogramming
------------------------
View
@@ -8,12 +8,12 @@ class MyStats(mmstats.BaseMmStats):
tid = mmstats.StaticInt64Field(label="sys.tid", value=libgettid.gettid)
uid = mmstats.StaticUInt64Field(label="sys.uid", value=os.getuid)
gid = mmstats.StaticUInt64Field(label="sys.gid", value=os.getgid)
errors = mmstats.UIntStat(label="com.urbanairship.app.errors")
warnings = mmstats.UIntStat(label="com.urbanairship.app.warnings")
queries = mmstats.UIntStat(label="com.urbanairship.app.queries")
cache_hits = mmstats.UIntStat(label="com.urbanairship.app.cache_hits")
cache_misses = mmstats.UIntStat(label="com.urbanairship.app.cache_misses")
degraded = mmstats.BoolStat(label="com.urbanairship.app.degraded")
errors = mmstats.UIntField(label="com.urbanairship.app.errors")
warnings = mmstats.UIntField(label="com.urbanairship.app.warnings")
queries = mmstats.UIntField(label="com.urbanairship.app.queries")
cache_hits = mmstats.UIntField(label="com.urbanairship.app.cache_hits")
cache_misses = mmstats.UIntField(label="com.urbanairship.app.cache_misses")
degraded = mmstats.BoolField(label="com.urbanairship.app.degraded")
foo = mmstats.StaticTextField(
label="com.idealist.app.name", value="webapp")
View
@@ -8,9 +8,9 @@
class Stats(mmstats.MmStats):
ok = mmstats.UIntStat(label="mmstats.example.ok")
bad = mmstats.UIntStat(label="mmstats.example.bad")
working = mmstats.BoolStat(label="mmstats.example.working")
ok = mmstats.UIntField(label="mmstats.example.ok")
bad = mmstats.UIntField(label="mmstats.example.bad")
working = mmstats.BoolField(label="mmstats.example.working")
stats = Stats()
Oops, something went wrong.

0 comments on commit 034a7f5

Please sign in to comment.