Skip to content

Commit

Permalink
adding initial sinks document, plenty of tweaks all around, a few glo…
Browse files Browse the repository at this point in the history
…ssary entries
  • Loading branch information
mahmoud committed Jun 7, 2016
1 parent 13a526e commit 92a9edf
Show file tree
Hide file tree
Showing 7 changed files with 135 additions and 37 deletions.
40 changes: 26 additions & 14 deletions docs/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,11 @@ Glossary
:sorted:

logger
An instance of the :class:`~lithoxyl.logger.Logger` type, and one
of the three fundamental Lithoxyl types. Responsible for
facilitating the creation and publication of :term:`records
<record>`. Generally there is one logger per aspect of an
application. For example, a request logger and a database query
logger.
An instance of the :class:`~lithoxyl.logger.Logger`
type. Responsible for facilitating the creation and publication
of :term:`records <record>`. Generally there is one logger per
aspect of an application. For example, a request logger and a
database query logger.

record
An instance of the :class:`~lithoxyl.logger.Record` type, and one
Expand All @@ -32,17 +31,30 @@ Glossary
* **Exception** - unanticipated or lower-level unsuccessful
completion (e.g., database connection interrupted)

event

An occurence associated with a Logger and Record. One of:

* **begin** - The start of a Record.
* **end** - The completion of a Record (success, failure, or exception)
* **warn** - A warning related to a Record.
* **comment** - A metadata event associated with a Logger
* **exception** - An unhandled exception during a Record.

:term:`Sinks <sink>` implement methods to handle each of these events.

sink
Any object implementing the Sink protocol, and one of the
three fundamental Lithoxyl types. Typically subscribed to
:term:`records <record>` by being attached to a
:term:`logger`. Some basic types of sinks include record
emitters, statistics collectors, and profilers.
Any object implementing the Sink protocol for handling
:term:`events <event>`. Typically subscribed to :term:`records
<record>` by being attached to a :term:`logger`. Some basic types
of sinks include record emitters, statistics collectors, and
profilers.

emitter
A sink or other object capable of publishing record entries out
of the process. Emitters commonly publish to network services,
local services, and files.
An object capable of publishing formatted messages out of the
process. Emitters commonly publish to network services, local
services, and files. The last step in the Sensible
Filter-Format-Emit logging process.

formatter
An object responsible for transforming a :term:`record` into a
Expand Down
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Section Listing
overview
logger
record
sinks
sensible
logging_tradition
glossary
Expand Down
2 changes: 1 addition & 1 deletion docs/logger.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
The ``Logger``
The Logger
==============

.. automodule:: lithoxyl.logger
Expand Down
40 changes: 23 additions & 17 deletions docs/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,21 @@ Lithoxyl Overview
The Lithoxyl approach to application instrumentation is a practical
one. First, write your code. Then, once you have half a module or find
yourself asking, "How long does this part take?" then it's time to
``pip install lithoxyl``. There are two steps. First comes
``pip install lithoxyl``. After that, here are two steps. First comes
instrumentation, then we get into configuration.

Instrumenting with Records
--------------------------

With Lithoxyl, logging and other instrumentation starts with knowing
your application. We want to find the important parts of your
application, and wrap them in microtransactions called
``Records``. Records can be created directly, but they are most often
created through ``Loggers``.
With Lithoxyl, all instrumentation, including logging, starts with
knowing your application. We want to find the important parts of your
application. Then, we wrap them in microtransactions, called Records,
that we create with Loggers.

Let's take a look::
Here's a basic example of creating an *info*-level Record with a
preconfigured Logger:

.. code-block:: python
import backend # some convenient backend logic for brevity
from log import app_log # preconfigured Lithoxyl Logger
Expand All @@ -27,29 +29,33 @@ Let's take a look::
backend.add_by_name(name)
return True
As you can see, the transactionality of records translates well to
Python's :term:`with` context manager syntax. One benefit of this
approach is that a single line of logging code is able to record both
the start and end events. Even better is that there is no chance of
missing an exception if one is raised unexpectedly. For instance, if
*name* is not a string, and ``.strip()`` raises an ``AttributeError``,
then that exception is guaranteed to be recorded.
As you can see, the transactionality of Records translates well to
Python's :term:`with` context manager syntax. A single line of logging
code succinctly records the beginning and ending of this code
block. Even better, there's no chance of missing an unexpected
exception. For instance, if **name** is not a string, and **.strip()**
raises an :exc:`AttributeError`, then that exception is guaranteed to be
recorded.

Records also support direct interaction. Arbitrary data can be added
to the record, with dictonary syntax. And while records finish with a
You can do much so more with records. Arbitrary data can be added to
the record, with dictonary syntax. And while records finish with a
success status and autogenerated message if no exception is raised,
failures and exceptions can also be set manually::
failures and exceptions can also be set manually:

.. code-block:: python
import backend
from log import app_log
def set_entry_state(name, state):
with app_log.info('setting entry state') as rec:
rec['name'] = name
status = backend.STATE_MAP[state.lower()]
success = backend.set_entry_state(name, state)
if not success:
rec.failure('set {name} status to {state} failed', state=state)
return success
As seen above, records can also have a custom completion message,
Expand Down
2 changes: 1 addition & 1 deletion docs/record.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
The ``Record``
The Record
==============

.. automodule:: lithoxyl.record
Expand Down
10 changes: 6 additions & 4 deletions docs/sensible.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
The Sensible Suite
==================

Lithoxyl has many uses, but one of them is as a toolkit for creating
structured logs, or logs with a uniform format to make them machine
readable. The Sensible approach offers structured logging without
sacrificing human readability.
Structured logging creates logs with a consistent format, allowing
them to be loaded later for further processing and analysis.

One of Lithoxyl's primary uses is as a toolkit for creating these
structured logs. The Sensible Suite is the first generalized approach
to offer structured logging without sacrificing human readability.

Let's look at an example. Perhaps the most common structured log is
the HTTP server access log, such as the one created by Apache or
Expand Down
77 changes: 77 additions & 0 deletions docs/sinks.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
The Sink
========

In Lithoxyl's system of instrumentation, Records are used to carry
messages, data, and timing metadata through the Loggers to their
destination, the Sinks. This chapter focuses in on this last
step.

Writing a simple Sink
---------------------

Sinks range from very complex to very simple. A useful Sink can be as
simple as::

import sys

class DotSink(object):
def on_end(self, end_event):
sys.stdout.write('.')
sys.stdout.flush()


Note that our new Sink does not have to inherit from any special
object. DotSink is a correct and capable Sink, ready to be
instantiated and installed with :meth:`Logger.add_sink`, just like in
the overview. Once added to your Logger, every time a Record ends, a
dot will be written out to your console.

In this example, ``on_end`` is the handler for just one of Lithoxyl's
events. The next section takes a look at all five of them.

Events
------

Five events can happen in the Lithoxyl system:

* **begin** - The beginning of a Record, whether manually or through
entering a context-managed block of code.

The begin event corresponds to the method signature ``on_begin(self,
begin_event)``. Designed to be called once per Record.
* **end** - The completion of a Record, whether manually
(``success()`` and ``failure()``) or through exiting a
context-managed block of code. There are three ways a Record can
end, **success**, **failure**, and **exception**, but all of them
result in an *end* event.

The end event corresponds to the method signature ``on_end(self,
end_event)``. Designed to be called once per Record.
* **exception** - Called immediately when an exception is raised from
within the context-managed block, or when an exception is manually
handled with Record.exception(). Records ending in exception state
typically fire two events, one for handling the exception, and one
for ending the Record.

The exception event corresponds to the Sink method signature
``on_exception(self, exc_event, exc_type, exc_obj, exc_tb)``.
Designed to be called up to once.
* **warn** - The registration of a warning within a Record.

Corresponds to the Sink method signature ``on_warn(self,
warn_event)``. Can be called an arbitrary number of times.

* **comment** - The registration of a comment from a Logger. Comments
are used for publishing metadata associated with a Logger.

The comment event corresponds to the Sink method signature
``on_comment(self, comment_event)``. See here for more about
comments. Can be called an arbitrary number of times.

A Sink handles the event by implementing the respective method. The
event objects that accompany every event are meant to be practically
immutable; their values are set once, at creation.


.. Lithoxyl's informal Sink taxonomy ideas: numeric, accumulating,
debug, stream.

0 comments on commit 92a9edf

Please sign in to comment.