Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor tweaks to formatting, grammar, and phrasing #2

Merged
merged 1 commit into from Jul 7, 2015
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
100 changes: 51 additions & 49 deletions content/learning-python-logging.md
Expand Up @@ -2,8 +2,8 @@ Title: Learning Python Logging
Date: 2012-02-17 17:46:00
Tags: python, logging

The Python logging module is often a source of confusion with
developers. Often logging is the final thought in a project. When we
The Python `logging` module is often a source of confusion with
developers. Often, logging is the final thought in a project. When we
are finishing up a project, the last thing we want to do is sift
through the [logging
documentation](http://docs.python.org/library/logging.html) to figure
Expand Down Expand Up @@ -31,11 +31,11 @@ module's fully qualified name. So if we have a module called
`mypackage.module1.module2`, that will be the logger's name.

Keep in mind that `__name__` is the string `"__main__"` if
your Python file is ran as a script. Other than that, that is all you
your Python file is run as a script. Other than that, this is all you
need to do.

You do not have to worry about logging levels, or handlers or anything
in your modules. How your loggers are configured is an application
You do not have to worry about logging levels, handlers, or anything
else in your modules. How your loggers are configured is an application
level setting. Your modules have no reason to worry about how to
configure loggers.

Expand Down Expand Up @@ -69,15 +69,15 @@ All `myapp.*` loggers will have the level `INFO` as their default.

Let us say that there are the following loggers used in our application:

* redis.connection
* mysql.connection
* mysql.query
* myapp.models.polls
* myapp.models.questions
* myapp.models.articles
* `redis.connection`
* `mysql.connection`
* `mysql.query`
* `myapp.models.polls`
* `myapp.models.questions`
* `myapp.models.articles`

If we only wanted connection errors from our databases, query debug
information from MySQL, and warnings from myapp's models; we could
information from MySQL, and warnings from `myapp`'s models; we could
configure it as so:

:::python
Expand All @@ -87,22 +87,24 @@ configure it as so:
logging.getLogger("myapp.models").setLevel(logging.WARNING)

That is it. This accomplishes what I described. We set the default
level to ERROR which causes "redis.connection" and "mysql.connection"
to only emit ERROR and EXCEPTION level messages. We set the
"mysql.query" logger to DEBUG to emit query debug messages. Finally we
set the "myapp.models" logger to WARNING.
level to `ERROR` which causes `redis.connection` and `mysql.connection`
to only emit `ERROR` and `EXCEPTION` level messages. We set the
`mysql.query` logger to `DEBUG` to emit query debug messages. Finally, we
set the `myapp.models` logger to `WARNING`.

# Configuring Python loggers

I just showed you how to configure the level of a logger
pragmatically. Where should that code exist? If you answered, "In the
module", I want you to get up, find the nearest blunt object, and hit
yourself with it. Do you not remember me saying that logging
configuration is an application level configuration?
pragmatically. Where should that code exist?

If you answered, "In the module," I want you to get up,
find the nearest blunt object, and hit yourself with it.
Do you not remember me saying that logging configuration
is an application level configuration?

Here is a hypothetical script that uses a Redis connection.

[import_presidents1.py](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents1.py):
[`import_presidents1.py`](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents1.py):

:::python

Expand All @@ -121,8 +123,8 @@ Here is a hypothetical script that uses a Redis connection.
client.set(key, doc)


Now, let us assume that the redis module used a "redis.connection"
logger that logs a DEBUG level message whenever the redis client needs
Now, let us assume that the redis module used a `redis.connection`
logger that logs a `DEBUG` level message whenever the redis client needs
to reconnect to the server. Here is a [mock Redis client](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/redis.py)
for this example.

Expand All @@ -131,7 +133,7 @@ When we run the code as it is, no connection messages will be printed.
Now, let us say our connection is flaky and we have to reconnect to
Redis often:

[import_presidents2.py](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents2.py):
[`import_presidents2.py`](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents2.py):

:::python

Expand All @@ -153,10 +155,10 @@ Redis often:

client.set(key, doc)

We can configure the "redis.connection" logger to output the
We can configure the `redis.connection` logger to output the
connection messages by configuring our script like so:

[import_presidents3.py](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents3.py):
[`import_presidents3.py`](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents3.py):

:::python

Expand Down Expand Up @@ -197,36 +199,36 @@ with:
logging.basicConfig(level=logging.DEBUG)


[logging.basicConfig()](http://docs.python.org/library/logging.html#logging.basicConfig)
[`logging.basicConfig()`](http://docs.python.org/library/logging.html#logging.basicConfig)
is a really handy function. I find myself using `basicConfig()` often
because most of the time I simply want to output to `stderr` or to a
single file rather than wanting complex routing using Handlers.

# What are Handlers and Formatters?

Apart from Loggers, there are three classes that are used to configure
an application's loggers: logging.Handler, logging.Formatter, and
logging.Filter.
an application's loggers: `logging.Handler`, `logging.Formatter`, and
`logging.Filter`.

A Handler defines how a message is handled. For instance, there is a
[StreamHandler](http://docs.python.org/library/logging.handlers.html#streamhandler)
that logs messages to stderr.
that logs messages to `stderr`.

There are 13 standard Handlers defined in the
[logging.handlers](http://docs.python.org/library/logging.handlers.html)
[`logging.handlers`](http://docs.python.org/library/logging.handlers.html)
module that will send messages to files, syslog, HTTP servers,
sockets, what have you.
sockets, and what have you.

A Formatter is exactly what it sounds like. It formats a
LogRecord. That's pretty much it.

A Filter filters a LogRecord. I have never needed one, so I will let
you define your own use case for one.
you define your own use case for Filters.

Let us change our presidential import script to log any exceptions
that occur when importing:

[import_presidents4.py](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents4.py):
[`import_presidents4.py`](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents4.py):

:::python

Expand Down Expand Up @@ -261,12 +263,12 @@ that occur when importing:
log.exception("dang it.")


Notice that we added "import config", this will let us separate
configuration from implementation. For our little one-off script,
having a config.py module is overkill and only exists to prove a
point.
Notice that we added `import config`. This will let us separate
configuration from implementation. (For our little one-off script,
having a `config.py` module is overkill; it exists here to prove a
point.)

Here's the contents of [config.py](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/config.py):
Here's the contents of [`config.py`](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/config.py):

:::python

Expand All @@ -291,13 +293,13 @@ Here's the contents of [config.py](https://github.com/ericmoritz/blog/blob/maste

An alternative way to configure your application's logger is to use
the
[logging.config.fileConfig](http://docs.python.org/library/logging.config.html#logging.config.fileConfig)
[`logging.config.fileConfig`](http://docs.python.org/library/logging.config.html#logging.config.fileConfig)
to configure the loggers using an INI file.

Here is the equivalent INI file for the configuration we previously
described using Python code:

[logging.ini](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/logging.ini):
[`logging.ini`](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/logging.ini):

:::ini

Expand Down Expand Up @@ -331,9 +333,9 @@ described using Python code:
handlers=console,error_file


To use this file, let us modify our config.py.
To use this file, let us modify our `config.py`.

[config_ini.py](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/config_ini.py):
[`config_ini.py`](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/config_ini.py):

:::python

Expand All @@ -346,7 +348,7 @@ To use this file, let us modify our config.py.

Finally, here's the updated import script:

[import_presidents5.py](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents5.py):
[`import_presidents5.py`](https://github.com/ericmoritz/blog/blob/master/example-code/learn-python-logging/import_presidents5.py):

import config_ini
import logging
Expand Down Expand Up @@ -379,14 +381,14 @@ Finally, here's the updated import script:
log.exception("Dang it.")


For some people INI files leave a bad taste is their mouth. If you do
For some people INI files leave a bad taste is the mouth. If you do
not like the taste of INI files, you can also configure the loggers
using a dictionary or with Python code. It is up to you.
using a dictionary or with Python code. It's up to you.

There we have it. Python 2's built-in logging module is one of those
warts of the standard library that goes against the "Zen of Python"
There we have it. Python 2's built-in `logging` module is one of those
warts of the standard library that goes against the "Zen of Python";
but once you learn the non-obvious way the designers intended to be
used it becomes simple.
used, it becomes simple.

I hope this helps clear up Python's logging module for you.

Expand Down