This repository is no longer activiely maintained.
If you are looking for similar functionality but for more uptodate versions of Python, we recommend you take a look at https://github.com/vklochan/python-logstash as an alternative.
This library is provided to allow standard python logging to send log data as json objects to logstash.
Pip:
pip install git+https://github.com/meltwater/threaded-logstasher.git
import logging
from logstasher import LogstashHandler, LogstashFormatterExtra
logger = logging.getLogger('')
logger.setLevel(logging.INFO)
handler = LogstashHandler('192.168.1.100', 4560,
threads=3, queue_size=100)
formatter = LogstashFormatterExtra(source_host='example.com', always='value')
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.info('See what you get', extra=dict(param1='value1'))
The library provides a handler and a formatter class:
LogstashHandler
and LogstashFormatterExtra
. The handler implements a
threadpool to separate network communication from the application logic, thus
the calling thread is not blocked by any socket call.
The messages are formatted and put immediately to an internal
queue.
The worker threads deliver messages to the logstash daemon using a slightly
modified version of logging.handlers.SocketHandler
.
LogstashHandler
takes the following parameters:
host
: hostname of the logstash receiver daemon.port
: port of the logstash receiver daemon.level
: the threshold level for this handler (default=INFO)threads
: the number of worker threads connecting to logstash daemon.queue_size
: size of the queue used for buffering log messages.timeout
: floating point number specifying a timeout for sending remaining messages to logstash receiver when the handler is closed. Timeout is specified in seconds.
LogstashFormatterExtra
is a wrapper class above LogstashFormatterV1
from
logstash_formatter library.
It is added to this library for convenience and may take the following named
parameters other than what is inherited from its super class:
source_host
: custom hostname of your application.- any additional keyword arguments will be added as an extra field to every JSON log.
The result of the logger.info
call in the example above:
{
"filename" => "test_logstasher.py",
"processName" => "MainProcess",
"funcName" => "<module>",
"pathname" => "test_logstasher.py",
"args" => [],
"@timestamp" => "2016-01-12T13:17:14.177Z",
"levelno" => 20,
"param1" => "value1",
"process" => 15852,
"source_host" => "example.com",
"stack_info" => nil,
"thread" => 140735093338112,
"threadName" => "MainThread",
"always" => "value",
"msecs" => 177.3838996887207,
"message" => "See what you get",
"levelname" => "INFO",
"name" => "root",
"created" => 1452604634.177384,
"lineno" => 13,
"@version" => 1,
"module" => "test_logstasher",
"relativeCreated" => 10.72382926940918,
"host" => "192.168.99.1"
}