Skip to content

added support for quart framework #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 17, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 36 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,19 @@ If you're using Cloud Foundry, it worth to check out the library [SAP/cf-python-
7. [References](#7-references)

# 1. Features
1. Emit JSON logs ([format detail](#0-full-logging-format-references))
1. Emit JSON logs ([format detail](#0-full-logging-format-references))
2. Support **correlation-id** [\[1\]](#1-what-is-correlation-idrequest-id)
3. Lightweight, no dependencies, minimal configuration needed (1 LoC to get it working)
4. Fully compatible with Python **logging** module. Support both Python 2.7.x and 3.x
5. Support HTTP request instrumentation. Built in support for [Flask](http://flask.pocoo.org/) & [Sanic](https://github.com/channelcat/sanic). Extensible to support other web frameworks. PR welcome :smiley: .
5. Support HTTP request instrumentation. Built in support for [Flask](http://flask.pocoo.org/) & [Sanic](https://github.com/channelcat/sanic) & [Quart](https://gitlab.com/pgjones/quart). Extensible to support other web frameworks. PR welcome :smiley: .
6. Support inject arbitrary extra properties to JSON log message.

# 2. Usage
Install by running this command:
> pip install json-logging

By default log will be emitted in normal format to ease the local development. To enable it on production set either **json_logging.ENABLE_JSON_LOGGING** or **ENABLE_JSON_LOGGING environment variable** to true.

To configure, call **json_logging.init(framework_name)**. Once configured library will try to configure all loggers (existing and newly created) to emit log in JSON format.
See following use cases for more detail.

Expand Down Expand Up @@ -99,6 +99,32 @@ if __name__ == "__main__":
app.run(host="0.0.0.0", port=8000)
```

### Quart

```python
import asyncio, logging, sys, json_logging, quart

app = quart.Quart(__name__)
json_logging.ENABLE_JSON_LOGGING = True
json_logging.init(framework_name='quart')
json_logging.init_request_instrument(app)

# init the logger as usual
logger = logging.getLogger("test logger")
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler(sys.stdout))

@app.route('/')
async def home():
logger.info("test log statement")
logger.info("test log statement", extra={'props': {"extra_property": 'extra_value'}})
return "Hello world"

if __name__ == "__main__":
loop = asyncio.get_event_loop()
app.run(host='0.0.0.0', port=int(5000), use_reloader=False, loop=loop)
```

## 2.3 Get current correlation-id
Current request correlation-id can be retrieved and pass to downstream services call as follow:

Expand Down Expand Up @@ -223,16 +249,16 @@ e.g.:
}
```
See following tables for detail format explanation:
- Common field
- Common field

Field | Description | Format | Example
--- | --- | --- | ---
written_at | The date when this log message was written. | ISO 8601 YYYY-MM-DDTHH:MM:SS.milliZ | 2017-12-23T15:14:02.208Z
written_at | The date when this log message was written. | ISO 8601 YYYY-MM-DDTHH:MM:SS.milliZ | 2017-12-23T15:14:02.208Z
written_ts | The timestamp in nano-second precision when this request metric message was written. | long number | 1456820553816849408
correlation_id | The timestamp in nano-second precision when this request metric message was written. | string | db2d002e-2702-41ec-66f5-c002a80a3d3f
type | Type of logging. "logs" or "request" | string |
type | Type of logging. "logs" or "request" | string |
component_id | Uniquely identifies the software component that has processed the current request | string | 9e6f3ecf-def0-4baf-8fac-9339e61d5645
component_name | A human-friendly name representing the software component | string | my-fancy-component
component_name | A human-friendly name representing the software component | string | my-fancy-component
component_instance | Instance's index of horizontally scaled service | string | 0

- application logs
Expand All @@ -245,7 +271,7 @@ thread | Identifies the execution thread in which this log message has been writ
logger | The logger name that emits the log message.
| string | requests-logger

- request logs:
- request logs:

Field | Description | Format | Example
--- | --- | --- | ---
Expand All @@ -267,7 +293,7 @@ referer | For HTTP requests, identifies the address of the webpage (i.e. the URI
x_forwarded_for | Comma-separated list of IP addresses, the left-most being the original client, followed by proxy server addresses that forwarded the client request. | string | 192.0.2.60,10.12.9.23

## [1] What is correlation-id/request id
https://stackoverflow.com/questions/25433258/what-is-the-x-request-id-http-header
https://stackoverflow.com/questions/25433258/what-is-the-x-request-id-http-header
## [2] Python logging propagate
https://docs.python.org/3/library/logging.html#logging.Logger.propagate
https://docs.python.org/2/library/logging.html#logging.Logger.propagate
Expand Down Expand Up @@ -303,7 +329,7 @@ python -m pip install --index-url https://test.pypi.org/simple/ json_logging

python setup.py sdist upload -r pypitest
python setup.py bdist_wheel --universal upload -r pypitest
pip3 install json_logging --index-url https://test.pypi.org/simple/
pip3 install json_logging --index-url https://test.pypi.org/simple/
```
pypi
```
Expand Down
40 changes: 34 additions & 6 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ searchable by logging infrastructure such as

| If you’re using Cloud Foundry, it worth to check out the library
`SAP/cf-python-logging-support <https://github.com/SAP/cf-python-logging-support>`_
which I’m also original author and contributor.
which I’m also original author and contributor.

1. Features
===========
Expand All @@ -22,14 +22,15 @@ searchable by logging infrastructure such as
2.7.x and 3.x
5. Support HTTP request instrumentation. Built in support for
`Flask <http://flask.pocoo.org/>`_ &
`Sanic <https://github.com/channelcat/sanic>`_. Extensible to support other web
`Sanic <https://github.com/channelcat/sanic>`_ &
`Quart <https://gitlab.com/pgjones/quart>`_. Extensible to support other web
frameworks. PR welcome :smiley: .
6. Support inject arbitrary extra properties to JSON log message.

2. Usage
========

Install by running this command:
Install by running this command:

.. code:: python

Expand Down Expand Up @@ -123,6 +124,33 @@ Sanic
if __name__ == "__main__":
app.run(host="0.0.0.0", port=8000)

Quart
~~~~~

.. code:: python

import asyncio, logging, sys, json_logging, quart

app = quart.Quart(__name__)
json_logging.ENABLE_JSON_LOGGING = True
json_logging.init(framework_name='quart')
json_logging.init_request_instrument(app)

# init the logger as usual
logger = logging.getLogger("test logger")
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler(sys.stdout))

@app.route('/')
async def home():
logger.info("test log statement")
logger.info("test log statement", extra={'props': {"extra_property": 'extra_value'}})
return "Hello world"

if __name__ == "__main__":
loop = asyncio.get_event_loop()
app.run(host='0.0.0.0', port=int(5000), use_reloader=False, loop=loop)


2.3 Get current correlation-id
------------------------------
Expand Down Expand Up @@ -317,7 +345,7 @@ you can install Sanic on windows by running these commands:
[0] Full logging format references
----------------------------------

2 types of logging statement will be emitted by this library:
2 types of logging statement will be emitted by this library:
- Application log: normal logging statement e.g.:

::
Expand Down Expand Up @@ -368,7 +396,7 @@ you can install Sanic on windows by running these commands:
"response_sent_at": "2017-12-23T16:55:37.280Z"
}

See following tables for detail format explanation:
See following tables for detail format explanation:

- Common field

Expand Down Expand Up @@ -589,4 +617,4 @@ https://docs.python.org/2/library/logging.html#logging.Logger.propagate
[3] more on flask use_reloader
------------------------------

http://flask.pocoo.org/docs/0.12/errorhandling/#working-with-debuggers
http://flask.pocoo.org/docs/0.12/errorhandling/#working-with-debuggers
29 changes: 29 additions & 0 deletions example/quart_sample_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import asyncio
import logging
import sys

import quart

import json_logging

app = quart.Quart(__name__)
json_logging.ENABLE_JSON_LOGGING = True
json_logging.init(framework_name='quart')
json_logging.init_request_instrument(app)

# init the logger as usual
logger = logging.getLogger("test logger")
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler(sys.stdout))


@app.route('/')
async def home():
logger.info("test log statement")
logger.info("test log statement", extra={'props': {"extra_property": 'extra_value'}})
return "Hello world"


if __name__ == "__main__":
loop = asyncio.get_event_loop()
app.run(host='0.0.0.0', port=int(5000), use_reloader=False, loop=loop)
8 changes: 8 additions & 0 deletions json_logging/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -292,3 +292,11 @@ def format(self, record):
SanicAppRequestInstrumentationConfigurator,
SanicRequestAdapter,
SanicResponseAdapter)

# register quart support
# noinspection PyPep8
import json_logging.framework.quart as quart_support

register_framework_support('quart', None, quart_support.QuartAppRequestInstrumentationConfigurator,
quart_support.QuartRequestAdapter,
quart_support.QuartResponseAdapter)
127 changes: 127 additions & 0 deletions json_logging/framework/quart/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# coding=utf-8
import logging
import sys

import json_logging
import json_logging.framework
from json_logging import JSONLogWebFormatter
from json_logging.framework_base import AppRequestInstrumentationConfigurator, RequestAdapter, ResponseAdapter


def is_quart_present():
# noinspection PyPep8,PyBroadException
try:
import quart
return True
except:
return False


if is_quart_present():
from quart import request as request_obj
import quart as quart

_current_request = request_obj
_quart = quart


class QuartAppRequestInstrumentationConfigurator(AppRequestInstrumentationConfigurator):
def config(self, app):
if not is_quart_present():
raise RuntimeError("quart is not available in system runtime")
from quart.app import Quart
if not isinstance(app, Quart):
raise RuntimeError("app is not a valid quart.app.Quart app instance")

# Remove quart logging handlers
from quart.logging import default_handler, serving_handler
logging.getLogger('quart.app').removeHandler(default_handler)
logging.getLogger('quart.serving').removeHandler(serving_handler)


json_logging.util.use_cf_logging_formatter([
# logging.getLogger('quart.app'),
# logging.getLogger('quart.serving'),
], JSONLogWebFormatter)

# noinspection PyAttributeOutsideInit
self.request_logger = logging.getLogger('quart.app')
self.request_logger.setLevel(logging.DEBUG)
self.request_logger.addHandler(logging.StreamHandler(sys.stdout))

from quart import g

@app.before_request
def before_request():
g.request_info = json_logging.RequestInfo(_current_request)

@app.after_request
def after_request(response):
request_info = g.request_info
request_info.update_response_status(response)
# TODO:handle to print out request instrumentation in non-JSON mode
self.request_logger.info("", extra={'request_info': request_info})
return response


class QuartRequestAdapter(RequestAdapter):
@staticmethod
def get_request_class_type():
raise NotImplementedError

@staticmethod
def support_global_request_object():
return True

@staticmethod
def get_current_request():
return _current_request

def get_remote_user(self, request):
if request.authorization is not None:
return request.authorization.username
else:
return json_logging.EMPTY_VALUE

def is_in_request_context(self, request_):
return _quart.has_request_context()

def get_http_header(self, request, header_name, default=None):
if header_name in request.headers:
return request.headers.get(header_name)
return default

def set_correlation_id(self, request_, value):
_quart.g.correlation_id = value

def get_correlation_id_in_request_context(self, request):
return _quart.g.get('correlation_id', None)

def get_protocol(self, request):
return request.scheme

def get_path(self, request):
return request.path

def get_content_length(self, request):
return request.content_length

def get_method(self, request):
return request.method

def get_remote_ip(self, request):
return request.remote_addr

def get_remote_port(self, request):
return request.host.split(":", 2)[1]


class QuartResponseAdapter(ResponseAdapter):
def get_status_code(self, response):
return response.status_code

def get_response_size(self, response):
return response.content_length

def get_content_type(self, response):
return response.content_type