RETURN_HEADER setting added, better docs, version 1.0.0 (Closes #16) (#…

…21)

* - Added a setting called ``RETURN_HEADER`` which defaults to true. It will return the GUID used in the application as a header in the outgoing package.
- Improved tests to also check for headers in the response
- Added tests for the new setting
- Added some more information about the package in the README and in the docs
- Added an API page to the docs, showing how to pass on the current header in a custom request to another application
- Fixed read the docs menu
- Bumped version to 1.0.0

* Added unittest for invalid RETURN_HEADER

* Apply suggestions from code review

Co-Authored-By: sondrelg <sondrelg@live.no>

* Update CHANGELOG.rst based on code review

Co-Authored-By: sondrelg <sondrelg@live.no>

* - Added an extended example to the readthedocs
- Referenced the example in the README.rst
- Added wsgi so users of this package can try it out them self

Co-authored-by: sondrelg <sondrelg@live.no>

.pre-commit-config.yaml

@@ -26,7 +26,7 @@ repos:
           'flake8-docstrings==1.5.0',  # Verifies that all functions/methods have docstrings
           'flake8-type-annotations==0.1.0',  # Looks for misconfigured type annotations
           'flake8-annotations==1.1.1',  # Enforces type annotation
-          'flake8-logging-format==0.6.0',
+          'flake8-logging-format==0.6.0',  # Enforces log formatting to be correct
         ]
         args: ['--enable-extensions=G']
   - repo: local

CHANGELOG.rst

@@ -2,8 +2,33 @@ Changelog
 =========
 
 
+`1.0.0`_ - 2020-01-14
+---------------------
+
+**Features**
+
+* Added a ``RETURN_HEADER`` setting, which will return the GUID as a header with the same name
+
+
+**Improvements**
+
+* Added a Django Rest Framework test and added DRF to the ``demoproj``
+
+* Improved tests to also check for headers in the response
+
+* Added tests for the new setting
+
+* Added examples to ``README.rst`` and docs, to show how the log messages get formatted
+
+* Added an API page to the docs
+
+* Fixed the ``readthedocs`` menu bug
+
+
+
 `0.3.1`_ - 2020-01-13
 ---------------------
+
 **Improvements**
 
 * Changed logging from f'strings' to %strings
@@ -77,3 +102,4 @@ Changelog
 .. _0.2.3: https://github.com/jonasks/django-guid/compare/0.2.2...0.2.3
 .. _0.3.0: https://github.com/jonasks/django-guid/compare/0.2.3...0.3.0
 .. _0.3.1: https://github.com/jonasks/django-guid/compare/0.3.0...0.3.1
+.. _1.0.0: https://github.com/jonasks/django-guid/compare/0.3.0...1.0.0

README.rst

@@ -24,6 +24,39 @@ allowing us to inject it into the logs.
 * Documentation: https://django-guid.readthedocs.io
 
 
+Example
+-------
+
+Using ``demoproj`` as an example, all the log messages **without** ``django-guid`` would look like this:
+
+.. code-block:: bash
+
+    INFO 2020-01-14 12:28:42,194 django_guid.middleware No Correlation-ID found in the header. Added Correlation-ID: 97c304252fd14b25b72d6aee31565843
+    INFO 2020-01-14 12:28:42,353 demoproj.views This is a DRF view log, and should have a GUID.
+    INFO 2020-01-14 12:28:42,354 demoproj.services.useless_file Some warning in a function
+
+
+With ``django-guid`` every log message has a GUID attached to it(``97c304252fd14b25b72d6aee31565843``),
+through the entire stack:
+
+.. code-block:: bash
+
+    INFO 2020-01-14 12:28:42,194 [None] django_guid.middleware No Correlation-ID found in the header. Added Correlation-ID: 97c304252fd14b25b72d6aee31565843
+    INFO 2020-01-14 12:28:42,353 [97c304252fd14b25b72d6aee31565843] demoproj.views This is a DRF view log, and should have a GUID.
+    INFO 2020-01-14 12:28:42,354 [97c304252fd14b25b72d6aee31565843] demoproj.services.useless_file Some warning in a function
+
+For multiple requests at the same time over multiple threads, see the `extended example docs <https://django-guid.readthedocs.io/en/latest/extended_example.html>`_.
+
+
+Why
+---
+
+``django-guid`` makes it extremely easy to track exactly what happened in any request. If you see an error
+in your log, you can use the attached GUID to search for any connected log message to that single request.
+The GUID can also be returned as a header and displayed to the end user of your application, allowing them
+to report an issue with a connected ID. ``django-guid`` makes troubleshooting easy.
+
+
 Settings
 --------
 
@@ -48,6 +81,12 @@ Settings
 
     Default: True
 
+* :code:`RETURN_HEADER`
+        Whether to return the GUID (Correlation-ID) as a header in the response or not.
+        It will have the same name as the :code:`GUID_HEADER_NAME` setting.
+
+    Default: True
+
 
 Installation
 ------------

demoproj/settings.py

@@ -29,6 +29,7 @@
     'django.contrib.sessions',
     'django.contrib.messages',
     'django.contrib.staticfiles',
+    'rest_framework',
 ]
 
 MIDDLEWARE = [

demoproj/urls.py

@@ -15,8 +15,9 @@
 """
 from django.urls import path
 
-from demoproj.views import index_view
+from demoproj.views import index_view, rest_view
 
 urlpatterns = [
     path('', index_view, name='index'),
+    path('api', rest_view, name='drf'),
 ]

demoproj/views.py

@@ -1,6 +1,9 @@
 import logging
 
 from django.http import HttpRequest, JsonResponse
+from rest_framework.decorators import api_view
+from rest_framework.response import Response
+from rest_framework.request import Request
 
 from demoproj.services.useless_file import useless_function
 
@@ -16,4 +19,17 @@ def index_view(request: HttpRequest) -> JsonResponse:
     """
     logger.info('This log message should have a GUID')
     useless_response = useless_function()
-    return JsonResponse({'Detail': f'It worked! Useless function response is {useless_response}'})
+    return JsonResponse({'detail': f'It worked! Useless function response is {useless_response}'})
+
+
+@api_view(('GET',))
+def rest_view(request: Request) -> Response:
+    """
+    Example DRF view that logs a log and calls a function that logs a log.
+
+    :param request: Request
+    :return: Response
+    """
+    logger.info('This is a DRF view log, and should have a GUID.')
+    useless_response = useless_function()
+    return Response(data={'detail': f'It worked! Useless function response is {useless_response}'})

demoproj/wsgi.py

@@ -0,0 +1,11 @@
+import os
+
+from django.core.wsgi import get_wsgi_application
+
+os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'demoproj.settings')
+
+# This application object is used by any WSGI server configured to use this
+# file. This includes Django's development server, if the WSGI_APPLICATION
+# setting points here.
+
+application = get_wsgi_application()

django_guid/__init__.py

@@ -4,4 +4,4 @@
 Adding imports here will break setup.py
 """
 
-__version__ = '0.3.1'
+__version__ = '1.0.0'

django_guid/config.py

@@ -13,6 +13,7 @@ def __init__(self) -> None:
         self.GUID_HEADER_NAME = 'Correlation-ID'
         self.VALIDATE_GUID = True
         self.SKIP_CLEANUP = False
+        self.RETURN_HEADER = True
 
         if hasattr(django_settings, 'DJANGO_GUID'):
             _settings = django_settings.DJANGO_GUID
@@ -30,6 +31,8 @@ def __init__(self) -> None:
                 raise ImproperlyConfigured('VALIDATE_GUID must be a boolean')
             if not isinstance(self.GUID_HEADER_NAME, str):
                 raise ImproperlyConfigured('GUID_HEADER_NAME must be a string')  # Note: Case insensitive
+            if not isinstance(self.RETURN_HEADER, bool):
+                raise ImproperlyConfigured('RETURN_HEADER must be a boolean')
         else:
             pass  # Do nothing if DJANGO_GUID not found in settings
 

django_guid/middleware.py

@@ -35,14 +35,16 @@ def __call__(self, request: HttpRequest) -> Union[HttpRequest, HttpResponse]:
         """
         # Ensure we don't get the previous request GUID attached to the logs from this file.
         if settings.SKIP_CLEANUP:
-            self.__class__.del_guid()
+            self.delete_guid()
         # Process request and store the GUID on the thread
         self.set_guid(self._get_id_from_header(request))
         # ^ Code above this line is executed before the view and later middleware
         response = self.get_response(request)
+        if settings.RETURN_HEADER:
+            response[settings.GUID_HEADER_NAME] = self.get_guid()  # Adds the GUID to the response header
         if not settings.SKIP_CLEANUP:
             # Delete the current request to avoid memory leak
-            self.__class__.del_guid()
+            self.delete_guid()
         return response
 
     @classmethod
@@ -67,7 +69,7 @@ def set_guid(cls, guid: str) -> None:
         cls._guid[threading.current_thread()] = guid
 
     @classmethod
-    def del_guid(cls) -> None:
+    def delete_guid(cls) -> None:
         """
         Delete the GUID that was stored for the current thread.
 
@@ -117,7 +119,7 @@ def _get_correlation_id_from_header(self, request: HttpRequest) -> str:
             return given_guid
         else:
             new_guid = self._generate_guid()
-            logger.info(f'%s is not a valid GUID. New GUID is %s', given_guid, new_guid)
+            logger.info('%s is not a valid GUID. New GUID is %s', given_guid, new_guid)
             return new_guid
 
     def _get_id_from_header(self, request: HttpRequest) -> str:

docs/api.rst

@@ -0,0 +1,61 @@
+API
+===
+
+Getting started
+---------------
+To use the API import the GuidMiddleware:
+
+.. code-block:: python
+
+    from django_guid.middleware import GuidMiddleware
+
+
+get_guid()
+----------
+* **Parameters**: ``default`` = ``None`` - What to return if no ``GUID`` is found.
+
+* **Returns**: ``str`` or ``default``
+
+Fetches the GUID.
+
+.. code-block:: python
+
+    guid = GuidMiddleware.get_guid(default=None)
+
+set_guid()
+----------
+* **Parameters**: ``guid``: ``str``
+
+Sets the GUID to the given input
+
+.. code-block:: python
+
+    GuidMiddleware.set_guid('My GUID')
+
+
+delete_guid()
+-------------
+Deletes the stored GUID
+
+.. code-block:: python
+
+    GuidMiddleware.delete_guid()
+
+
+Example usage
+-------------
+
+.. code-block:: python
+
+    import requests
+    from django.conf import settings
+
+    from django_guid.middleware import GuidMiddleware
+
+    requests.get(
+        url='http://localhost/api',
+        headers={
+            'Accept': 'application/json',
+            settings.DJANGO_GUID['GUID_HEADER_NAME']: GuidMiddleware.get_guid(),
+        }
+    )

docs/conf.py

@@ -26,7 +26,8 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = []
+
+extensions = ['sphinx_rtd_theme']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']

docs/contributing.rst

@@ -0,0 +1 @@
+.. include:: ../CONTRIBUTING.rst

docs/extended_example.rst

@@ -0,0 +1,72 @@
+.. _extended_example:
+
+Extended example
+================
+
+Using tools like ``ab`` (`Apache Benchmark <https://httpd.apache.org/docs/2.4/programs/ab.html>`_) we can benchmark our application with concurrent requests, simulating
+heavy load. This is an easy way to display the strength of ``django-guid``.
+
+Experiment
+----------
+
+First, we run our application like we would in a production environment:
+
+.. code-block:: bash
+
+    gunicorn demoproj.wsgi:application --bind 127.0.0.1:8080 -k gthread -w 4
+
+Then, we do 3 concurrent requests to one of our endpoints:
+
+.. code-block:: bash
+
+    ab -c 3 -n 3 http://127.0.0.1:8080/api
+
+
+This results in these logs:
+
+.. code-block:: bash
+
+    django-guid git:(master) ✗ gunicorn demoproj.wsgi:application --bind 127.0.0.1:8080 -k gthread -w 4
+
+    [2020-01-14 16:36:15 +0100] [8624] [INFO] Starting gunicorn 20.0.4
+    [2020-01-14 16:36:15 +0100] [8624] [INFO] Listening at: http://127.0.0.1:8080 (8624)
+    [2020-01-14 16:36:15 +0100] [8624] [INFO] Using worker: gthread
+    [2020-01-14 16:36:15 +0100] [8627] [INFO] Booting worker with pid: 8627
+    [2020-01-14 16:36:15 +0100] [8629] [INFO] Booting worker with pid: 8629
+    [2020-01-14 16:36:15 +0100] [8630] [INFO] Booting worker with pid: 8630
+    [2020-01-14 16:36:15 +0100] [8631] [INFO] Booting worker with pid: 8631
+
+    # First request
+    INFO 2020-01-14 15:40:48,953 [None] django_guid.middleware No Correlation-ID found in the header. Added Correlation-ID: 773fa6885e03493498077a273d1b7f2d
+    INFO 2020-01-14 15:40:48,954 [773fa6885e03493498077a273d1b7f2d] demoproj.views This is a DRF view log, and should have a GUID.
+    WARNING 2020-01-14 15:40:48,954 [773fa6885e03493498077a273d1b7f2d] demoproj.services.useless_file Some warning in a function
+    DEBUG 2020-01-14 15:40:48,954 [773fa6885e03493498077a273d1b7f2d] django_guid.middleware Deleting 773fa6885e03493498077a273d1b7f2d from _guid
+
+    # Second and third request arrives at the same time
+    INFO 2020-01-14 15:40:48,955 [None] django_guid.middleware No Correlation-ID found in the header. Added Correlation-ID: 0d1c3919e46e4cd2b2f4ac9a187a8ea1
+    INFO 2020-01-14 15:40:48,955 [None] django_guid.middleware No Correlation-ID found in the header. Added Correlation-ID: 99d44111e9174c5a9494275aa7f28858
+    INFO 2020-01-14 15:40:48,955 [0d1c3919e46e4cd2b2f4ac9a187a8ea1] demoproj.views This is a DRF view log, and should have a GUID.
+    INFO 2020-01-14 15:40:48,955 [99d44111e9174c5a9494275aa7f28858] demoproj.views This is a DRF view log, and should have a GUID.
+    WARNING 2020-01-14 15:40:48,955 [0d1c3919e46e4cd2b2f4ac9a187a8ea1] demoproj.services.useless_file Some warning in a function
+    WARNING 2020-01-14 15:40:48,955 [99d44111e9174c5a9494275aa7f28858] demoproj.services.useless_file Some warning in a function
+    DEBUG 2020-01-14 15:40:48,955 [0d1c3919e46e4cd2b2f4ac9a187a8ea1] django_guid.middleware Deleting 0d1c3919e46e4cd2b2f4ac9a187a8ea1 from _guid
+    DEBUG 2020-01-14 15:40:48,955 [99d44111e9174c5a9494275aa7f28858] django_guid.middleware Deleting 99d44111e9174c5a9494275aa7f28858 from _guid
+
+If we have a close look, we can see that the first request is completely done before the second and third arrives.
+How ever, the second and third request arrives at the exact same time, and since ``gunicorn`` is run with multiple workers,
+they are also handled concurrently. The result is logs that get mixed together, making them impossible to differentiate.
+
+Now, depending on how you view your logs you can easily track a single request down. In these docs, try using ``ctrl`` + ``f``
+and search for ``99d44111e9174c5a9494275aa7f28858``
+
+If you're logging to a file you could use ``grep``:
+
+
+.. code-block:: bash
+
+    ➜  ~ cat demoproj/logs.log | grep 99d44111e9174c5a9494275aa7f28858
+
+    INFO 2020-01-14 15:40:48,955 [None] django_guid.middleware No Correlation-ID found in the header. Added Correlation-ID: 99d44111e9174c5a9494275aa7f28858
+    INFO 2020-01-14 15:40:48,955 [99d44111e9174c5a9494275aa7f28858] demoproj.views This is a DRF view log, and should have a GUID.
+    WARNING 2020-01-14 15:40:48,955 [99d44111e9174c5a9494275aa7f28858] demoproj.services.useless_file Some warning in a function
+    DEBUG 2020-01-14 15:40:48,955 [99d44111e9174c5a9494275aa7f28858] django_guid.middleware Deleting 99d44111e9174c5a9494275aa7f28858 from _guid