Integrate Application Insights

.github/workflows/python-package.yml

@@ -27,7 +27,7 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install flake8 pytest
+        python -m pip install flake8 pytest pytest-mock pytest-xdist pytest-forked
         if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
     - name: Lint with flake8
       run: |
@@ -37,4 +37,4 @@ jobs:
         flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
     - name: Test with pytest
       run: |
-        pytest
+        pytest --forked

.vscode/settings.json

@@ -4,5 +4,8 @@
     "azureFunctions.pythonVenv": ".venv",
     "azureFunctions.projectLanguage": "Python",
     "azureFunctions.projectRuntime": "~4",
-    "debug.internalConsoleOptions": "neverOpen"
+    "debug.internalConsoleOptions": "neverOpen",
+    "python.testing.pytestArgs": [
+        "--forked"
+    ]
 }

README.md

@@ -1,4 +1,4 @@
-# FuncTown
+# 🎷 FuncTown 🎷
 
 `FuncTown` is a python library to make working with azure functions easier and remove boilerplate.
 
@@ -49,6 +49,117 @@ All this should remove boilerplate from Azure-Functions.
 
 🎷 Welcome to FuncTown! 🎷
 
+## Decorators
+
+Most of the functionality that `FuncTown` provides is through decorators.
+These decorators can be used to wrap your functions and provide additional functionality.
+
+Note that almost all decorators pass along additional arguments to your function.
+So it is generally a good idea to modify your function signature to accept these arguments:
+
+```python
+# instead of main(req: func.HttpRequest) -> func.HttpResponse:
+def main(req: func.HttpRequest, *params, **kwargs) -> func.HttpResponse:
+    # ...
+```
+
+> **Note:** The `last_decorator` parameter is used to indicate that this is the last decorator in the chain.
+> This makes sure that the signature of the function is consumable by Azure Functions.
+> Alternatively you can also the the `@functown.clean` decorator to clean up the signature.
+
+### `handle_errors`
+
+The `handle_errors` decorator can be used to wrap your function and provide error handling.
+It will catch any exception that is thrown in the function and provide a response to the user.
+
+```python
+from functown import handle_errors
+
+@handle_errors(debug=True, last_decorator=True)
+def main(req: func.HttpRequest, *params, **kwargs) -> func.HttpResponse:
+    logging.info('Python HTTP trigger function processed a request.')
+
+    # ...
+
+    return func.HttpResponse("success", status_code=200)
+```
+
+You can also specify the degree to which debug information is returned as part of the response
+with the following parameters:
+
+* `debug` (bool): Defines if general exceptions are written to the logs with stack trace (default: `False`)
+* `log_all_errors` (bool): Defines if all exceptions are written to the logs with stack trace (including errors that might explicitly contain user data such as `TokenError`) (default: `False`)
+* `return_errors` (bool): Defines if the error message is returned as part of the response (for quicker debugging) (default: `False`)
+
+### `metrics_all`
+
+The `metrics_all` decorator can be used to wrap your function and provide logging of metrics in Application Insights.
+This includes 4 major categories:
+
+* `enable_logger`: Enables sending of general `Trace` data to Application Insights through the use of a `logger` object passed to the function (of type `logging.Logger`)
+* `enable_events`: Enables sending of `CustomEvent` data to Application Insights through the use of a `events` object passed to the function (of type `logging.logger`)
+* `enable_tracer`: Enables sending of `Request` data to Application Insights through the use of a `tracer` object passed to the function (of type `opencensus.trace.tracer.Tracer`)
+FIXME: update metrics here
+* `enable_metrics`: Enables sending of `Metric` data to Application Insights through the use of a `metrics` object passed to the function (of type `opencensus.stats.stats.Stats`)
+
+Note that each of these elements has additional sub-parameters that can be set.
+
+#### logger
+
+```python
+from functown import metrics_logger
+from logging import Logger
+
+@metrics_logger(enable_logger=True, last_decorator=True)
+def main(req: func.HttpRequest, logger: Logger, *params, **kwargs) -> func.HttpResponse:
+    logger.info('Python HTTP trigger function processed a request.')
+
+    # ...
+
+    return func.HttpResponse("success", status_code=200)
+```
+
+#### events
+
+```python
+from functown import metrics_events
+from logging import Logger
+
+@metrics_events(enable_events=True, last_decorator=True)
+def main(req: func.HttpRequest, events: Logger, *params, **kwargs) -> func.HttpResponse:
+    # log event
+    events.info('Custom Event', extra={'custom_dimensions': {'foo': 'bar'}})
+
+    # ...
+
+    return func.HttpResponse("success", status_code=200)
+```
+
+#### tracer
+
+```python
+from functown import metrics_tracer
+from opencensus.trace.tracer import Tracer
+
+@metrics_tracer(enable_tracer=True, tracer_sample=1.0, last_decorator=True)
+def main(req: func.HttpRequest, tracer: Tracer, *params, **kwargs) -> func.HttpResponse:
+    # start span
+    with tracer.span(name="span_name") as span:
+        # everything in this block will be part of the span (and send sampled to Application Insights)
+        # ...
+
+    # ...
+
+    return func.HttpResponse("success", status_code=200)
+```
+
+#### metrics
+
+```python
+from functown import log_metrics
+# FIXME: implement
+```
+
 ## Versioning
 
 We use [SemVer](http://semver.org/) for versioning. For the versions available, see the tags on this repository.
@@ -131,18 +242,31 @@ curl -X POST -H "Content-Type: application/json" -d '{"body_param": "some body p
 >
 > (esp. when using a non-consumption tier)
 
+**Note on App Insights:** Metrics and traces take a time to show up in the App Insights. You can also use the `logs` field in the response to see the logs of the function.
+
 ### Testing functown Code
 
 This function also allows to test changes to functown in the development process.
-For that simply copy the current `functown` folder into the `example` folder and redeploy the function app.
+For that simply copy the current `functown` folder into the `example` folder and rename it to `functown_local`, then redeploy the function app.
 
 You should also update the version number in the `__init__.py` file of the `functown` folder before (which needs to be done for any changes, see section on `Versioning`).
 
+> **Note:** When you update dependencies you also need to temporarily add these dependencies to the `requirements.txt` file in the `example` folder (and remove them before commiting!).
+
 You can verify that the new version of the code was picked up by the first log statement in your return.
 
 ## Notes
 
 * The `@handle_error` decorator returns additional information to be used in the development process. This can also expose infrormation to attackers, so use responsibly (i.e. make sure to disable for production environments).
 
+If you want to develop on the library or execute tests, you can install the conda environment:
+
+```bash
+# remove old:
+# conda env remove -n functown
+conda env create -f conda-dev.yml
+conda activate functown
+```
+
 > ‼️ If you find this library helpful or have suggestions please let me know.
 Also any contributions are welcome! ‼️

conda-dev.yaml

@@ -0,0 +1,14 @@
+name: functown
+channels:
+    - conda-forge
+    - defaults
+dependencies:
+    - python=3.10
+    - pip
+    - pip:
+        - -r requirements.txt
+        - pytest
+        - pytest-cov
+        - pytest-mock
+        - pytest-xdist
+        - pytest-forked

example/.funcignore

@@ -3,4 +3,5 @@
 local.settings.json
 test
 .venv
-config.json
+config.json
+__pycache__

example/.gitignore

@@ -136,4 +136,5 @@ __azurite_db*__.json
 
 # ignore local artifacts
 config.json
-functown_local
+functown_local
+functown

example/TestFuncTown/__init__.py → example/TestErrorHandler/__init__.py

@@ -1,5 +1,5 @@
 """
-Example function using functown
+Example function to test error handling capabilities.
 
 Copyright (c) 2023, Felix Geilert
 """
@@ -8,6 +8,7 @@
 import logging
 import json
 import os
+from typing import List
 
 import azure.functions as func
 
@@ -16,21 +17,27 @@
 except ImportError:
     import functown as ft
 
-from logging_helper import ListHandler
-
 
 # retrieve the debug flag from the environment
 DEBUG = bool(strtobool(os.getenv("FUNC_DEBUG", "False")))
 
 
-@ft.handle_errors(debug=True, log_all_errors=DEBUG, return_errors=DEBUG)
-def main(req: func.HttpRequest) -> func.HttpResponse:
+@ft.ErrorHandler(
+    debug=True,
+    log_all_errors=DEBUG,
+    return_errors=DEBUG,
+    enable_logger=True,
+    return_logs=True,
+)
+def main(
+    req: func.HttpRequest,
+    logger: logging.Logger,
+    logs: List[str],
+    **kwargs,
+) -> func.HttpResponse:
     logging.info("Python HTTP trigger function processed a request.")
 
     # create a logger (allow to return log as list)
-    logger = logging.getLogger("func_logger")
-    logs = []
-    logger.addHandler(ListHandler(logs))
     logger.info(f"Using functown v{ft.__version__}")
 
     # generate args parser
@@ -52,6 +59,15 @@ def main(req: func.HttpRequest) -> func.HttpResponse:
         logger.info("raising exception")
         raise Exception("This is a test exception")
 
+    use_special_exc = args.get_body_query(
+        "use_special_exception", False, "bool", default=False
+    )
+    logger.info(f"use_special_exc: {use_special_exc}")
+    if use_special_exc:
+        # this should raise an exception that is handled by the decorator
+        logger.info("raising special exception")
+        raise ft.errors.TokenError("Your token is invalid", 400)
+
     # retrieve numbers
     print_num = args.get_body_query("print_num", False, map_fct=int, default=0)
     logger.info(f"print_num: {print_num}")

example/TestInsightsEvents/__init__.py

@@ -0,0 +1,71 @@
+"""
+Example function to test insights events
+
+Copyright (c) 2023, Felix Geilert
+"""
+
+from distutils.util import strtobool
+import logging
+import json
+import os
+from typing import List
+
+import azure.functions as func
+
+try:
+    import functown_local as ft
+except ImportError:
+    import functown as ft
+
+
+# retrieve the debug flag from the environment
+DEBUG = bool(strtobool(os.getenv("FUNC_DEBUG", "False")))
+INST_KEY = os.getenv("APP_INSIGHTS_KEY", None)
+
+
+@ft.ErrorHandler(
+    debug=True,
+    log_all_errors=DEBUG,
+    return_errors=DEBUG,
+    enable_logger=True,
+    return_logs=True,
+)
+@ft.InsightsEventHandler(
+    instrumentation_key=INST_KEY,
+    enable_events=True,
+)
+def main(
+    req: func.HttpRequest,
+    logger: logging.Logger,
+    logs: List[str],
+    events: logging.Logger,
+    **kwargs,
+) -> func.HttpResponse:
+    logging.info("Python HTTP trigger function processed a request.")
+
+    # create a logger (allow to return log as list)
+    logger.info(f"Using functown v{ft.__version__}")
+
+    # generate args parser
+    args = ft.RequestArgHandler(req)
+
+    # check if event should be logged
+    use_event = args.get_body_query("use_event", False, "bool", default=False)
+    if use_event is True:
+        events.info("This is a test event")
+        events.info(
+            "This is a test event with a dict",
+            extra={"custom_dimensions": {"test": "dict"}},
+        )
+
+    # generate report
+    payload = {
+        "completed": True,
+        "results": {
+            "use_event": use_event,
+        },
+        "logs": logs,
+    }
+    return func.HttpResponse(
+        json.dumps(payload), mimetype="application/json", status_code=200
+    )

example/TestInsightsEvents/function.json

@@ -0,0 +1,20 @@
+{
+  "scriptFile": "__init__.py",
+  "bindings": [
+    {
+      "authLevel": "function",
+      "type": "httpTrigger",
+      "direction": "in",
+      "name": "req",
+      "methods": [
+        "get",
+        "post"
+      ]
+    },
+    {
+      "type": "http",
+      "direction": "out",
+      "name": "$return"
+    }
+  ]
+}