Skip to content

Latest commit

 

History

History
867 lines (717 loc) · 19.7 KB

README.md

File metadata and controls

867 lines (717 loc) · 19.7 KB
Raw

Moesif Middleware for Python ASGI-based Frameworks

Built For Latest Version Language Versions Software License Source Code

With Moesif middleware for Python ASGI-based frameworks, you can automatically log API calls and send them to Moesif for API analytics and monitoring.

If you're new to Moesif, see our Getting Started resources to quickly get up and running.

Overview

This middleware allows you to integrate Moesif's API analytics and API monetization features with minimal configuration into APIs that are built on Python ASGI-based (Asynchronous Server Gateway Interface) frameworks.

ASGI is a spiritual successor to WSGI (Web Server Gateway Interface). ASGI provides a standard interface between async-capable Python web servers, frameworks, and applications. Many Python Frameworks are built on top of ASGI, such as FastAPI.

Prerequisites

Before using this middleware, make sure you have the following:

Get Your Moesif Application ID

After you log into Moesif Portal, you can get your Moesif Application ID during the onboarding steps. You can always access the Application ID any time by following these steps from Moesif Portal after logging in:

  1. Select the account icon to bring up the settings menu.
  2. Select Installation or API Keys.
  3. Copy your Moesif Application ID from the Collector Application ID field.

Accessing the settings menu in Moesif Portal

Install the Middleware

Install with pip using the following command:

pip install moesifasgi

Configure the Middleware

See the available configuration options to learn how to configure the middleware for your use case.

How to Use

FastAPI

Simply add MoesifMiddleware to a FastAPI application using the add_middleware method:

from moesifasgi import MoesifMiddleware

moesif_settings = {
    'APPLICATION_ID': 'YOUR_APPLICATION_ID',
    'LOG_BODY': True,
    # ... For other options see below.
}

app = FastAPI()

app.add_middleware(MoesifMiddleware, settings=moesif_settings)

Replace YOUR_MOESIF_APPLICATION_ID with your Moesif Application ID.

Other ASGI Frameworks

If you are using a framework that is built on top of ASGI, it should work just by adding the Moesif middleware. Please read the documentation for your specific framework on how to add middlewares.

Optional: Capturing Outgoing API Calls

In addition to your own APIs, you can also start capturing calls out to third party services through by setting the CAPTURE_OUTGOING_REQUESTS option:

from moesifasgi import MoesifMiddleware

moesif_settings = {
    'APPLICATION_ID': 'YOUR_APPLICATION_ID',
    'LOG_BODY': True,
    'CAPTURE_OUTGOING_REQUESTS': False,
}

app = FastAPI()

app.add_middleware(MoesifMiddleware, settings=moesif_settings)

For configuration options specific to capturing outgoing API calls, see Options For Outgoing API Calls.

Troubleshoot

For a general troubleshooting guide that can help you solve common problems, see Server Troubleshooting Guide.

Other troubleshooting supports:

Repository Structure

.
├── BUILDING.md
├── examples/
├── images/
├── LICENSE
├── MANIFEST.in
├── moesifasgi/
├── README.md
├── requirements.txt
├── setup.cfg
└── setup.py

Configuration Options

The following sections describe the available configuration options for this middleware. You can set these options in a Python dictionary and then pass that as a parameter to add_middleware when you add this middleware. See the sample FastAPI code for an example.

Some configuration options use request and response as input arguments. These correspond to the Requests and Responses objects respectively.

APPLICATION_ID (Required)

Data type
String

A string that identifies your application in Moesif.

SKIP

Data type Parameters Return type
Function (request, response) Boolean

Optional.

A function that takes a request and a response, and returns True if you want to skip this particular event.

IDENTIFY_USER

Data type Parameters Return type
Function (request, response) String

Optional, but highly recommended.

A function that takes a request and a response, and returns a string that represents the user ID used by your system.

Moesif identifies users automatically. However, due to the differences arising from different frameworks and implementations, provide this function to ensure user identification properly.

IDENTIFY_COMPANY

Data type Parameters Return type
Function (request, response) String

Optional.

A function that takes a request and response, and returns a string that represents the company ID for this event.

GET_METADATA

Data type Parameters Return type
Function (request, response) Dictionary

Optional.

This function allows you to add custom metadata that Moesif can associate with the event. The metadata must be a simple Python dictionary that can be converted to JSON.

For example, you may want to save a virtual machine instance ID, a trace ID, or a resource ID with the request.

GET_SESSION_TOKEN

Data type Parameters Return type
Function (request, response) String

Optional.

A function that takes a request and response, and returns a string that represents the session token for this event.

Similar to users and companies, Moesif tries to retrieve session tokens automatically. But if it doesn't work for your service, provide this function to help identify sessions.

MASK_EVENT_MODEL

Data type Parameters Return type
Function (EventModel) EventModel

Optional.

A function that takes the final Moesif event model and returns an event model with desired data removed.

The return value must be a valid eventt model required by Moesif data ingestion API. For more information about the EventModel object, see the Moesif Python API documentation.

DEBUG

Data type
Boolean

Optional.

Set to True to print debug logs if you're having integration issues.

LOG_BODY

Data type Default
Boolean True

Optional.

Whether to log request and response body to Moesif.

BATCH_SIZE

Data type Default
int 25

An optional field name that specifies the maximum batch size when sending to Moesif.

EVENT_BATCH_TIMEOUT

Data type Default
int 1

Optional.

Maximum time in seconds to wait before sending a batch of events to Moesif when reading from the queue.

EVENT_QUEUE_SIZE

Data type Default
int 1000_000

Optional.

The maximum number of event objects queued in memory pending upload to Moesif. For a full queue, additional calls to MoesifMiddleware returns immediately without logging the event. Therefore, set this option based on the event size and memory capacity you expect.

AUTHORIZATION_HEADER_NAME

Data type Default
String authorization

Optional.

A request header field name used to identify the User in Moesif. It also supports a comma separated string. Moesif checks headers in order like "X-Api-Key,Authorization".

AUTHORIZATION_USER_ID_FIELD

Data type Default
String sub

Optional.

A field name used to parse the user from authorization header in Moesif.

BASE_URI

Data type
String

Optional.

A local proxy hostname when sending traffic through secure proxy. Remember to set this field when using secure proxy. For more information, see Secure Proxy documentation..

Options For Outgoing API Calls

The following options apply to outgoing API calls. These are calls you initiate using the Python Requests library to third parties like Stripe or to your own services.

Several options use request and response as input arguments. These correspond to the Requests library's request or response objects.

If you are not using ASGI, you can import moesifpythonrequest directly.

CAPTURE_OUTGOING_REQUESTS (Required)

Data type Default
Boolean False

Set to True to capture all outgoing API calls.

GET_METADATA_OUTGOING
Data type Parameters Return type
Function (req, res) Dictionary

Optional.

A function that enables you to return custom metadata associated with the logged API calls.

Takes in the Requests request and response objects as arguments.

We recommend that you implement a function that returns a dictionary containing your custom metadata. The dictionary must be a valid one that can be encoded into JSON. For example, you may want to save a virtual machine instance ID, a trace ID, or a resource ID with the request.

SKIP_OUTGOING
Data type Parameters Return type
Function (req, res) Boolean

Optional.

A function that takes a Requests request and response objects, and returns True if you want to skip this particular event.

IDENTIFY_USER_OUTGOING
Data type Parameters Return type
Function (req, res) String

Optional, but highly recommended.

A function that takes Requests request and response objects, and returns a string that represents the user ID used by your system.

While Moesif tries to identify users automatically, different frameworks and your implementation might vary. So we highly recommend that you accurately provide a user ID using this function.

IDENTIFY_COMPANY_OUTGOING
Data type Parameters Return type
Function (req, res) String

Optional.

A function that takes Requests request and response objects, and returns a string that represents the company ID for this event.

GET_SESSION_TOKEN_OUTGOING
Data type Parameters Return type
Function (req, res) String

Optional.

A function that takes Requests request and response objects, and returns a string that corresponds to the session token for this event.

Similar to user IDs, Moesif tries to get the session token automatically. However, if you setup differs from the standard, this function can help tying up events together and help you replay the events.

LOG_BODY_OUTGOING
Data type Default
Boolean True

Optional.

Set to False to remove logging request and response body.

Examples

See the example FastAPI app in the examples/ folder of this repository that uses this middleware.

Here's another sample FastAPI app:

# Your custom code that returns a user id string
custom_user = "12345"

async def custom_identify_user():
    return custom_user

# identify user using async mode
async def identify_user(request, response):
    user = await custom_identify_user()
    return user

# identify user not using async mode
def identify_user(request, response):
    return custom_user

# Your custom code that returns a company id string
custom_company = "67890"

async def custom_identify_company():
    return custom_company

# identify company using async mode
async def identify_company(request, response):
    company = await custom_identify_company()
    return company

# identify company not using async mode
def identify_company(request, response):
    return custom_company

custom_session_token = "XXXXXXXXXXXXXX"

async def custom_get_token():
    # If you don't want to use the standard ASGI session token,
    # add your custom code that returns a string for session/API token
    return custom_session_token

# get session token using async mode
async def get_token(request, response):
    result = await custom_get_token()
    return result

# get session token not using async mode
def get_token(request, response):
    return custom_session_token

custom_metadata = {
    'datacenter': 'westus',
    'deployment_version': 'v1.2.3',
}

async def custom_get_metadata():
    return custom_metadata

# get metadata using async mode
async def get_metadata(request, response):
    result = await custom_get_metadata()
    return result

# get metadata not using async mode
def get_metadata(request, response):
    return custom_metadata

skip_route = "health/probe"

async def custom_should_skip(request):
    # Your custom code that returns `True` to skip logging
    return skip_route in request.url._url

# should skip check using async mode
async def should_skip(request, response):
    result = await custom_should_skip(request)
    return result

# should skip check not using async mode
def should_skip(request, response):
    return skip_route in request.url._url

def custom_mask_event(eventmodel):
    # Your custom code to change or remove any sensitive fields
    if 'password' in eventmodel.response.body:
        eventmodel.response.body['password'] = None
    return eventmodel

# mask event using async mode
async def mask_event(eventmodel):
    return custom_mask_event(eventmodel)

# mask event not using async mode
def mask_event(eventmodel):
    return custom_mask_event(eventmodel)


moesif_settings = {
    'APPLICATION_ID': 'YOUR_MOESIF_APPLICATION_ID',
    'DEBUG': False,
    'LOG_BODY': True,
    'IDENTIFY_USER': identify_user,
    'IDENTIFY_COMPANY': identify_company,
    'GET_SESSION_TOKEN': get_token,
    'SKIP': should_skip,
    'MASK_EVENT_MODEL': mask_event,
    'GET_METADATA': get_metadata,
    'CAPTURE_OUTGOING_REQUESTS': False
}

app = FastAPI()

app.add_middleware(MoesifMiddleware, settings=moesif_settings)

You can use OAuth2 in your FastAPI app with this middleware. For more information, see OAuth2 with Password (and hashing), Bearer with JWT tokens.

Tested versions

Moesif has validated this middleware against the following frameworks and framework versions:

Framework Version
fastapi > 0.51.0 - 0.78.0
fastapi 0.108.0

Explore Other Integrations

Explore other integration options from Moesif: