title	description	services	author	ms.service	ms.topic	ms.date	ms.author
How to log events to Azure Event Hubs in Azure API Management \| Microsoft Docs	Learn how to log events to Azure Event Hubs in Azure API Management. Event Hubs is a highly scalable data ingress service.	api-management	dlepow	azure-api-management	how-to	09/04/2024	danlep

How to log events to Azure Event Hubs in Azure API Management

[!INCLUDE api-management-availability-all-tiers]

This article describes how to log API Management events using Azure Event Hubs.

Azure Event Hubs is a highly scalable data ingress service that can ingest millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Event Hubs acts as the "front door" for an event pipeline, and once data is collected into an event hub, it can be transformed and stored using any real-time analytics provider or batching/storage adapters. Event Hubs decouples the production of a stream of events from the consumption of those events, so that event consumers can access the events on their own schedule.

[!INCLUDE api-management-workspace-availability]

Prerequisites

An API Management service instance. If you don't have one, see Create an API Management service instance.
An Azure Event Hubs namespace and event hub. For detailed steps, see Create an Event Hubs namespace and an event hub using the Azure portal.

[!NOTE] The Event Hubs resource can be in a different subscription or even a different tenant than the API Management resource

Configure access to the event hub

To log events to the event hub, you need to configure credentials for access from API Management. API Management supports either of the two following access mechanisms:

A managed identity for your API Management instance (recommended)
An Event Hubs connection string

Note

Where possible, Microsoft recommends using managed identity credentials for enhanced security.

Option 1: Configure API Management managed identity

Enable a system-assigned or user-assigned managed identity for API Management in your API Management instance.
- If you enable a user-assigned managed identity, take note of the identity's Client ID.
Assign the identity the Azure Event Hubs Data sender role, scoped to the Event Hubs namespace or to the event hub used for logging. To assign the role, use the Azure portal or other Azure tools.

Option 2: Configure Event Hubs connection string

To create an Event Hubs connection string, see Get an Event Hubs connection string.

You can use a connection string for the Event Hubs namespace or for the specific event hub you use for logging from API Management.
The shared access policy for the connection string must enable at least Send permissions.

Create an API Management logger

The next step is to configure a logger in your API Management service so that it can log events to the event hub.

Create and manage API Management loggers by using the API Management REST API directly or by using tools including Azure PowerShell, a Bicep template, or an Azure Resource Management template.

Option 1: Logger with managed identity credentials (recommended)

You can configure an API Management logger to an event hub using either system-assigned or user-assigned managed identity credentials.

Logger with system-assigned managed identity credentials

For prerequisites, see Configure API Management managed identity.

REST API

Use the API Management Logger - Create or Update REST API with the following request body.

{
  "properties": {
    "loggerType": "azureEventHub",
    "description": "Event Hub logger with system-assigned managed identity",
    "credentials": {
         "endpointAddress":"<EventHubsNamespace>.servicebus.windows.net",
         "identityClientId":"SystemAssigned",
         "name":"<EventHubName>"
    }
  }
}

Bicep

Include a snippet similar to the following in your Bicep template.

resource ehLoggerWithSystemAssignedIdentity 'Microsoft.ApiManagement/service/loggers@2022-08-01' = {
  name: 'ContosoLogger1'
  parent: '<APIManagementInstanceName>'
  properties: {
    loggerType: 'azureEventHub'
    description: 'Event hub logger with system-assigned managed identity'
    credentials: {
      endpointAddress: '<EventHubsNamespace>.servicebus.windows.net'
      identityClientId: 'systemAssigned'
      name: '<EventHubName>'
    }
  }
}

ARM

Include a JSON snippet similar to the following in your Azure Resource Manager template.

{
  "type": "Microsoft.ApiManagement/service/loggers",
  "apiVersion": "2022-08-01",
  "name": "ContosoLogger1",
  "properties": {
    "loggerType": "azureEventHub",
    "description": "Event Hub logger with system-assigned managed identity",
    "resourceId": "<EventHubsResourceID>",
    "credentials": {
      "endpointAddress": "<EventHubsNamespace>.servicebus.windows.net",
      "identityClientId": "SystemAssigned",
      "name": "<EventHubName>"
    },
  }
}

Logger with user-assigned managed identity credentials

For prerequisites, see Configure API Management managed identity.

REST API

Use the API Management Logger - Create or Update REST API with the following request body.

{
  "properties": {
    "loggerType": "azureEventHub",
    "description": "Event Hub logger with user-assigned managed identity",
    "credentials": {
         "endpointAddress":"<EventHubsNamespace>.servicebus.windows.net",
         "identityClientId":"<ClientID>",
         "name":"<EventHubName>"
    }
  }
}

Bicep

Include a snippet similar to the following in your Bicep template.

resource ehLoggerWithUserAssignedIdentity 'Microsoft.ApiManagement/service/loggers@2022-08-01' = {
  name: 'ContosoLogger1'
  parent: '<APIManagementInstanceName>'
  properties: {
    loggerType: 'azureEventHub'
    description: 'Event Hub logger with user-assigned managed identity'
    credentials: {
      endpointAddress: '<EventHubsNamespace>.servicebus.windows.net'
      identityClientId: '<ClientID>'
      name: '<EventHubName>'
    }
  }
}

ARM

Include a JSON snippet similar to the following in your Azure Resource Manager template.

{
  "type": "Microsoft.ApiManagement/service/loggers",
  "apiVersion": "2022-08-01",
  "name": "ContosoLogger1",
  "properties": {
    "loggerType": "azureEventHub",
    "description": "Event Hub logger with user-assigned managed identity",
    "resourceId": "<EventHubsResourceID>",
    "credentials": {
      "endpointAddress": "<EventHubsNamespace>.servicebus.windows.net",
      "identityClientId": "<ClientID>",
      "name": "<EventHubName>"
    },
  }
}

Option 2. Logger with connection string credentials

For prerequisites, see Configure Event Hubs connection string.

Note

Where possible, Microsoft recommends configuring the logger with managed identity credentials. See Configure logger with managed identity credentials, earlier in this article.

PowerShell

The following example uses the New-AzApiManagementLogger cmdlet to create a logger to an event hub by configuring a connection string.

# API Management service-specific details
$apimServiceName = "apim-hello-world"
$resourceGroupName = "myResourceGroup"

# Create logger
$context = New-AzApiManagementContext -ResourceGroupName $resourceGroupName -ServiceName $apimServiceName
New-AzApiManagementLogger -Context $context -LoggerId "ContosoLogger1" -Name "ApimEventHub" -ConnectionString "Endpoint=sb://<EventHubsNamespace>.servicebus.windows.net/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<key>" -Description "Event hub logger with connection string"

Bicep

Include a snippet similar to the following in your Bicep template.

resource ehLoggerWithConnectionString 'Microsoft.ApiManagement/service/loggers@2022-08-01' = {
  name: 'ContosoLogger1'
  parent: '<APIManagementInstanceName>'
  properties: {
    loggerType: 'azureEventHub'
    description: 'Event Hub logger with connection string credentials'
    credentials: {
      connectionString: 'Endpoint=sb://<EventHubsNamespace>.servicebus.windows.net/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<key>'
      name: 'ApimEventHub'
    }
  }
}

ARM

Include a JSON snippet similar to the following in your Azure Resource Manager template.

{
  "type": "Microsoft.ApiManagement/service/loggers",
  "apiVersion": "2022-08-01",
  "name": "ContosoLogger1",
  "properties": {
    "loggerType": "azureEventHub",
    "description": "Event Hub logger with connection string credentials",
    "resourceId": "<EventHubsResourceID>"
    "credentials": {
      "connectionString": "Endpoint=sb://<EventHubsNamespace>/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<key>",
      "name": "ApimEventHub"
    },
  }
}

Configure log-to-eventhub policy

Once your logger is configured in API Management, you can configure your log-to-eventhub policy to log the desired events. For example, use the log-to-eventhub policy in the inbound policy section to log requests, or in the outbound policy section to log responses.

Browse to your API Management instance.
Select APIs, and then select the API to which you want to add the policy. In this example, we're adding a policy to the Echo API in the Unlimited product.
Select All operations.
On the top of the screen, select the Design tab.
In the Inbound processing or Outbound processing window, select the </> (code editor) icon. For more information, see How to set or edit policies.
Position your cursor in the inbound or outbound policy section.

In the window on the right, select Advanced policies > Log to EventHub. This inserts the log-to-eventhub policy statement template.

<log-to-eventhub logger-id="logger-id">
    @{
        return new JObject(
            new JProperty("EventTime", DateTime.UtcNow.ToString()),
            new JProperty("ServiceName", context.Deployment.ServiceName),
            new JProperty("RequestId", context.RequestId),
            new JProperty("RequestIp", context.Request.IpAddress),
            new JProperty("OperationName", context.Operation.Name)
        ).ToString();
    }
</log-to-eventhub>

Replace logger-id with the name of the logger that you created in the previous step.
You can use any expression that returns a string as the value for the log-to-eventhub element. In this example, a string in JSON format containing the date and time, service name, request ID, request IP address, and operation name is logged.

Select Save to save the updated policy configuration. As soon as it's saved, the policy is active and events are logged to the designated event hub.

Note

The maximum supported message size that can be sent to an event hub from this API Management policy is 200 kilobytes (KB). If a message that is sent to an event hub is larger than 200 KB, it will be automatically truncated, and the truncated message will be transferred to the event hub. For larger messages, consider using Azure Storage with Azure API Management as a workaround to bypass the 200KB limit. More details can be found in this article.

Preview the log in Event Hubs by using Azure Stream Analytics

You can preview the log in Event Hubs by using Azure Stream Analytics queries.

In the Azure portal, browse to the event hub that the logger sends events to.
Under Features, select the Process data tab.
On the Enable real time insights from events card, select Start.
You should be able to preview the log on the Input preview tab. If the data shown isn't current, select Refresh to see the latest events.

Next steps

Learn more about Azure Event Hubs
Learn more about API Management and Event Hubs integration
- Logger entity reference
- log-to-eventhub policy reference
Learn more about integration with Azure Application Insights

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api-management-howto-log-event-hubs.md

api-management-howto-log-event-hubs.md

How to log events to Azure Event Hubs in Azure API Management

Prerequisites

Configure access to the event hub

Option 1: Configure API Management managed identity

Option 2: Configure Event Hubs connection string

Create an API Management logger

Option 1: Logger with managed identity credentials (recommended)

Logger with system-assigned managed identity credentials

REST API

Bicep

ARM

Logger with user-assigned managed identity credentials

REST API

Bicep

ARM

Option 2. Logger with connection string credentials

PowerShell

Bicep

ARM

Configure log-to-eventhub policy

Preview the log in Event Hubs by using Azure Stream Analytics

Next steps

Files

api-management-howto-log-event-hubs.md

Latest commit

History

api-management-howto-log-event-hubs.md

File metadata and controls

How to log events to Azure Event Hubs in Azure API Management

Prerequisites

Configure access to the event hub

Option 1: Configure API Management managed identity

Option 2: Configure Event Hubs connection string

Create an API Management logger

Option 1: Logger with managed identity credentials (recommended)

Logger with system-assigned managed identity credentials

Logger with user-assigned managed identity credentials

Option 2. Logger with connection string credentials

Configure log-to-eventhub policy

Preview the log in Event Hubs by using Azure Stream Analytics

Next steps