1- Creating a Event Hub to send events from Azure to Devo
2- Tables
3- Structure folder of EventHubTrigger
4.1- Prerequisites
6.1- Uploading files to the trigger function
6.3- Instalation
6.4- Credentials
6.4.1- Upload the credentials to Azure Function App
6.4.2- Upload the credentials to Azure Key Vault
7- Enabling Azure function to access Event Hub stream
7.1- Event Hub side
7.2- Azure Function
8- Sending events from Azure Active Directory
9- Sending events from Azure Activity Logs
10- Links
This is not an agent, but it is a general guide on how to sent events from Azure.
The Azure Activity Log and Azure Active Directory logs can be easily ingested into the Devo platform by exporting the logs to an Azure event hub, which in turn triggers a cloud function to tag and send the events to Devo.
You can follow these instructions to send events from an EventHub to the Devo platform. There are two types of events that could be sent from Azure to Devo: from Monitor (Azure activities logs) and from Active Directory (Sign-In and Audit logs).
The Azure Activity Log is a subscription log that provides insight into subscription-level events that have occurred in Azure.
Sign-ins logs provides information about the usage of managed applications and user sign-in activities.
Audit logs provides traceability through logs for all changes done by various features within Azure AD. Examples of audit logs include changes made to any resources within Azure AD like adding or removing users, apps, groups, roles and policies.
Note that only some roles can create EventHubs in Azure. At the end of this article, there are some links that indicate who can create and access the data.
All Azure events Azure are storage in cloud.azure tech. Then, depending on the source of the events, these are saved in custom tables. Events that come from Azure Monitor (Azure Activity Log) are store in cloud.azure.activity.events and Azure Active Directory are store in cloud.azure.ad.
| Tag | Azure Operation Type | Description |
|---|---|---|
| cloud.azure.ad.audit.<zone> | AuditLogs | The Azure AD audit logs provide records of system activities for compliance. |
| cloud.azure.ad.signin.<zone> | SignInLogs | The user sign-ins report provides records about activities of AD users. |
| cloud.azure.activity.events.<zone> | Activity logs | Azure Activity logs (Action, Write, Delete) |
In this repository you can find the EventHubTrigger folder. It should be similar to your eventhubtrigger folder in Azure.
In this folder you will find the following files:
- index.js: it is the main file that contain the logic to send events from Azure to Devo.
- config.json: contain the initial configuration values.
- util.js: contain some utilities to use in the main file. For example, send logs or stats to Devo.
- function.json: Azure configuration file.
- package.json: list of modules to install.
If you want to send customs log to Devo according to keep your records, you can use sendLog and sendStats functions. In the index.js file are some examples.
- Have a Azure account with the permissions
- Have a Devo account
Go to your Azure portal account and follow the steps below.
Click Create a resource on the left side, find and select the Event Hubs resource and click on the Create button.
Fill the fields with the corresponding values. You may need to create a new Resource group if you have not already done so. Click on Create. This will take a few seconds.
Once the namespace is created, you can access it by clicking on All resources on the right side of the menu and then on the namespace.
Click on the Monitor option in the left side menu, then on Activity Log and then on the Export to Event Hub option.
Select the corresponding options with the susbcription, the namespace and the regions. Be sure to check Export to an event hub option. Then save the changes.
This may take several minutes. Once the event hub is created you can see it in the associated namespace resource.
Click on Create a resource option in left-hand menu, then search and select the Function App option. Then click on Create.
Fill in and select the fields corresponding your requirements. Make sure to select JavaScript in the Runtime Stack option. Click on Create. This may take several seconds.
Once that it was created you can check it in All resources option. Select the function app and then click on + icon in the Functions option. Choose the In-portal option as the development environment, and then click on the Continue button.
Choose the More templates... option and then click on the Finish and view templates button.
Choose the Azure Event Hub trigger. This could ask you to install an extension. Install it.
Fill in and select the fields according your requirements. In the Event Hub connection you must select the associated namespace.
Once the function app has been created, something like the following image should be displayed.
On the right side you can see two files and the option "Test". In the lower part you can see the console and the generated logs. And, on the left side, you can see the structure of the function app.
Now, you need to send the events to Devo.
First, you need to update:
- index.js file
and upload the
- package.json
- config.json
- util.js
files contained in this tutorial, inside the folder "EventHubTrigger".
Note: "function.json" is also included in the folder, but it is automatically generated with the trigger definition.
There are several ways to do this:
- Using Deployment Center (multiple alternatives: git, OneDrive, FTP, etc.)
- Using App Service Editor
In this guide we are going to explain how to work with the last one.
By default we will see something like this:
In this case, the function has a one trigger with two files, function.json and index.js.
The usage is simple, by right clicking in the "EXPLORE" menu, you can either create, upload, rename or delete files or folders. You can also directly modify any text in each file.
For this configuration we are going to do the following changes:
- Upload "config.json", "package.json" and "utils.js" from the EventHubTrigger folder mentioned in this documentation, by right clicking under the menu with the name of our trigger, in the white space, as you can see in the previous screenshot.
- Now we must include each uploaded file in the trigger folder. Select each element by holding the left mouse button and move the element inside the folder with the trigger name.
- You can either remove the index.js and upload the new definition or just copy the content of index.js in this documentation and replace the content autogenerated in the trigger function (see previous screenshot).
You can see the result in the following screenshot:
File config.json:
-
The domain_name attribute is the name of your domain in Devo.
-
The zone attribute correspond to one Azure region. For more detail about region read this article.
-
For detail about host and port attributes read the Devo official documentation
-
send_logs and send_stats attributes indicate if you want to send custom logs from the FunctionApp to Devo and send the statistics of the events.
-
CA_in_KV, Cert_in_KV and Key_in_KV attributes indicate if the certificate file is stored in Key Vault (true) or in the Function App (false).
This is a possible definition example:
{
"domain_name": "my_domain",
"zone": "West Europe",
"host": "eu.elb.relay.logtrust.net",
"port": 443,
"send_logs": false,
"send_stats": true,
"CA_in_KV": true,
"Cert_in_KV": true,
"Key_in_KV": true
}Return to the function screen and under the "Development Tools" menu select "Console".
Locate the trigger folder and install the devo js SDK and all dependencies with the following command:
npm install
This will generate a new folder (node_modules) with the packages.
After that, you must upload the credentials of your Devo domain according to the configuration file. You have two options:
- Upload the credentials to Azure Function App (local usage)
- Upload the credentials to Azure Key Vault
For more information about the credentials, please visit the Devo documentation: https://docs.devo.com/confluence/ndt/domain-administration/security-credentials
You can do it like the stepts defined in the title "Uploading files to the trigger function" or do the following if you need to upload a ZIP file.
On your computer, create a folder with the name certs, paste the credentials here and then compress this folder in zip format. Keep in mind that you should not change the credentials files names. Then, select the upload option and select the newly created zip file.
Unzip the certs.zip file from the console.
> unzip certs.zipDelete the zip file.
> rm certs.zipThe structure of your event hub function app should look like the following image
Go to Function App resource and click on Platform features and then in Identity.
Activate it and click on Save button.
Now, click on Create a resource option in left-hand menu, then search Key Vault option. Then click on Create
Fill in and select the fields corresponding your requirements and click on Next : Access policy > button.
Click on + Add Access Policy option and add the permission Get for Secret permissions and find your Function App. Click on select and Add buttons.
Finally, select Review + create option and click on Create button.
Go to the new resource and click on Secrets option and then in Generate/Import
Fill in and select the fields corresponding to your certificates. You must create one secret for each certificate file (CA, public cert and key cert)
You should have something like the following image
Now, you need to link each secret into Function App. For each secret copy the Secret Identifier.
Go to Configuration option in your Function App
Add a new application setting for each secret. The name of the application setting must be one of those: domainCA, domainCert and domainKey and the value must be like
@Microsoft.KeyVault(SecretUri=<kv_secret_identifier>)
⚠️ This step is usually performed by portal when integration with Event Hub for Azure Function is created: Just here for reference, troubleshooting or manual set up
A shared access signature (SAS) provides you with a way to grant limited access to resources in your Event Hubs namespace. SAS guards access to Event Hubs resources based on authorization rules. These rules are configured either on a namespace, or an entity (event hub or topic).
You can review SAS rules at All services -> Event Hubs. Then select the Event Hubs namespace you want to give access to:
Inside the Overview screen click on the Shared access policies option in the left side menu:
Then you can add a new policy or edit an existent policy:
RootManageSharedAccessKey is the default Shared access policy created by default for the namespace, but it has priviledges for all actions (Manage, Send, Listen). We encourage to create a new one with only Send priviledges.
After creating the policy, click on it again to see its details. Copy Connection string-primary key from the right side panel. It would be needed as the value for the applicaction property in the Azure Function.
In order to integrate Event Hub stream in the Azure Function as inbound binding we must configure it at function.json:
{
"bindings": [
{
"name": "eventHubMessages",
"connection": "DevoPSTestingNamespace_RootManageSharedAccessKey_EVENTHUB",
"eventHubName": "insights-operational-logs",
"consumerGroup": "$Default",
"cardinality": "many",
"dataType": "string",
"direction": "in",
"type": "eventHubTrigger"
}
]
}The connection parameter contains as value the key of the application settings pair that contains the value we copied from the last step in former section (Connection string-primary key). Therefore, inside Function's Application settings there is an entry whose key is DevoPSTestingNamespace_RootManageSharedAccessKey_EVENTHUB that contains the Connection string-primary key to authorize the access to the Event Hub by means of the Shared access policy we have just created.
You can check Function's Application settings at All services -> App_Services. Then select the App Service that contains the Azure Function listening for Event Hub events:
Inside the Overview screen click on the Configuration option under Settings section in the left side menu:
You will find Applications settings tab containing all the key-value settings for the application. There must be a setting whose key is DevoPSTestingNamespace_RootManageSharedAccessKey_EVENTHUB or whatever name was used in function.json. The vale, hidden by default but it can be disclosed by clicking on it, must contain the Connection string-primary key value we copied from the Shared access policy of the Event Hub namespace.
Before to start to retrieve Audit Logs and Sign-ins events from Azure Active Directory you will need to have the permissions necessaries and In order to export Sign-in data, your organization needs Azure AD P1 or P2 license.
Go to All services -> Azure Active Directory. Click on the Audit logs or Sign-ins option in the left side menu and then click in Export Data Settings.
Turn on diagnostics option
Fill in and select the corresponding values according to requirements and save your configuration.
Now you should start to retrieve events from Azure Active Directory.
Before to start to retrieve Activity Logs you will need to have the permissions necessaries.
Go to All services -> Activity log.
Click on the Diagnostic settings option on the top menu.
And then edit a existent setting Edit setting or create a new one + Add disgnostic setting.
Select the category details to be sent. Select the Stream to an event hub option. Fill in and select the corresponding values according to your Event Hub settings and save your configuration.
Now you should start to retrieve events from Activity logs.
See how to set up the export of Azure logs to event hub
Activity Log
https://docs.microsoft.com/en-us/azure/azure-monitor/platform/activity-logs-overview
Audit Log
https://docs.microsoft.com/en-us/azure/active-directory/reports-monitoring/concept-audit-logs
Sing-in logs
https://docs.microsoft.com/en-us/azure/active-directory/reports-monitoring/concept-sign-ins