Skip to content

Commit

Permalink
address PR comments
Browse files Browse the repository at this point in the history
  • Loading branch information
kcreddy committed Mar 14, 2024
1 parent 616e021 commit 208825c
Show file tree
Hide file tree
Showing 9 changed files with 48 additions and 55 deletions.
8 changes: 4 additions & 4 deletions packages/azure/_dev/build/docs/graphactivitylogs.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Microsoft Graph Activity Logs

Microsoft Graph Activity Logs provide an audit trail of all HTTP requests that the Microsoft Graph service received and processed for a tenant. Microsoft Graph Activity Logs gives full visibility into all transactions made by applications and other API clients that you have consented to in the tenant. Refer to [Microsoft Graph Activity Common Usecases](https://learn.microsoft.com/en-us/graph/microsoft-graph-activity-logs-overview#common-use-cases-for-microsoft-graph-activity-logs) for more use cases.
Microsoft Graph Activity Logs provide an audit trail of all HTTP requests that the Microsoft Graph service has received and processed for a tenant. Microsoft Graph Activity Logs gives full visibility into all transactions made by applications and other API clients that you have consented to in the tenant. Refer to [Microsoft Graph Activity Common Usecases](https://learn.microsoft.com/en-us/graph/microsoft-graph-activity-logs-overview#common-use-cases-for-microsoft-graph-activity-logs) for more use cases.

Tenant administrators can configure the collection and storage destinations of Microsoft Graph Activity Logs through Diagnostic Setting in the Entra Portal. This integrtaion uses Azure Event Hubs destination to stream Microsoft Graph Activity Logs to Elastic.
Tenant administrators can configure the collection and storage destinations of Microsoft Graph Activity Logs through Diagnostic Setting in the Entra Portal. This integration uses Azure Event Hubs destination to stream Microsoft Graph Activity Logs to Elastic.

## Requirements and Setup

Expand All @@ -27,7 +27,7 @@ Refer to [Microsoft Graph Activity Limitations](https://learn.microsoft.com/en-u

`eventhub` :
_string_
It is a fully managed, real-time data ingestion service. Elastic recommends using only letters, numbers, and the hyphen (-) character for Event Hub names to maximize compatibility. You can use existing Event Hubs having underscores (_) in the Event Hub name; in this case, the integration will replace underscores with hyphens (-) when it uses the Event Hub name to create dependent Azure resources behind the scenes (e.g., the storage account container to store Event Hub consumer offsets). Elastic also recommends using a separate event hub for each log type as the field mappings of each log type differ.
It is a fully managed, real-time data ingestion service. Elastic recommends using only letters, numbers, and the hyphen (-) character for Event Hub names to maximize compatibility. You _can_ use existing Event Hubs having underscores (_) in the Event Hub name; in this case, the integration will replace underscores with hyphens (-) when it uses the Event Hub name to create dependent Azure resources behind the scenes (e.g., the storage account container to store Event Hub consumer offsets). Elastic also recommends using a separate event hub for each log type as the field mappings of each log type differ.
Default value `insights-operational-logs`.

`consumer_group` :
Expand Down Expand Up @@ -55,7 +55,7 @@ The storage account container where the integration stores the checkpoint data f

`resource_manager_endpoint` :
_string_
Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different azure environment.
Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different Azure environment.

Resource manager endpoints:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{#if connection_string}}
connection_string: {{connection_string}}
{{/if}}
{{#if storage_account_container }}
{{#if storage_account_container}}
storage_account_container: {{storage_account_container}}
{{else}}
{{#if eventhub}}
Expand All @@ -27,7 +27,7 @@ tags:
{{#if preserve_original_event}}
- preserve_original_event
{{/if}}
{{#each tags as |tag i|}}
{{#each tags as |tag|}}
- {{tag}}
{{/each}}
{{#contains "forwarded" tags}}
Expand All @@ -43,4 +43,4 @@ sanitize_options:
{{/if}}
{{#if sanitize_singlequotes}}
- SINGLE_QUOTES
{{/if}}
{{/if}}
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
paths:
{{#each paths as |path i|}}
{{#each paths as |path|}}
- {{path}}
{{/each}}
exclude_files: [".gz$"]
tags:
{{#if preserve_original_event}}
- preserve_original_event
{{/if}}
{{#each tags as |tag i|}}
{{#each tags as |tag|}}
- {{tag}}
{{/each}}
{{#contains "forwarded" tags}}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,4 +88,4 @@ processors:
on_failure:
- set:
field: error.message
value: '{{ _ingest.on_failure_message }}'
value: '{{ _ingest.on_failure_message }}'
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,6 @@ processors:
target_field: azure-eventhub
ignore_missing: true
tag: rename-azure
- script:
source: ctx.message = ctx.message.replace(params.empty_field_name, '')
params:
empty_field_name: '"":"",'
ignore_failure: true
tag: script-message-emptyfields
- rename:
field: message
target_field: event.original
Expand Down Expand Up @@ -175,10 +169,10 @@ processors:
tag: rename-properties-durationMs
- script:
lang: painless
source: if (ctx.event.duration!= null) {ctx.event.duration = ctx.event.duration
* params.param_nano;}
params:
param_nano: 1000000
source: |
if (ctx.event.duration!= null) {
ctx.event.duration = ctx.event.duration * 1000000;
}
ignore_failure: true
tag: script-duration
- remove:
Expand Down
12 changes: 6 additions & 6 deletions packages/azure/data_stream/graphactivitylogs/fields/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,23 @@
- name: category
type: keyword
description: |
Azure Event Category. For example, Graph Activity Logs has value `MicrosoftGraphActivityLogs`
Azure Event Category. For example, Graph Activity Logs has value `MicrosoftGraphActivityLogs`.
- name: operation_version
type: keyword
description: |
The Graph API version of the event.
- name: operation_name
type: keyword
description: |
Operation name
Operation name.
- name: result_signature
type: keyword
description: |
Result signature
Result signature.
- name: properties
type: group
description: |
Event properties
Event properties.
fields:
- name: api_version
type: keyword
Expand Down Expand Up @@ -53,7 +53,7 @@
- name: is_billable
type: boolean
description: |
The record size in bytes.
Specifies whether ingesting the data is billable. When _IsBillable is false ingestion isn't billed to your Azure account.
- name: operation_id
type: keyword
description: |
Expand Down Expand Up @@ -81,7 +81,7 @@
- name: source_system
type: keyword
description: |
The type of agent the event was collected by. For example, OpsManager for Windows agent, either direct connect or Operations Manager, Linux for all Linux agents, or Azure for Azure Diagnostics
The type of agent the event was collected by. For example, OpsManager for Windows agent, either direct connect or Operations Manager, Linux for all Linux agents, or Azure for Azure Diagnostics.
- name: time_generated
type: date
description: |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,39 +4,39 @@
- name: subscription_id
type: keyword
description: |
Azure subscription ID
Azure subscription ID.
- name: correlation_id
type: keyword
description: |
Correlation ID
Correlation ID.
- name: tenant_id
type: keyword
description: |
tenant ID
tenant ID.
- name: resource
type: group
fields:
- name: id
type: keyword
description: |
Resource ID
Resource ID.
- name: group
type: keyword
description: |
Resource group
Resource group.
- name: provider
type: keyword
description: |
Resource type/namespace
Resource type/namespace.
- name: namespace
type: keyword
description: |
Resource type/namespace
Resource type/namespace.
- name: name
type: keyword
description: |
Name
Name.
- name: authorization_rule
type: keyword
description: |
Authorization rule
Authorization rule.
3 changes: 1 addition & 2 deletions packages/azure/data_stream/graphactivitylogs/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,8 @@ streams:
multi: false
required: false
show_user: false
description: >
description: >-
The storage account container where the integration stores the checkpoint data for the consumer group. It is an advanced option to use with extreme care. You MUST use a dedicated storage account container for each Azure log type (activity, sign-in, audit logs, and others). DO NOT REUSE the same container name for more than one Azure log type. See [Container Names](https://docs.microsoft.com/en-us/rest/api/storageservices/naming-and-referencing-containers--blobs--and-metadata#container-names) for details on naming rules from Microsoft. The integration generates a default container name if not specified.
- name: tags
type: text
title: Tags
Expand Down
36 changes: 18 additions & 18 deletions packages/azure/docs/graphactivitylogs.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Microsoft Graph Activity Logs

Microsoft Graph Activity Logs provide an audit trail of all HTTP requests that the Microsoft Graph service received and processed for a tenant. Microsoft Graph Activity Logs gives full visibility into all transactions made by applications and other API clients that you have consented to in the tenant. Refer to [Microsoft Graph Activity Common Usecases](https://learn.microsoft.com/en-us/graph/microsoft-graph-activity-logs-overview#common-use-cases-for-microsoft-graph-activity-logs) for more use cases.
Microsoft Graph Activity Logs provide an audit trail of all HTTP requests that the Microsoft Graph service has received and processed for a tenant. Microsoft Graph Activity Logs gives full visibility into all transactions made by applications and other API clients that you have consented to in the tenant. Refer to [Microsoft Graph Activity Common Usecases](https://learn.microsoft.com/en-us/graph/microsoft-graph-activity-logs-overview#common-use-cases-for-microsoft-graph-activity-logs) for more use cases.

Tenant administrators can configure the collection and storage destinations of Microsoft Graph Activity Logs through Diagnostic Setting in the Entra Portal. This integrtaion uses Azure Event Hubs destination to stream Microsoft Graph Activity Logs to Elastic.
Tenant administrators can configure the collection and storage destinations of Microsoft Graph Activity Logs through Diagnostic Setting in the Entra Portal. This integration uses Azure Event Hubs destination to stream Microsoft Graph Activity Logs to Elastic.

## Requirements and Setup

Expand All @@ -27,7 +27,7 @@ Refer to [Microsoft Graph Activity Limitations](https://learn.microsoft.com/en-u

`eventhub` :
_string_
It is a fully managed, real-time data ingestion service. Elastic recommends using only letters, numbers, and the hyphen (-) character for Event Hub names to maximize compatibility. You can use existing Event Hubs having underscores (_) in the Event Hub name; in this case, the integration will replace underscores with hyphens (-) when it uses the Event Hub name to create dependent Azure resources behind the scenes (e.g., the storage account container to store Event Hub consumer offsets). Elastic also recommends using a separate event hub for each log type as the field mappings of each log type differ.
It is a fully managed, real-time data ingestion service. Elastic recommends using only letters, numbers, and the hyphen (-) character for Event Hub names to maximize compatibility. You _can_ use existing Event Hubs having underscores (_) in the Event Hub name; in this case, the integration will replace underscores with hyphens (-) when it uses the Event Hub name to create dependent Azure resources behind the scenes (e.g., the storage account container to store Event Hub consumer offsets). Elastic also recommends using a separate event hub for each log type as the field mappings of each log type differ.
Default value `insights-operational-logs`.

`consumer_group` :
Expand Down Expand Up @@ -55,7 +55,7 @@ The storage account container where the integration stores the checkpoint data f

`resource_manager_endpoint` :
_string_
Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different azure environment.
Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different Azure environment.

Resource manager endpoints:

Expand Down Expand Up @@ -220,9 +220,9 @@ An example event for `graphactivitylogs` looks as following:
| Field | Description | Type |
|---|---|---|
| @timestamp | Event timestamp. | date |
| azure.correlation_id | Correlation ID | keyword |
| azure.graphactivitylogs.category | Azure Event Category. For example, Graph Activity Logs has value `MicrosoftGraphActivityLogs` | keyword |
| azure.graphactivitylogs.operation_name | Operation name | keyword |
| azure.correlation_id | Correlation ID. | keyword |
| azure.graphactivitylogs.category | Azure Event Category. For example, Graph Activity Logs has value `MicrosoftGraphActivityLogs`. | keyword |
| azure.graphactivitylogs.operation_name | Operation name. | keyword |
| azure.graphactivitylogs.operation_version | The Graph API version of the event. | keyword |
| azure.graphactivitylogs.properties.api_version | The API version of the event. | keyword |
| azure.graphactivitylogs.properties.app_id | The identifier for the application. | keyword |
Expand All @@ -231,28 +231,28 @@ An example event for `graphactivitylogs` looks as following:
| azure.graphactivitylogs.properties.client_auth_method | Indicates how the client was authenticated. For a public client, the value is 0. If client ID and client secret are used, the value is 1. If a client certificate was used for authentication, the value is 2. | integer |
| azure.graphactivitylogs.properties.client_request_id | The client request identifier when sent. If no client request identifier is sent, the value will be equal to the operation identifier. | keyword |
| azure.graphactivitylogs.properties.identity_provider | The identity provider that authenticated the subject of the token. | keyword |
| azure.graphactivitylogs.properties.is_billable | The record size in bytes. | boolean |
| azure.graphactivitylogs.properties.is_billable | Specifies whether ingesting the data is billable. When _IsBillable is false ingestion isn't billed to your Azure account. | boolean |
| azure.graphactivitylogs.properties.operation_id | The identifier for the batch. For non-batched requests, this will be unique per request. For batched requests, this will be the same for all requests in the batch. | keyword |
| azure.graphactivitylogs.properties.request_uri | The URI of the request. | keyword |
| azure.graphactivitylogs.properties.roles | The roles in token claims. | keyword |
| azure.graphactivitylogs.properties.scopes | The scopes in token claims. | keyword |
| azure.graphactivitylogs.properties.service_principal_id | The identifier of the servicePrincipal making the request. | keyword |
| azure.graphactivitylogs.properties.sign_in_activity_id | The identifier representing the sign-in activitys. | keyword |
| azure.graphactivitylogs.properties.source_system | The type of agent the event was collected by. For example, OpsManager for Windows agent, either direct connect or Operations Manager, Linux for all Linux agents, or Azure for Azure Diagnostics | keyword |
| azure.graphactivitylogs.properties.source_system | The type of agent the event was collected by. For example, OpsManager for Windows agent, either direct connect or Operations Manager, Linux for all Linux agents, or Azure for Azure Diagnostics. | keyword |
| azure.graphactivitylogs.properties.time_generated | The date and time the request was received. | date |
| azure.graphactivitylogs.properties.token_issued_at | The timestamp the token was issued at. | date |
| azure.graphactivitylogs.properties.type | The name of the table. | keyword |
| azure.graphactivitylogs.properties.user_agent | The user agent information related to request. | keyword |
| azure.graphactivitylogs.properties.wids | Denotes the tenant-wide roles assigned to this user. | keyword |
| azure.graphactivitylogs.result_signature | Result signature | keyword |
| azure.resource.authorization_rule | Authorization rule | keyword |
| azure.resource.group | Resource group | keyword |
| azure.resource.id | Resource ID | keyword |
| azure.resource.name | Name | keyword |
| azure.resource.namespace | Resource type/namespace | keyword |
| azure.resource.provider | Resource type/namespace | keyword |
| azure.subscription_id | Azure subscription ID | keyword |
| azure.tenant_id | tenant ID | keyword |
| azure.graphactivitylogs.result_signature | Result signature. | keyword |
| azure.resource.authorization_rule | Authorization rule. | keyword |
| azure.resource.group | Resource group. | keyword |
| azure.resource.id | Resource ID. | keyword |
| azure.resource.name | Name. | keyword |
| azure.resource.namespace | Resource type/namespace. | keyword |
| azure.resource.provider | Resource type/namespace. | keyword |
| azure.subscription_id | Azure subscription ID. | keyword |
| azure.tenant_id | tenant ID. | keyword |
| client.geo.city_name | City name. | keyword |
| client.geo.continent_name | Name of the continent. | keyword |
| client.geo.country_iso_code | Country ISO code. | keyword |
Expand Down

0 comments on commit 208825c

Please sign in to comment.