Skip to content

Commit

Permalink
Adds documentation for log type categories (opensearch-project#5181)
Browse files Browse the repository at this point in the history 
* new log categories

Signed-off-by: Heather Halter <hdhalter@amazon.com>

* fixed topic name in link

Signed-off-by: Heather Halter <hdhalter@amazon.com>

* added log names to table

Signed-off-by: Heather Halter <hdhalter@amazon.com>

* Update log-types.md

minor changes needed in the names would be Microsoft Azure for Azure,  Linux System logs instead of Sys logs

Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update log-types.md

Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* remove log name column

Signed-off-by: Heather Halter <hdhalter@amazon.com>

* remove table column formatting

Signed-off-by: Heather Halter <hdhalter@amazon.com>

* Update _security-analytics/sec-analytics-config/custom-log-type.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update _security-analytics/sec-analytics-config/log-types.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update _security-analytics/sec-analytics-config/log-types.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update _security-analytics/sec-analytics-config/log-types.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update _security-analytics/sec-analytics-config/log-types.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update _security-analytics/sec-analytics-config/log-types.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update _security-analytics/sec-analytics-config/log-types.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update _security-analytics/sec-analytics-config/log-types.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Apply suggestions from code review

Editorial updates.

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update log-types.md

Fixed case in table.

Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update log-types.md

Double-checked the UI and category names are capitalized.

Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

* Update _security-analytics/sec-analytics-config/log-types.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>

---------

Signed-off-by: Heather Halter <hdhalter@amazon.com>
Signed-off-by: Heather Halter <HDHALTER@AMAZON.COM>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
  • Loading branch information
@hdhalter @kolchfa-aws
@natebower @harshavamsi
3 people authored and harshavamsi committed Oct 31, 2023
1 parent 87123f3 commit 394ddc2
Show file tree
Hide file tree
Showing 3 changed files with 47 additions and 39 deletions.
28 changes: 7 additions & 21 deletions _security-analytics/sec-analytics-config/custom-log-type.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,34 +8,20 @@ nav_order: 18

# Creating custom log types

Log types represent the different sources of data used for threat detection in Security Analytics. In addition to the standard log types supported by Security Analytics, you can create custom log types for your threat detectors. Follow the steps provided here to create a custom log type.


## The Log types page

To navigate to the **Log types** page, select **Log types** under **Detectors** in the navigation menu. The following image shows the **Log types** landing page.

<img src="{{site.url}}{{site.baseurl}}/images/Security/c-log-type.png" alt="The Log types landing page." width="85%">

The table that lists the log types provides the name of the log type, its description, and identifies whether it's a standard OpenSearch-defined log type or a custom log type. The following list describes the main features found on the **Log types** page and the actions you can take:

* Select the log type **Name** to open the log type's details page. The **Details** tab is shown by default. This tab includes the log type's ID. You can also select the **Detection rules** tab to show all detection rules associated with the log type.
* In the **Actions** column, you can select the trash can icon ({::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/alerting/trash-can-icon.png" class="inline-icon" alt="trash can icon"/>{:/}) to delete a custom log type (you cannot delete a standard OpenSearch-defined log type). Follow the prompts to confirm and safely remove the custom log type.
* Select **Create log type** in the upper-right corner of the screen to begin creating a custom log type. The **Create log type** page opens. Continue with the steps in the section that follows to create a custom log type.

Log types represent the different sources of data used for threat detection in Security Analytics. In addition to the standard [log types]({{site.url}}{{site.baseurl}}/security-analytics/sec-analytics-config/log-types/) supported by Security Analytics, you can create custom log types for your threat detectors.

## Creating a custom log type

After selecting **Create log type** on the **Log types** page, the **Create log type** page opens and provides the necessary fields for creating a new log type:

1. Enter a name for the log type.
To create a custom log type:
1. From the dashboard, select **OpenSearch Plugins** > **Security Analytics**, and then select **Detectors** > **Log types**.
1. Select **Create log type**.
1. Enter a name and, optionally, a description for the log type.

The log type name supports characters a--z (lowercase), 0--9, hyphens, and underscores.
{: .note }

1. Enter a description for the log type.
1. Select **Create log type** in the lower-right corner of the screen. The screen returns to the **Log types** page, and the new log type appears in the list of all log types. Note that the source for the new log type indicates **Custom**.

1. Select a category. The categories are listed in [Supported log types]({{site.url}}{{site.baseurl}}/security-analytics/sec-analytics-config/log-types/).
1. Select **Create log type** in the lower-right corner of the screen. The screen returns to the **Log types** page, and the new log type appears in the list. Note that the source for the new log type indicates **Custom**.

## Log type API

Expand Down
58 changes: 40 additions & 18 deletions _security-analytics/sec-analytics-config/log-types.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,44 @@ nav_order: 16

# Supported log types

Logs contain raw data about events that happen throughout a system and in its separate parts. The following table shows the log types that are currently supported by Security Analytics for ingestion, mapping, and monitoring.

| Log type | Description |
| :--- |:--- |
| **Network events** | A log that records events that happen in a system's network, such as login attempts and application events. |
| **DNS logs** | A log that stores Domain Name System (DNS) activity. <br> <br> Security Analytics supports [Open Cybersecurity Schema Framework](https://docs.aws.amazon.com/security-lake/latest/userguide/open-cybersecurity-schema-framework.html) (OCSF) log and event data, which includes the Security Lake log type Route 53. Given that Route 53 is a log that captures DNS activity, its log type should be specified as DNS logs when defining a detector. |
| **Apache access logs** | A log type that is responsible for recording data for all requests processed by an Apache HTTP server.
| **Windows logs** | Logs that record events that have happened in the operating system, applications, and other system services for Windows.
| **AD/LDAP logs** | Active Directory logs that track such things as LDAP queries, errors from the LDAP server, time-out events, and unsecure LDAP binds.
| **System logs** | Logs that record events happening in the operating system.
| **AWS CloudTrail logs** | Logs that monitor events for an AWS CloudTrail account. OpenSearch can ingest CloudTrail log data from both [AWS Simple Storage Service](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) (S3) accounts and [Amazon Security Lake](https://docs.aws.amazon.com/security-lake/latest/userguide/what-is-security-lake.html) service accounts.
| **Amazon S3 access logs** | These logs track requests for access to an Amazon S3 bucket.
| **Google Workspace logs** | Logs for Google Workspace that can monitor log entries such as admin actions, group and group membership actions, and events having to do with logging in.
| **GitHub actions** | Logs that monitor workflows created by [GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions).
| **Microsoft 365 logs** | Microsoft 365 audit logs collect a range of data for Microsoft 365 including records from call details, performance data, SQL Server, security events, and access control activity.
| **Okta events** | These logs record Okta events from a range of actions such as downloading an export file, requesting an application access, or revoking privileges.
| **Microsoft Azure logs** | Logs that monitor log data for cloud applications managed by Microsoft Azure Cloud Services.
| **VPC Flow logs** | [VPC Flow Logs](https://docs.aws.amazon.com/prescriptive-guidance/latest/logging-monitoring-for-application-owners/vpc-flow-logs.html) capture information about the IP traffic going to and from network interfaces in your VPC.
Logs contain raw data about events that happen throughout a system and in its separate parts. As of OpenSearch 2.11, log types are grouped by category to help select, filter, and search the log types.

To navigate to the **Log types** page, select **Log types** under **Detectors** in the **Security Analytics** navigation menu. The page shows the name of the log type, its description, its category, and identifies whether it's a standard OpenSearch-defined log type or a custom log type. The following image shows the **Log types** landing page with the Category column selected and the **Category** filter you can use to filter the list by the category.

<img src="{{site.url}}{{site.baseurl}}/images/Security/c-log-type.png" alt="The Log types landing page." width="85%">

The following table shows the log types that are currently supported by Security Analytics for ingestion, mapping, and monitoring.

| Category | Log type | Description |
| :--- |:--- |:--- |
| Access Management | `Ad_ldap` | Active Directory logs that track LDAP queries, errors from the LDAP server, timeout events, and unsecure LDAP binds. |
| Access Management | `Apache_access` | Apache access logs that record data for all requests processed by an Apache HTTP server. |
| Access Management | `Okta` | Okta logs that record Okta events from a range of actions, such as downloading an export file, requesting application access, or revoking privileges. |
| Applications | `GitHub` | GitHub logs that monitor workflows created by [GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions). |
| Applications| `Gworkspace` | Google Workspace logs that monitor log entries such as admin actions, group and group membership actions, and events related to logging in. |
| Applications| `M365` | Microsoft 365 audit logs that collect a range of data for Microsoft 365, including records from call details, performance data, SQL Server, security events, and access control activity. |
| Cloud Services | `Azure` | Microsoft Azure logs that monitor log data for cloud applications managed by Azure Cloud Services. |
| Cloud Services | `CloudTrail` | AWS CloudTrail logs that monitor events for an AWS CloudTrail account. OpenSearch can ingest CloudTrail log data from both [Amazon Simple Storage Service](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) (Amazon S3) accounts and [Amazon Security Lake](https://docs.aws.amazon.com/security-lake/latest/userguide/what-is-security-lake.html) service accounts. |
| Cloud Services | `S3` | Amazon S3 logs that track requests for access to an S3 bucket. |
| Network Activity| `Dns` | DNS logs that store DNS activity. |
| Network Activity | `Network` | Network logs that record events that happen in a system's network, such as login attempts and application events. |
| Network Activity | `vpcflow` | [VPC Flow Logs](https://docs.aws.amazon.com/prescriptive-guidance/latest/logging-monitoring-for-application-owners/vpc-flow-logs.html) that capture information about the IP traffic going to and from network interfaces in your virtual private cloud (VPC). |
| Security | `Waf` | Web Application Firewall (WAF) logs (introduced in OpenSearch 2.11) for users that require monitoring of the WAF use case that's provided out of the box in the Security plugin. The role of WAF is to monitor and filter HTTP traffic between a web application and the internet. WAF prevents common security attacks, such as cross-site scripting (XSS) and SQL Injection (SQi). |
| System Activity | `Linux` | Linux system logs that record Linux syslog events. |
| System Activity | `Windows` | Windows logs that record events that have happened in the operating system, applications, and other Windows system services. |
| Other | `Email` | Logs that record email activity. |


## Page actions

The following list describes the main features found on the **Log types** page and the actions you can take:

* Select the log type **Name** to open the log type's details page. The **Details** tab is shown by default. This tab includes the log type's ID. You can also select the **Detection rules** tab to show all detection rules associated with the log type.
* In the **Actions** column, you can select the trash can icon ({::nomarkdown}<img src="{{site.url}}{{site.baseurl}}/images/alerting/trash-can-icon.png" class="inline-icon" alt="trash can icon"/>{:/}) to delete a custom log type (you cannot delete a standard OpenSearch-defined log type). Follow the prompts to confirm and safely remove the custom log type.
* Select **Create log type** in the upper-right corner of the screen to begin creating a custom log type. The **Create log type** page opens. Continue with the steps in the section that follows to create a custom log type.
* Using the **Category** and **Source** dropdowns, you can sort by the log type category or source, respectively.

## Related articles
[Creating custom log types]({{site.url}}{{site.baseurl}}/security-analytics/sec-analytics-config/custom-log-type/)


Binary file modified images/Security/c-log-type.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 394ddc2

Please sign in to comment.