Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[winlog] Add agent fields to align with Windows integration #5135

Merged
merged 4 commits into from
Jan 31, 2023
Merged

[winlog] Add agent fields to align with Windows integration #5135

merged 4 commits into from
Jan 31, 2023

Conversation

nicpenning
Copy link
Contributor

@nicpenning nicpenning commented Jan 28, 2023

There is no mapping to the agent/host.* fields which causes conflicts in the data views. These fields should probably be mapped for consistency and usability.

This file was copied directly from: https://github.com/elastic/integrations/blob/main/packages/windows/data_stream/sysmon_operational/fields/agent.yml

Please label this PR with one of the following labels, depending on the scope of your change:

  • Enhancement

What does this PR do?

This PR makes the Custom Windows Event log integration more consistent with the other Windows Event ingestion. By default, for example, the host.ip field will get mapped as a keyword which is not ideal and causes a field conflict. This seems to be the logical way to get this implemented unless this integration is being overhauled or deprecated. This is all predicated on having the agent.yml fields actually doing what I think it does which is adding these fields to the logs-winlog.winlog@pacakge template.

Using version 1.10.0 of the integration you can see that that host.* fields don't exist:

image

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

There is no mapping to the agent/host.* fields which causes conflicts in the data views. These fields should probably be mapped for consistency and usability.

This file was copied directly from: https://github.com/elastic/integrations/blob/main/packages/windows/data_stream/sysmon_operational/fields/agent.yml
@nicpenning nicpenning requested a review from a team as a code owner January 28, 2023 21:40
@elasticmachine
Copy link

elasticmachine commented Jan 28, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-01-31T07:22:38.261+0000

  • Duration: 14 min 57 sec

Test stats 🧪

Test Results
Failed 0
Passed 2
Skipped 0
Total 2

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@efd6
Copy link
Contributor

efd6 commented Jan 31, 2023

/test

@efd6 efd6 added the enhancement New feature or request label Jan 31, 2023
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please run elastic-package build to update the docs with the new fields.

Alternatively, if you cannot do this, please make these changes.

diff --git a/packages/winlog/docs/README.md b/packages/winlog/docs/README.md
index b29900062..d4f6043e9 100644
--- a/packages/winlog/docs/README.md
+++ b/packages/winlog/docs/README.md
@@ -26,6 +26,19 @@ To achieve this, `renderXml` needs to be set to `1` in your [inputs.conf](https:
 | Field | Description | Type |
 |---|---|---|
 | @timestamp | Event timestamp. | date |
+| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
+| cloud.availability_zone | Availability zone in which this host is running. | keyword |
+| cloud.image.id | Image ID for the cloud instance. | keyword |
+| cloud.instance.id | Instance ID of the host machine. | keyword |
+| cloud.instance.name | Instance name of the host machine. | keyword |
+| cloud.machine.type | Machine type of the host machine. | keyword |
+| cloud.project.id | Name of the project in Google Cloud. | keyword |
+| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
+| cloud.region | Region in which this host is running. | keyword |
+| container.id | Unique container id. | keyword |
+| container.image.name | Name of the image the container was built on. | keyword |
+| container.labels | Image labels. | object |
+| container.name | Container name. | keyword |
 | data_stream.dataset | Data stream dataset. | constant_keyword |
 | data_stream.namespace | Data stream namespace. | constant_keyword |
 | data_stream.type | Data stream type. | constant_keyword |
@@ -33,6 +46,23 @@ To achieve this, `renderXml` needs to be set to `1` in your [inputs.conf](https:
 | event.created | event.created contains the date/time when the event was first read by an agent, or by your pipeline. This field is distinct from @timestamp in that @timestamp typically contain the time extracted from the original event. In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent's or pipeline's ability to keep up with your event source. In case the two timestamps are identical, @timestamp should be used. | date |
 | event.dataset | Event dataset | constant_keyword |
 | event.module | Event module | constant_keyword |
+| host.architecture | Operating system architecture. | keyword |
+| host.containerized | If the host is a container. | boolean |
+| host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
+| host.hostname | Hostname of the host. It normally contains what the `hostname` command returns on the host machine. | keyword |
+| host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`. | keyword |
+| host.ip | Host ip addresses. | ip |
+| host.mac | Host mac addresses. | keyword |
+| host.name | Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
+| host.os.build | OS build information. | keyword |
+| host.os.codename | OS codename, if any. | keyword |
+| host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
+| host.os.kernel | Operating system kernel version as a raw string. | keyword |
+| host.os.name | Operating system name, without the version. | keyword |
+| host.os.name.text | Multi-field of `host.os.name`. | text |
+| host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
+| host.os.version | Operating system version as a raw string. | keyword |
+| host.type | Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
 | input.type | Type of Filebeat input. | keyword |
 | log.level | Original log level of the log event. If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn't specify one, you may put your event transport's severity here (e.g. Syslog severity). Some examples are `warn`, `err`, `i`, `informational`. | keyword |
 | message | For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message. | match_only_text |

@nicpenning
Copy link
Contributor Author

I believe I added the necessary fields to the docs. Please let me know how that works out.

@efd6
Copy link
Contributor

efd6 commented Jan 31, 2023

/test

@elasticmachine
Copy link

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (0/0) 💚
Files 100.0% (0/0) 💚 2.504
Classes 100.0% (0/0) 💚 2.504
Methods 66.667% (2/3) 👎 -24.683
Lines 100.0% (0/0) 💚 7.81
Conditionals 100.0% (0/0) 💚

Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@efd6 efd6 merged commit 3884b51 into elastic:main Jan 31, 2023
@elasticmachine
Copy link

Package winlog - 1.11.0 containing this change is available at https://epr.elastic.co/search?package=winlog

@nicpenning nicpenning deleted the patch-3 branch February 1, 2023 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants