Update Beats to ECS 1.8.0 #23465
Merged
Update Beats to ECS 1.8.0 #23465
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adriansr
added
enhancement
in progress
botelastic
bot
added
the
needs_team
adriansr
changed the title
adriansr
changed the title
botelastic
bot
removed
the
needs_team
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪Steps errors
Expand to view the steps failures
|
| Test | Results |
|---|---|
| Failed | 0 |
| Passed | 46595 |
| Skipped | 4804 |
| Total | 51399 |
adriansr
force-pushed
the
feature-ecs-1.8
branch
from
05e507a
to
6ab99ef
Compare
Upgrading ECS Go definitions to 1.8 caused Packetbeat's HTTP event_test to fail due to a couple of new ECS fields introduced in v1.7 not being expected. Those are: - request.mime_type - response.mime_type Packetbeat doesn't actually fill those fields. That task is acomplished by the detect_mime_type processor.
adriansr
force-pushed
the
feature-ecs-1.8
branch
from
8979980
to
376b26f
Compare
Adds the host.os.type field introduced by ECS 1.8.0. Possible values for this field are: - linux - macos - unix - windows The field will be missing for OSes not in the list. Related #23118
* User enhancements for powershell module * User enhancements for security and sysmon module * Add registry category to events * Add session category to events * Set target group when possible
* Improve ECS mappings and upgrade to ecs 1.8 * Run mage update
* Add new ECS user and categories features to google_workspace/gsuite * Update CHANGELOG.next.asciidoc Co-authored-by: Adrian Serrano <adrisr83@gmail.com> Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
* Packetbeat changes for ECS 1.8 * Remove unused parameter
Updates Auditbeat to new ECS 1.8. - Support new user/group fields provided by go-libaudit. - Support AUDIT_LOGIN. - Adds golden file tests to auditd. - Updates elastic/go-libaudit dependency to v2.2.0.
Update the auditd module in Filebeat to apply the same ECS enrichments as Auditbeat / go-libaudit. This is achieved by an autogenerated processor that performs the enrichments defined in go-libaudit's normalizations.yaml.
…ine (#23929) * Upgrade okta to ecs 1.8.0 and move js processor to ingest pipeline * Add description field and set _id properly
Updates zoom pipeline with new ECS 1.8 mappings (multiuser). Fixes a couple of issues with the existing module: - user events: missing mapping for event.category (wrongly mapped to event.type). - chat_channel events: fixed an error in the pipeline that caused some events to be dropped on ingestion.
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
adriansr
added
review
and removed
in progress
adriansr
requested review from
andrewkroh,
P1llus,
andrewstucki and
leehinman
andrewkroh
approved these changes
Feb 16, 2021
28 tasks
adriansr
added a commit
to adriansr/beats
that referenced
this pull request
Feb 17, 2021
Incorporates ECS 1.8 changes from the following PRs: Support host.type field in add_host_metadata processor and Auditbeat's system/host elastic#23513 Winlogbeat elastic#23563 Auditbeat auditd elastic#23594 Journalbeat elastic#23737 Packetbeat elastic#23783 Filebeat: auditd elastic#23723 cisco elastic#23819 cef elastic#23832 crowdstrike falcon elastic#23875 fortinet firewall elastic#23902 microsoft elastic#23897 elasticsearch/audit elastic#24000 Gsuite/Workspace elastic#23709 o365 elastic#23896 zoom elastic#23904 okta elastic#23929 aws/cloudtrail elastic#23911 aws/s3access elastic#23920 azure elastic#23927 juniper/srx elastic#23936 panw elastic#23931 sophos/xg elastic#23967 system/auth elastic#23961 mysqlenterprise elastic#23978 zeek elastic#23847 Make all Beats and modules report ECS 1.8.0 elastic#23992 Closes elastic#23118 Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com> (cherry picked from commit 048c3cc)
adriansr
added a commit
that referenced
this pull request
Feb 17, 2021
Incorporates ECS 1.8 changes from the following PRs: Support host.type field in add_host_metadata processor and Auditbeat's system/host #23513 Winlogbeat #23563 Auditbeat auditd #23594 Journalbeat #23737 Packetbeat #23783 Filebeat: auditd #23723 cisco #23819 cef #23832 crowdstrike falcon #23875 fortinet firewall #23902 microsoft #23897 elasticsearch/audit #24000 Gsuite/Workspace #23709 o365 #23896 zoom #23904 okta #23929 aws/cloudtrail #23911 aws/s3access #23920 azure #23927 juniper/srx #23936 panw #23931 sophos/xg #23967 system/auth #23961 mysqlenterprise #23978 zeek #23847 Make all Beats and modules report ECS 1.8.0 #23992 Closes #23118 Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com> (cherry picked from commit 048c3cc)
v1v
added a commit
to v1v/beats
that referenced
this pull request
Feb 17, 2021
…-arm * upstream/master: [CI] install docker-compose with retry (elastic#24069) Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052) updating manifest files for filebeat threatintel module (elastic#24074) Add Zeek Signatures (elastic#23772) Update Beats to ECS 1.8.0 (elastic#23465) Support running Docker logging plugin on ARM64 (elastic#24034) Fix ec2 metricset fields.yml and add integration test (elastic#23726) Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060) [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773) [Elastic Agent] Enroll with Fleet Server (elastic#23865) [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944) [Ingest Management] Fix reloading of log level for services (elastic#24055) Add Agent standalone k8s manifest (elastic#23679)
v1v
added a commit
to v1v/beats
that referenced
this pull request
Feb 17, 2021
…dows-7 * upstream/master: (332 commits) Use ECS v1.8.0 (elastic#24086) Add support for postgresql csv logs (elastic#23334) [Heartbeat] Refactor config system (elastic#23467) [CI] install docker-compose with retry (elastic#24069) Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052) updating manifest files for filebeat threatintel module (elastic#24074) Add Zeek Signatures (elastic#23772) Update Beats to ECS 1.8.0 (elastic#23465) Support running Docker logging plugin on ARM64 (elastic#24034) Fix ec2 metricset fields.yml and add integration test (elastic#23726) Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060) [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773) [Elastic Agent] Enroll with Fleet Server (elastic#23865) [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944) [Ingest Management] Fix reloading of log level for services (elastic#24055) Add Agent standalone k8s manifest (elastic#23679) [Metricbeat][Kubernetes] Extend state_node with more conditions (elastic#23905) [CI] googleStorageUploadExt step (elastic#24048) Check fields are documented for aws metricsets (elastic#23887) Update go-concert to 0.1.0 (elastic#23770) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Incorporates ECS 1.8 changes from the following PRs:
Update with final 1.8.0 ECS release (code & fields) (pending)
Support
host.typefield inadd_host_metadataprocessor and Auditbeat'ssystem/hostAdd os.type field from ECS 1.8 #23513Winlogbeat [ECS] Winlogbeat ecs 1.8 changes #23563
Auditbeat auditd Update Auditbeat auditd module to ECS 1.8 #23594
Journalbeat [Journalbeat][ecs] Journalbeat ecs 1.8 #23737
Packetbeat [ECS] Packetbeat ecs 1.8 #23783
Filebeat:
Make all Beats and modules report ECS 1.8.0 Update all Beats to report ECS version 1.8.0 #23992
Closes [ECS] Upgrade modules to 1.8 #23118