-
Notifications
You must be signed in to change notification settings - Fork 419
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AWS] Update AWS cloud.region parsing #4024
Conversation
Relates to elastic/beats#32703 |
/test |
🌐 Coverage report
|
ignore_missing: true | ||
ignore_failure: true | ||
patterns: | ||
- ^%{DATA}s3\.%{DATA:cloud.region}\.%{DATA}$ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@legoguy1000 Should we add a check to see if aws.s3acccess.host_header exists?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isnt that the idea of the ignore_missing??
What does this PR do?
The AWS-S3 input sets the
cloud.region
field to the region the S3 bucket is located. This is not always the same as the region the event originated per https://discuss.elastic.co/t/filebeat-aws-cloudtrail-processor-parses-incorrect-aws-region-from-logs/312150. The current ingest pipelines only use a rename processor which doesn't work if the field already exists. This changes the processor a set to override from the input. Also adds additional parsing for datastreams that don't already havecloud.region
set.Checklist
changelog.yml
file.Author's Checklist
How to test this PR locally
Related issues
Screenshots