Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backup to bucket feature in filebeat aws-s3 input #30696

Closed
0ccupi3R opened this issue Mar 5, 2022 · 0 comments · Fixed by #33559
Closed

Backup to bucket feature in filebeat aws-s3 input #30696

0ccupi3R opened this issue Mar 5, 2022 · 0 comments · Fixed by #33559
Labels
Team:Cloud-Monitoring Label for the Cloud Monitoring team

Comments

@0ccupi3R
Copy link

0ccupi3R commented Mar 5, 2022

Implementing move(backup) object in filebeat

Usually for the organization marinating multiple AWS accounts, the amount of logs gets very high which takes more time to verify the processed event from the log file.
Can filebeat s3 input have backup_to_bucket feature similar to logstash so it will move the objects after being processed.

Something like below

- module: aws
  cloudtrail:
    enabled: true
    # AWS S3 bucket arn
    var.bucket_arn: 'arn:aws:s3:::<source_bucket_name>'
    var.bucket_list_prefix: 'cloudtrail/AWSLogs/<path>'

    var.backup_bucket_arn: 'arn:aws:s3:::<destination_bucket_name>'
    var.backup_bucket_list_prefix: 'cloudtrail/processed/<path>'
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 5, 2022
@aspacca aspacca added the Team:Cloud-Monitoring Label for the Cloud Monitoring team label Mar 7, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Cloud-Monitoring Label for the Cloud Monitoring team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants