-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add traceability_id field to alb log parse #862
Conversation
Add traceability_id field to alb log parse
This is currently breaking all of our ALB log parsing. However, is it possible to adjust so that new fields are ignored, not breaking everything? This is apparently being rolled out slowly region to region, so some amount of logs will be broken. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this contribution @skbki !
I think the traceability_id
field needs to be optional since it appears that AWS seems to still be rolling out this change and so there may be users where this field doesn't exist yet (or if they are processing older ALB logs). Your change here makes the field required.
I agree with #862 (comment) that we should also be conservative and just discard unknown fields to future-proof this function, but that could be handled separately from the PR simply adding support for traceability_id
.
Good point. I've managed to make the field optional |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @skbki . This looks good to me!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll just need a changelog entry too.
Hi @skbki, thank you for this contribution! A |
Clippy flagged an issue too ( |
Head branch was pushed to by a user without write access
Corrected issues, should be fine now |
Apparently AWS added new field to ALB log which broke our ingestion pipeline. Only place I found it is AWS Athena guide to parse ALB logs
https://docs.aws.amazon.com/athena/latest/ug/application-load-balancer-logs.html
Closes: #852