-
Notifications
You must be signed in to change notification settings - Fork 184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(logs): parse JSON logs #2773
Conversation
8197e12
to
e264534
Compare
If so, we should make it configurable or move to separate processor eventually |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This pull request contains invalid labels. Please remove all of the following labels: ['do-not-merge/hold']
e264534
to
edbecd3
Compare
edbecd3
to
bed0f3c
Compare
Depends on #2791 |
bed0f3c
to
ebd10f4
Compare
Parse JSON logs bodies into OTLP data structures. As a result, Sumo renders the JSON correctly. Without this change, JSON log bodies are just escaped strings.
This change is somewhat dangerous, and I'm wondering if we shouldn't make it possible to disable. The problem is that currently, if the JSON parsing returns an error, it'll result in a 500 returned to the collector, and potentially block the whole batch from being processed indefinitely. Discussion is ongoing in otel upstream about how the errors should be handled, but right now there doesn't seem to be a way to just ignore parse errors.
I've switched to logstransform processor, and this actually lets the malformed data through while logging an error. It seems to be buggy in
0.68.0
though, and fine in0.69.0
, so this PR shouldn't be merged until we upgrade to the latter.