Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graylog fails to parse grouped messages #17

Closed
kosborn opened this issue Sep 30, 2019 · 1 comment
Closed

Graylog fails to parse grouped messages #17

kosborn opened this issue Sep 30, 2019 · 1 comment
Labels
🐛 bug Something isn't working 🎃 hacktoberfest https://hacktoberfest.digitalocean.com/details
Milestone

Comments

@kosborn
Copy link
Contributor

kosborn commented Sep 30, 2019

Issue

Using osctrl v0.1.8, using logging = "graylog" in the service configuration (tls.json) for osctrl-tls, using Graylog's HTTP GELF receiver, messages that are group in JSON dictionaries are not accepted.

Steps to reproduce

1 - Set logging to graylog, logging = "graylog" in tls.json
2 - Create graylog HTTP GELF input.
3 - Generate logs

GELF seems to expect individual log statements in JSON {GELF_OBJ}. The osctrl graylog plugin is grouping logs together, and the final document being sent to graylog is [{GELF_OBJ},{GELF_OBJ},{GELF_OBJ},...], which evidently is not cool. This was validated via test GELF messages and tcpdump to view the raw texts to graylog - it dumps them silently otherwise.

To resolve this, I removed the grouping components of the plugin, and send a request for each log.

I haven't used Graylog much, so maybe I've missed something, but the message won't even make it to an index - it gets dropped.

@javuto javuto added 🐛 bug Something isn't working 🎃 hacktoberfest https://hacktoberfest.digitalocean.com/details labels Sep 30, 2019
@javuto javuto added this to the v0.1.9 milestone Sep 30, 2019
@javuto
Copy link
Collaborator

javuto commented Sep 30, 2019

I see how this is a problem. Sending one request per log line, it may impact performance but at least Graylog may be able to ingest data. Do you want to put together a PR with the changes you made? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working 🎃 hacktoberfest https://hacktoberfest.digitalocean.com/details
Projects
None yet
Development

No branches or pull requests

2 participants