New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GELF message with missing fields is dropped #3970

Closed
jalogisch opened this Issue Jul 3, 2017 · 0 comments

Comments

Projects
None yet
2 participants
@jalogisch
Member

jalogisch commented Jul 3, 2017

Expected Behavior

If a message send via GELF (TCP) is missing some specifications or a field that should not only be visible when switching the log to debug.

https://github.com/Graylog2/graylog2-server/blob/2.2.3/graylog2-server/src/main/java/org/graylog2/shared/buffers/processors/DecodingProcessor.java#L179-L186

Current Behavior

GELF Message missing a field - short_message for example the message silently disappear.

Possible Solution

Change the logging level in case the decoding does not work.

Steps to Reproduce (for bugs)

  1. Create GELF TCP Input
  2. send a message with whitespaced short message
echo -e '{"version": "1.1","host":"example.org","short_message":" ","full_message":"Backtrace here\n\nmore stuff","level":1,"_user_id":9001,"_some_info":"foo","_some_env_var":"bar"}\0' | nc -w 1 GRAYLOG_SERVER 12201
  1. To verify send a working message
echo -e '{"version": "1.1","host":"example.org","short_message":"Short","full_message":"Backtrace here\n\nmore stuff","level":1,"_user_id":9001,"_some_info":"foo","_some_env_var":"bar"}\0' | nc -w 1 GRAYLOG_SERVER 12201
  1. Change log level to debug to see that the message with empty short_message is discarded

Your Environment

  • Graylog Version: 2.2.3

joschi added a commit that referenced this issue Jul 3, 2017

Fail fast and loud for invalid GELF messages
Instead of waiting for a later stage and "silently" dropping (logged on DEBUG)
invalid messages, `GelfCodec` now actively checks for the existence and validity
of mandatory GELF message fields (such as "version", "host", "short_message", and
"timestamp", according to the GELF spec).

Refs http://docs.graylog.org/en/2.2/pages/gelf.html
Fixes #3970

@joschi joschi self-assigned this Jul 3, 2017

@joschi joschi added the improvement label Jul 3, 2017

@joschi joschi added this to the 2.3.0 milestone Jul 3, 2017

@wafflebot wafflebot bot added the in progress label Jul 3, 2017

@joschi joschi removed their assignment Jul 3, 2017

@joschi joschi removed the in progress label Jul 3, 2017

@joschi joschi self-assigned this Jul 5, 2017

@joschi joschi added the in progress label Jul 5, 2017

@bernd bernd closed this in #3972 Jul 5, 2017

@wafflebot wafflebot bot removed the in progress label Jul 5, 2017

bernd added a commit that referenced this issue Jul 5, 2017

Fail fast and loud for invalid GELF messages (#3972)
* Fail fast and loud for invalid GELF messages

Instead of waiting for a later stage and "silently" dropping (logged on DEBUG)
invalid messages, `GelfCodec` now actively checks for the existence and validity
of mandatory GELF message fields (such as "version", "host", "short_message", and
"timestamp", according to the GELF spec).

Refs http://docs.graylog.org/en/2.2/pages/gelf.html
Fixes #3970

* Don't check "version" field for backward-compatibility

There are still many GELF client libraries out there using either "1.0" or
no value at all for the GELF "version" field.

* Make GELF validation more lenient

* Add test for minimal GELF messages

* Fix logic for validating "host" message field
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment