Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GELF message with missing fields is dropped #3970

Closed
jalogisch opened this issue Jul 3, 2017 · 0 comments
Closed

GELF message with missing fields is dropped #3970

jalogisch opened this issue Jul 3, 2017 · 0 comments
Assignees
@joschi
Labels
improvement
Milestone
2.3.0

Comments

@jalogisch
Copy link
Contributor

jalogisch commented Jul 3, 2017 •
edited by joschi

Expected Behavior

If a message send via GELF (TCP) is missing some specifications or a field that should not only be visible when switching the log to debug.

https://github.com/Graylog2/graylog2-server/blob/2.2.3/graylog2-server/src/main/java/org/graylog2/shared/buffers/processors/DecodingProcessor.java#L179-L186

Current Behavior

GELF Message missing a field - short_message for example the message silently disappear.

Possible Solution

Change the logging level in case the decoding does not work.

Steps to Reproduce (for bugs)

  1. Create GELF TCP Input
  2. send a message with whitespaced short message
echo -e '{"version": "1.1","host":"example.org","short_message":" ","full_message":"Backtrace here\n\nmore stuff","level":1,"_user_id":9001,"_some_info":"foo","_some_env_var":"bar"}\0' | nc -w 1 GRAYLOG_SERVER 12201
  1. To verify send a working message
echo -e '{"version": "1.1","host":"example.org","short_message":"Short","full_message":"Backtrace here\n\nmore stuff","level":1,"_user_id":9001,"_some_info":"foo","_some_env_var":"bar"}\0' | nc -w 1 GRAYLOG_SERVER 12201
  1. Change log level to debug to see that the message with empty short_message is discarded

Your Environment

  • Graylog Version: 2.2.3
joschi pushed a commit that referenced this issue Jul 3, 2017
Fail fast and loud for invalid GELF messages
1293a6e 
Instead of waiting for a later stage and "silently" dropping (logged on DEBUG)
invalid messages, `GelfCodec` now actively checks for the existence and validity
of mandatory GELF message fields (such as "version", "host", "short_message", and
"timestamp", according to the GELF spec).

Refs http://docs.graylog.org/en/2.2/pages/gelf.html
Fixes #3970
@joschi joschi self-assigned this Jul 3, 2017
@joschi joschi added this to the 2.3.0 milestone Jul 3, 2017
@joschi joschi mentioned this issue Jul 3, 2017
Merged
@ghost ghost added the in progress label Jul 3, 2017
@joschi joschi removed their assignment Jul 3, 2017
@joschi joschi removed the in progress label Jul 3, 2017
@joschi joschi self-assigned this Jul 5, 2017
@ghost ghost removed the in progress label Jul 5, 2017
bernd pushed a commit that referenced this issue Jul 5, 2017
@joschi @bernd
Fail fast and loud for invalid GELF messages (#3972)
090805b 
* Fail fast and loud for invalid GELF messages

Instead of waiting for a later stage and "silently" dropping (logged on DEBUG)
invalid messages, `GelfCodec` now actively checks for the existence and validity
of mandatory GELF message fields (such as "version", "host", "short_message", and
"timestamp", according to the GELF spec).

Refs http://docs.graylog.org/en/2.2/pages/gelf.html
Fixes #3970

* Don't check "version" field for backward-compatibility

There are still many GELF client libraries out there using either "1.0" or
no value at all for the GELF "version" field.

* Make GELF validation more lenient

* Add test for minimal GELF messages

* Fix logic for validating "host" message field
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@joschi joschi

Labels
improvement
Projects
None yet
Milestone
2.3.0
Development

No branches or pull requests
2 participants
@joschi @jalogisch