Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not allow arbitrary value for timestamp field #2064

Closed
kroepke opened this issue Apr 12, 2016 · 0 comments
Closed

Do not allow arbitrary value for timestamp field #2064

kroepke opened this issue Apr 12, 2016 · 0 comments
Assignees
Labels
Milestone

Comments

@kroepke
Copy link
Member

@kroepke kroepke commented Apr 12, 2016

Problem description

Message inputs that use part of an incoming message as timestamp value or extractors that overwrite timestamp can lead to malformed timestamp fields. This leads to losing messages, because the mapping for timestamp in elasticsearch is very strict.

Steps to reproduce the problem

  1. Send a minimal gelf message such as {"version":"1.1", "short_message":"foo","host":"hostname"}.
  2. Create an extractor copy_input on field message, overwriting the field timestamp.
  3. The message is discarded during indexing.

Environment

  • Graylog Version: all
  • Elasticsearch Version: all
  • MongoDB Version: -
  • Operating System: -
  • Browser version: -
@kroepke kroepke self-assigned this Apr 12, 2016
@kroepke kroepke added the improvement label Apr 12, 2016
@kroepke kroepke added this to the 2.0.0 milestone Apr 12, 2016
kroepke added a commit that referenced this issue Apr 12, 2016
If the timestamp field of a message is incompatible to the ES format we are using, or an unknown data type (non Date, DateTime or String), force the timestamp to be "now".

This prevents bad extractors or inputs from causing these malformed messages to be discarded.

fix #2064
@bernd bernd closed this in #2065 Apr 12, 2016
bernd added a commit that referenced this issue Apr 12, 2016
If the timestamp field of a message is incompatible to the ES format we are using, or an unknown data type (non Date, DateTime or String), force the timestamp to be "now".

This prevents bad extractors or inputs from causing these malformed messages to be discarded.

fix #2064
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

1 participant