Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Time_Key is dropped even when not parsed successfully in json parser #7322

Closed
braydonk opened this issue May 4, 2023 · 0 comments · Fixed by #7323
Closed

Time_Key is dropped even when not parsed successfully in json parser #7322

braydonk opened this issue May 4, 2023 · 0 comments · Fixed by #7323

Comments

@braydonk
Copy link
Contributor

braydonk commented May 4, 2023

Bug Report

Describe the bug
In the parser configuration docs: https://github.com/fluent/fluent-bit-docs/blob/4f06d31577aa476cf888d2829f94d84e6d1c323d/pipeline/parsers/configuring-parser.md#configuration-parameters

The Time_Keep field documentation says the following:

By default when a time key is recognized and parsed, the parser will drop the original time field.

This implies to me that if the time value fails to parse, the key should not be dropped. This is not the case with the json parser. If a time value fails to parse, the Time_Key is dropped regardless.

To Reproduce

  • Example log message if applicable:
{"time":"nonsense"}
  • Steps to reproduce the problem:
  1. Using the following parser:
[PARSER]
    Format      json
    Name        parse_json
    Time_Format %m-%dT%H:%M:%S.%L%Z
    Time_Key    time

Configured on a pipeline however you'd like, it's not really important. As long as it reads the example log line above and has a valid output you'll be able to see the result.

Expected behavior
time should still be present because it will fail to parse with the given Time_Format.

Your Environment

  • Version used: Fluent Bit 2.0.12 and master branch

Additional context
It is possible that this is intended behaviour and that the docs are simply misleading and need an update, however I think it makes sense that the Time_Key would be kept if parsing fails.

@braydonk braydonk changed the title Time_Key is dropped even when not parsed successfully Time_Key is dropped even when not parsed successfully in json parser May 4, 2023
@braydonk braydonk mentioned this issue May 5, 2023
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant