section <format> is not used in <filter> of grok plugin

[xhanj02]

No matter how i set-up the grok parser plugin using latest stable fluentd 1.10.4, this warning comes up, requiring a section inside the parser. But the following format is not supported by the plugin itself:

<format>
    @type grok
</format>

So you are always going to have this warnig in fluentd 1.x until this is patched.

My grok configuration:

<filter>
  @type parser # https://github.com/tagomoris/fluent-plugin-parser
  key_name log

  format grok  
  custom_pattern_path /fluentd/etc/patterns
  reserve_data true # keep the original attributes - they'll be used by record_transformer later

  # The format is specified in https://plus4u.net/ues/sesm?SessFree=ues%253A%255B29537617%255D%253A%255B44191586301754938%255D%253A
  <grok>
    pattern %{LOGSTORE_RECORD_TYPE:record_type}%{SPACE}%{LOGSTORE_LOG_LEVEL:log_level}%{SPACE}%{GREEDYDATA:message}
  </grok>
  <grok>
    pattern %{LOGSTORE_LOG_LEVEL:log_level}%{SPACE}%{GREEDYDATA:message}
  </grok>
  <grok>
    pattern %{GREEDYDATA:message}
  </grok>
</filter>

And the error:

2020-04-21 13:54:33 +0000 [warn]: section <format> is not used in <filter> of multiline_grok plugin
2020-04-21 13:54:33 +0000 [warn]: section <format> is not used in <filter> of grok plugin

Or does anybody know of a v1 configuration that would eliminate this warning?

[okkez]

Prelase read the documentation https://docs.fluentd.org/filter/parser

[cosmo0920]

Fluentd documentation says that using <parse> directive is needed instead of format parameter:

ref: https://docs.fluentd.org/filter/parser

<filter>
  @type parser # https://github.com/tagomoris/fluent-plugin-parser
  key_name log

  custom_pattern_path /fluentd/etc/patterns
  reserve_data true # keep the original attributes - they'll be used by record_transformer later

  <parse>
    @type grok
    # The format is specified in https://plus4u.net/ues/sesm?SessFree=ues%253A%255B29537617%255D%253A%255B44191586301754938%255D%253A
    <grok>
      pattern %{LOGSTORE_RECORD_TYPE:record_type}%{SPACE}%{LOGSTORE_LOG_LEVEL:log_level}%{SPACE}%{GREEDYDATA:message}
    </grok>
    <grok>
      pattern %{LOGSTORE_LOG_LEVEL:log_level}%{SPACE}%{GREEDYDATA:message}
    </grok>
    <grok>
      pattern %{GREEDYDATA:message}
    </grok>
  </parse>
</filter>