Flush_interval doesn't work after set up timekey.

[yuchen9459]

Describe the bug
Since I would like to set the index name as the format fluentd.${tag}.%Y%m, I need to set up the timekey for the buffer. However, after setting up the timekey, the flush_interval stops to work. It takes much longer to dump the data to the elastic search. If I remove the timekey setup, the log records can be dumped to elastic search immediately, while the index_name isn't correct.

To Reproduce
Fluentd config file:

<source>
  @type tail
  path /var/log/test.log
  pos_file /var/log/td-agent/test.log.pos
  tag logtest
  read_from_head true
  <parse>
     @type json
   </parse>
</source>
<match logtest>
   @type elasticsearch
   host localhost
   port 9200
   index_name fluentd.${tag}.%Y%m
   <buffer tag, time>
     timekey 5s
     flush_interval 5s
   </buffer>
</match>

Log file example:

{"asctime": "2020-01-07 15:21:36", "levelname": "INFO", "message": "Watcher Stop", "module": "watcher.watcher", "action": "Watcher is stopped"}
{"asctime": "2020-01-07 15:21:37", "levelname": "INFO", "message": "Watcher Start", "module": "watcher.watcher", "action": "Watcher is started"}
{"asctime": "2020-01-07 15:21:38", "levelname": "INFO", "message": "Observing: test/data/receive", "module": "watcher.watcher", "action": "Watcher Schedules"}

Expected behavior
The log records should be dumped into elastic search quickly, rather than takes around 1 min.

Your Environment

• Fluentd-1.7.4
• Operating system: Windows10

[repeatedly]

Set flush_mode interval in <buffer>.
You see following message in fluentd logs.

'flush_interval' is ignored because default 'flush_mode' is not 'interval': 'lazy'

[yuchen9459]

Set flush_mode interval in <buffer>.
You see following message in fluentd logs.

'flush_interval' is ignored because default 'flush_mode' is not 'interval': 'lazy'

Thanks for your response. I removed the timekey attribute from the buffer section and change the flush_mode into immediate. In this case, even though the index was created immediately when I start the fluentd, the new records were not inserted into index immediately when there're new records appended to the log file. Do you know the possible reason for that?