fluentd stop send log to Elasticsearch for some while

[yanrong1990]

Problem

I would build my own fluentd image ,including fluent-plugin-slastiucsearch ,and deploy it on K8S.
This is my fluentd.conf

@include systemd.conf
@include kubernetes.conf


<source>
  @type forward
  @id input_forward
</source>


<match log.*>
  @type elasticsearch
#  hosts 10.23.XX:9200,10.23.XX:9200,10.23.XX:9200
  hosts "#{ENV['ELASTICSEARCH_HOST']}"
#  port "#{ENV['ELASTICSEARCH_PORT']}"
  ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY']}"
  target_index_key ESIndex
  type_name system_log
  reload_connections false
  reconnect_on_error true
  reload_on_failure true
  <buffer>
    @type memory
    flush_mode interval
    flush_interval 5s
    flush_thread_count 4
    retry_timeout 1m
    retry_max_interval 30
    overflow_action drop_oldest_chunk
  </buffer>
</match>

The image will work well at first. But some minutes later, it will not write log to elasticsearch, and I can confirm that some log is still transfered to fluentd by TCP.

And some minutes later, it will resume without any restart.

And I have read #525 and my image will not print failed log like the issue above.

So Is there any wrong with my fluentd.conf? Any suggestion will be helpful.
...

Using Fluentd and ES plugin versions

• OS version
• Bare Metal or within Docker or Kubernetes or others?
• Fluentd v0.12 or v0.14/v1.0
  • paste result of fluentd --version or td-agent --version
    fluentd-1.4.2
• ES plugin 3.x.y/2.x.y or 1.x.y
  • paste boot log of fluentd or td-agent
  • paste result of fluent-gem list, td-agent-gem list or your Gemfile.lock
• ES version (optional)
  ES 6.5.4
• ES template(s) (optional)

[yanrong1990]

I have read other issue and found ,maybe I can set http_backend for a try. This is my Dockerfile to install typhoeus

FROM fluent/fluentd:v1.5-1

# Use root account to use apk
USER root

# below RUN includes plugin as examples elasticsearch is not required
# you may customize including plugins as you wish
RUN apk add --no-cache --update --virtual .build-deps \
        sudo build-base ruby-dev \
 && sudo gem install fluent-plugin-elasticsearch \
 && sudo gem install typhoeus \
 && sudo gem sources --clear-all \
 && apk del .build-deps \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

COPY fluent.conf /fluentd/etc/

USER fluent

and add 'http_backend typhoeus' into fluent.conf.But the image cannot work at all.

So how to install typeoeus for my case?

Thanks.

[cosmo0920]

So how to install typeoeus for my case?

#608