Fluentd Daemonset for Kubernetes

Supported tags and respective Dockerfile links

See also dockerhub tags page: https://hub.docker.com/r/fluent/fluentd-kubernetes-daemonset/tags

Debian

• v1.4.2-debian-elasticsearch-1.1,v1.4-debian-elasticsearch-1 docker-image/v1.4/debian-elasticsearch/Dockerfile
• v1.4.2-debian-loggly-1.1,v1.4-debian-loggly-1 docker-image/v1.4/debian-loggly/Dockerfile
• v1.4.2-debian-logentries-1.1,v1.4-debian-logentries-1 docker-image/v1.4/debian-logentries/Dockerfile
• v1.4.2-debian-cloudwatch-1.1,v1.4-debian-cloudwatch-1 docker-image/v1.4/debian-cloudwatch/Dockerfile
• v1.4.2-debian-stackdriver-1.1,v1.4-debian-stackdriver-1 docker-image/v1.4/debian-stackdriver/Dockerfile
• v1.4.2-debian-s3-1.1,v1.4-debian-s3-1 docker-image/v1.4/debian-s3/Dockerfile
• v1.4.2-debian-syslog-1.1,v1.4-debian-syslog-1 docker-image/v1.4/debian-syslog/Dockerfile
• v1.4.2-debian-forward-1.1,v1.4-debian-forward-1 docker-image/v1.4/debian-forward/Dockerfile
• v1.4.2-debian-gcs-1.1,v1.4-debian-gcs-1 docker-image/v1.4/debian-gcs/Dockerfile
• v1.4.2-debian-graylog-1.1,v1.4-debian-graylog-1 docker-image/v1.4/debian-graylog/Dockerfile
• v1.4.2-debian-papertrail-1.1,v1.4-debian-papertrail-1 docker-image/v1.4/debian-papertrail/Dockerfile
• v1.4.2-debian-logzio-1.1,v1.4-debian-logzio-1 docker-image/v1.4/debian-logzio/Dockerfile
• v1.4.2-debian-kafka-1.1,v1.4-debian-kafka-1 docker-image/v1.4/debian-kafka/Dockerfile
• v1.4.2-debian-kinesis-1.1,v1.4-debian-kinesis-1 docker-image/v1.4/debian-kinesis/Dockerfile
• v1.3.3-debian-elasticsearch-1.8,v1.3-debian-elasticsearch-1 docker-image/v1.3/debian-elasticsearch/Dockerfile
• v1.3.3-debian-loggly-1.3,v1.3-debian-loggly-1 docker-image/v1.3/debian-loggly/Dockerfile
• v1.3.3-debian-logentries-1.3,v1.3-debian-logentries-1 docker-image/v1.3/debian-logentries/Dockerfile
• v1.3.3-debian-cloudwatch-1.4,v1.3-debian-cloudwatch-1 docker-image/v1.3/debian-cloudwatch/Dockerfile
• v1.3.3-debian-stackdriver-1.3,v1.3-debian-stackdriver-1 docker-image/v1.3/debian-stackdriver/Dockerfile
• v1.3.3-debian-s3-1.3,v1.3-debian-s3-1 docker-image/v1.3/debian-s3/Dockerfile
• v1.3.3-debian-syslog-1.4,v1.3-debian-syslog-1 docker-image/v1.3/debian-syslog/Dockerfile
• v1.3.3-debian-forward-1.3,v1.3-debian-forward-1 docker-image/v1.3/debian-forward/Dockerfile
• v1.3.3-debian-gcs-1.3,v1.3-debian-gcs-1 docker-image/v1.3/debian-gcs/Dockerfile
• v1.3.3-debian-graylog-1.4,v1.3-debian-graylog-1 docker-image/v1.3/debian-graylog/Dockerfile
• v1.3.3-debian-papertrail-1.3,v1.3-debian-papertrail-1 docker-image/v1.3/debian-papertrail/Dockerfile
• v1.3.3-debian-logzio-1.3,v1.3-debian-logzio-1 docker-image/v1.3/debian-logzio/Dockerfile
• v1.3.3-debian-kafka-1.4,v1.3-debian-kafka-1 docker-image/v1.3/debian-kafka/Dockerfile
• v1.3.3-debian-kinesis-1.3,v1.3-debian-kinesis-1 docker-image/v1.3/debian-kinesis/Dockerfile
• v0.12-debian-elasticsearch docker-image/v0.12/debian-elasticsearch/Dockerfile
• v0.12-debian-loggly docker-image/v0.12/debian-loggly/Dockerfile
• v0.12-debian-logentries docker-image/v0.12/debian-logentries/Dockerfile
• v0.12-debian-cloudwatch docker-image/v0.12/debian-cloudwatch/Dockerfile
• v0.12-debian-stackdriver docker-image/v0.12/debian-stackdriver/Dockerfile
• v0.12-debian-s3 docker-image/v0.12/debian-s3/Dockerfile
• v0.12-debian-gcs docker-image/v0.12/debian-gcs/Dockerfile
• v0.12-debian-papertrail docker-image/v0.12/debian-papertrail/Dockerfile
• v0.12-debian-syslog docker-image/v0.12/debian-syslog/Dockerfile
• v0.12-debian-graylog docker-image/v0.12/debian-graylog/Dockerfile
• v0.12-debian-logzio docker-image/v0.12/debian-logzio/Dockerfile
• v0.12-debian-kafka docker-image/v0.12/debian-kafka/Dockerfile
• v0.12-debian-splunkhec docker-image/v0.12/debian-splunkhec/Dockerfile
• v0.12-debian-kinesis docker-image/v0.12/debian-kinesis/Dockerfile

Alpine Linux (This is deprecated. Use Debian images instead)

• v0.12-alpine-elasticsearch docker-image/v0.12/alpine-elasticsearch/Dockerfile
• v0.12-alpine-loggly docker-image/v0.12/alpine-loggly/Dockerfile
• v0.12-alpine-logentries docker-image/v0.12/alpine-logentries/Dockerfile
• v0.12-alpine-cloudwatch docker-image/v0.12/alpine-cloudwatch/Dockerfile
• v0.12-alpine-stackdriver docker-image/v0.12/alpine-stackdriver/Dockerfile
• v0.12-alpine-s3 docker-image/v0.12/alpine-s3/Dockerfile
• v0.12-alpine-gcs docker-image/v0.12/alpine-gcs/Dockerfile
• v0.12-alpine-papertrail docker-image/v0.12/alpine-papertrail/Dockerfile
• v0.12-alpine-syslog docker-image/v0.12/alpine-syslog/Dockerfile
• v0.12-alpine-graylog docker-image/v0.12/alpine-graylog/Dockerfile
• v0.12-alpine-logzio docker-image/v0.12/alpine-logzio/Dockerfile
• v0.12-alpine-kafka docker-image/v0.12/alpine-kafka/Dockerfile
• v0.12-alpine-kinesis docker-image/v0.12/alpine-kinesis/Dockerfile
• v0.12-alpine-splunkhec docker-image/v0.12/alpine-splunkhec/Dockerfile

What is Fluentd?

Fluentd is an open source data collector, which lets you unify the data collection and consumption for a better use and understanding of data.

www.fluentd.org

Image versions

The following repository expose images based on Alpine Linux and Debian. For production environments we strongly suggest to use Debian images.

Fluentd versioning is as follows:

Series	Description
v1.x	stable
v0.12	Old stable, no longer updated

Settings

Run as root

This is for v0.12 images.

In Kubernetes and default setting, fluentd needs root permission to read logs in /var/log and write pos_file to /var/log. To avoid permission error, you need to set FLUENT_UID environment variable to 0 in your Kubernetes configuration.

Disable systemd input

If you don't setup systemd in the container, fluentd shows following messages by default configuration.

[warn]: #0 [in_systemd_bootkube] Systemd::JournalError: No such file or directory retrying in 1s
[warn]: #0 [in_systemd_kubelet] Systemd::JournalError: No such file or directory retrying in 1s
[warn]: #0 [in_systemd_docker] Systemd::JournalError: No such file or directory retrying in 1s

You can suppress these messages by setting disable to FLUENTD_SYSTEMD_CONF environment variable in your kubernetes configuration.

Disable prometheus input plugins

By default, latest images launch prometheus plugins to monitor fluentd. You can disable prometheus input plugin by setting disable to FLUENTD_PROMETHEUS_CONF environment variable in your kubernetes configuration.

Disable sed execution on elasticsearch image

By historical reason, elasaticsearch image executes sed command during startup phase when FLUENT_ELASTICSEARCH_USER or FLUENT_ELASTICSEARCH_PASSWORD is specified. This sometimes causes a problem with read only mount. To avoid this problem, set "true" to FLUENT_ELASTICSEARCH_SED_DISABLE environment variable in your kubernetes configuration.

Maintainers

Some images are contributed by users. If you have a problem/question for following images, ask it to contributors.

• cloudwatch : @so0k
• papertrail : @alexouzounis
• kafka : @erhudy
• graylog : @rtnpro
• gcs : @andor-pierdelacabeza
• Amazon Kinesis : @shiftky
• logz.io : @SaMnCo / @jamielennox
• splunkhec: @FutureSharks

References

Kubernetes Logging with Fluentd

Running on OpenShift

This daemonset setting mounts /var/log as service account fluentd so you need to run containers as privileged container. Here is command example:

oc project kube-system
oc create -f https://raw.githubusercontent.com/fluent/fluentd-kubernetes-daemonset/master/fluentd-daemonset-elasticsearch-rbac.yaml
oc adm policy add-scc-to-user privileged -z fluentd
oc patch ds fluentd -p "spec:
  template:
    spec:
      containers:
      - name: fluentd
        securityContext:
          privileged: true"
oc delete pod -l k8s-app = fluentd-logging

This is from nekop's japanese article.

Issues

We can't notice comments in the DockerHub so don't use them for reporting issues or asking question.

If you have any problems with or questions about this image, please contact us through a GitHub issue.

Note: This file is generated from templates/README.md.erb