Skip to content
This repository has been archived by the owner on Aug 26, 2021. It is now read-only.

GovTechSG/fluentd-elasticsearch

Repository files navigation

fluentd-elasticsearch

Build Status

This repository is an automated build job for a docker image containing fluentd service with a elasticsearch plugin installed and ready to use as an output_plugin.


Plugins Available

The following plugins are available for this image:


Tag Descriptions

latest

Basically fluentd-plugin-elasticsearch

x.y.z

The semver version of the FluentD installed

$DATETIME

Daily build of the image

x.y.z-service-X.Y.Z_service-Y.Z.X

Full tag with all versions of relevant installed software.


Usage

Running

docker run -v ${PWD}/fluent.conf:/fluentd/etc/fluent.conf -p 24224:24224 -it govtechsg/fluentd-elasticsearch:latest

Environment variables

For AWS ElasticSearch

To activate this, see the file at ./config/fluent/default.conf and uncomment the section that indicates it's for an AWS ElasticSearch service in the match *.** codeblock.

Key Description
LOGSTASH_PREFIX The prefix to use for the log entry
AWS_ACCESS_KEY Access key to access the cluster
AWS_SECRET_KEY Secret key to access the cluster
AWS_ES_ENDPOINT The AWS ElasticSearch cluster endpoint URL

For Self-Hosted ElasticSearch

To activate this, see the file at ./config/fluent/default.conf and uncomment the section that indicates it's for a self-hosted ElasticSearch service in the match *.** codeblock.

Key Description
LOGSTASH_PREFIX The prefix to use for the log entry
ELASTIC_HOST Hostname of the ElasticSearch service
ELASTIC_PASSWORD Password of the ElasticSearch user
ELASTIC_USER Username of the ElasticSearch user

Available debug commands in container

Outputs fluentd and plugin versions

version-info


Runbook

Building the image

To build the image:

make build;

Testing the image

You will need container-structure-test to run the tests:

make test;

To run this in a Docker container without installing the container-structure-test, use:

make test.docker;

Versioning the image

To retrieve the versioning details, use:

make version;

Publishing the image

make publish;

CI/CD environment variables

Key Description
DH_USERNAME Username for DockerHub
DH_PASSWORD Password for DockerHub

License

Available as open source under the terms of the MIT License.