Logging operator for Kubernetes based on Fluentd and Fluent-bit.
The Logging operator automates the deployment and configuration of a Kubernetes logging pipeline. The operator deploys and configures a Fluent Bit daemonset on every node to collect container and application logs from the node file system. Fluent Bit queries the Kubernetes API and enriches the logs with metadata about the pods, and transfers both the logs and the metadata to Fluentd. Fluentd receives, filters, and transfer logs to multiple outputs. Your logs will always be transferred on authenticated and encrypted channels.
What is this operator for?
This operator helps you bundle logging information with your applications: you can describe the behavior of your application in its charts, the Logging operator does the rest.
- Namespace isolation
- Native Kubernetes label selectors
- Secure communication (TLS)
- Configuration validation
- Multiple flow support (multiply logs for different transformations)
- Multiple output support (store the same logs in multiple storage: S3, GCS, ES, Loki and more...)
- Multiple logging system support (multiple fluentd, fluent-bit deployment on the same cluster)
You can define
outputs (destinations where you want to send your log messages, for example, Elasticsearch, or and Amazon S3 bucket), and
flows that use filters and selectors to route log messages to the appropriate outputs. You can also define cluster-wide outputs and flows, for example, to use a centralized output that namespaced users cannot modify.
You can configure the Logging operator using the following Custom Resource Descriptions.
- logging - Represents a logging system. Includes
Fluent-bitconfiguration. Specifies the
controlNamespace. Fluentd and Fluent-bit will be deployed in the
- output - Defines an Output for a logging flow. This is a namespaced resource. See also
- flow - Defines a logging flow with
outputs. You can specify
selectorsto filter logs by labels. Outputs can be
clusteroutput. This is a namespaced resource. See also
- clusteroutput - Defines an output without namespace restriction. Only effective in
- clusterflow - Defines a logging flow without namespace restriction.
The detailed CRD documentation can be found here.
Follow these quickstart guides to try out the Logging operator!
Caution: The master branch is under heavy development. Use releases instead of the master branch to get stable software.
You can find the complete documentation of Logging operator v2 here
If you are using the Logging operator in a production environment and require commercial support, contact Banzai Cloud, the company backing the development of the Logging operator. If you are looking for the ultimate observability tool for multi-cluster Kubernetes infrastructures to automate the collection, correlation, and storage of logs and metrics, check out One Eye.
If you find this project useful, help us:
- Support the development of this project and star this repo!
- If you use the Logging operator in a production environment, add yourself to the list of production adopters.
- Help new users with issues they may encounter
- Send a pull request with your new features and bug fixes
For more information, read the developer documentation.
Copyright (c) 2017-2020 Banzai Cloud, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.