Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions installation/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,110 @@ The default configuration of Fluent Bit makes sure of the following:
* The default backend in the configuration is Elasticsearch set by the [Elasticsearch Ouput Plugin](https://docs.fluentbit.io/manual/v/1.0/output/elasticsearch). It uses the Logstash format to ingest the logs. If you need a different Index and Type, please refer to the plugin option and do your own adjustments.
* There is an option called **Retry\_Limit** set to False, that means if Fluent Bit cannot flush the records to Elasticsearch it will re-try indefinitely until it succeed.

## Windows Deployment

Since v1.5.0, Fluent Bit supports deployment to Windows pods.

### Log files overview

When deploying Fluent Bit to Kubernetes, there are three log files that you need to pay attention to.

`C:\k\kubelet.err.log`

* This is the error log file from kubelet daemon running on host.
* You will need to retain this file for future troubleshooting (to debug deployment failures etc.)

`C:\var\log\containers\<pod>_<namespace>_<container>-<docker>.log`

* This is the main log file you need to watch. Configure Fluent Bit to follow this file.
* It is actually a symlink to the Docker log file in `C:\ProgramData\`, with some additional metadata on its file name.

`C:\ProgramData\Docker\containers\<docker>\<docker>.log`

* This is the log file produced by Docker.
* Normally you don't directly read from this file, but you need to make sure that this file is visible from Fluent Bit.

Typically, your deployment yaml contains the following volume configuration.

```yaml
spec:
containers:
- name: fluent-bit
image: my-repo/fluent-bit:1.5.0
volumeMounts:
- mountPath: C:\k
name: k
- mountPath: C:\var\log
name: varlog
- mountPath: C:\ProgramData
name: progdata
volumes:
- name: k
hostPath:
path: C:\k
- name: varlog
hostPath:
path: C:\var\log
- name: progdata
hostPath:
path: C:\ProgramData
```

### Configure Fluent Bit

Assuming the basic volume configuration described above, you can apply the following config to start logging.

```yaml
fluent-bit.conf: |
[SERVICE]
Parsers_File C:\\fluent-bit\\parsers.conf

[INPUT]
Name tail
Tag kube.*
Path C:\\var\\log\\containers\\*.log
Parser docker
DB C:\\fluent-bit\\tail_docker.db
Mem_Buf_Limit 7MB
Refresh_Interval 10

[INPUT]
Name tail
Tag kubelet.err
Path C:\\k\\kubelet.err.log
DB C:\\fluent-bit\\tail_kubelet.db

[FILTER]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc.cluster.local:443

[OUTPUT]
Name stdout
Match *

parsers.conf: |
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
```

### Mitigate unstable network on Windows pods

Windows pods often lack working DNS immediately after boot ([#78479](https://github.com/kubernetes/kubernetes/issues/78479)). To mitigate this issue, `filter_kubernetes` provides a built-in mechanism to wait until the network starts up:

* `DNS_Retries` - Retries N times until the network start working (6)
* `DNS_Wait_Time` - Lookup interval between network status checks (30)

By default, Fluent Bit waits for 3 minutes (30 seconds x 6 times). If it's not enough for you, tweak the configuration as follows.

```
[filter]
Name kubernetes
...
DNS_Retries 10
DNS_Wait_Time 30
```