Grep Filter not working as expected #4174
Description
Bug Report
Describe the bug
We are using Grep Filter as below to filter out log messages based on a string.
[FILTER]
Name grep
Match kube.*
Exclude $log['contextMap']['operation'] hello
To Reproduce
- Example log message if applicable:
{"log":"{\"gs\":\"test\"},\"contextMap\":{\"operation\":\"world\"}\r\n"}
{"log":"{\"gs\":\"test\"},\"contextMap\":{\"operation\":\"hello\"}\r\n"}
{"log":"{\"gs\":\"test\"},\"contextMap\":{\"operation\":\"world123\"}\r\n"}
- Steps to reproduce the problem:
Start fluent bit
Add the filter stated above
Create a TAIL input plugin to read from a file
Go to that file and paste the above log message
Expected behavior
Grep should filter out the 2nd message. However we see all the 3 messages flowing to the output
Your Environment
- Version used: latest
- Configuration:
[INPUT]
Name tail
Tag kube.*
Path /var/log/containers/gs*.log
Parser docker
DB /var/log/tail-containers-state.db
DB.Sync Normal
Mem_Buf_Limit 5MB
Skip_Long_Lines On
Refresh_Interval 10
Rotate_Wait 60
Docker_Mode On
[FILTER]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Kube_Tag_Prefix kube.var.log.conatiners.
Merge_Log On
Keep_Log Off
K8S-Logging.Parser On
K8S-Logging.Exclude Off
Labels On
Annotations On
[FILTER]
Name grep
Match kube.*
Exclude $log['contextMap']['operation'] hello
[OUTPUT]
Name cloudwatch
Match kube.*
region ${AWS_REGION}
log_group_name eks-logs-${ENVIRONMENT}
log_stream_prefix from-eks-fluentbit-
auto_create_group true
- Environment name and version EKS
- Operating System and version: Amazon linux EC2