Skip to content

Commit a40311a

Browse files
authored
filter_grep: add examples for yaml configuration (#1313)
* filter_grep: add examples for yaml configuration Signed-off-by: Takahiro Yamashita <nokute78@gmail.com> * filter_grep: add parsers_file configuration Signed-off-by: Takahiro Yamashita <nokute78@gmail.com> --------- Signed-off-by: Takahiro Yamashita <nokute78@gmail.com>
1 parent cc37a9c commit a40311a

File tree

1 file changed

+79
-0
lines changed

1 file changed

+79
-0
lines changed

pipeline/filters/grep.md

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,12 @@ $ bin/fluent-bit -i tail -p 'path=lines.txt' -F grep -p 'regex=log aa' -m '*' -o
4747

4848
### Configuration File
4949

50+
{% tabs %}
51+
{% tab title="fluent-bit.conf" %}
5052
```python
53+
[SERVICE]
54+
parsers_file /path/to/parsers.conf
55+
5156
[INPUT]
5257
name tail
5358
path lines.txt
@@ -62,6 +67,28 @@ $ bin/fluent-bit -i tail -p 'path=lines.txt' -F grep -p 'regex=log aa' -m '*' -o
6267
name stdout
6368
match *
6469
```
70+
{% endtab %}
71+
72+
{% tab title="fluent-bit.yaml" %}
73+
```yaml
74+
service:
75+
parsers_file: /path/to/parsers.conf
76+
pipeline:
77+
inputs:
78+
- name: tail
79+
path: lines.txt
80+
parser: json
81+
filters:
82+
- name: grep
83+
match: '*'
84+
regex: log aa
85+
outputs:
86+
- name: stdout
87+
match: '*'
88+
89+
```
90+
{% endtab %}
91+
{% endtabs %}
6592

6693
The filter allows to use multiple rules which are applied in order, you can have many _Regex_ and _Exclude_ entries as required.
6794

@@ -88,12 +115,25 @@ If you want to match or exclude records based on nested values, you can use a [R
88115

89116
if you want to exclude records that match given nested field \(for example `kubernetes.labels.app`\), you can use the following rule:
90117

118+
{% tabs %}
119+
{% tab title="fluent-bit.conf" %}
91120
```python
92121
[FILTER]
93122
Name grep
94123
Match *
95124
Exclude $kubernetes['labels']['app'] myapp
96125
```
126+
{% endtab %}
127+
128+
{% tab title="fluent-bit.yaml" %}
129+
```yaml
130+
filters:
131+
- name: grep
132+
match: '*'
133+
exclude: $kubernetes['labels']['app'] myapp
134+
```
135+
{% endtab %}
136+
{% endtabs %}
97137
98138
### Excluding records missing/invalid fields
99139
@@ -103,6 +143,9 @@ A simple way to do this is just to `exclude` with a regex that matches anything,
103143

104144
Here is an example that checks for a specific valid value for the key as well:
105145

146+
147+
{% tabs %}
148+
{% tab title="fluent-bit.conf" %}
106149
```
107150
# Use Grep to verify the contents of the iot_timestamp value.
108151
# If the iot_timestamp key does not exist, this will fail
@@ -113,6 +156,18 @@ Here is an example that checks for a specific valid value for the key as well:
113156
Match iots_thread.*
114157
Regex iot_timestamp ^\d{4}-\d{2}-\d{2}
115158
```
159+
{% endtab %}
160+
161+
{% tab title="fluent-bit.yaml" %}
162+
```yaml
163+
filters:
164+
- name: grep
165+
alias: filter-iots-grep
166+
match: iots_thread.*
167+
regex: iot_timestamp ^\d{4}-\d{2}-\d{2}
168+
```
169+
{% endtab %}
170+
{% endtabs %}
116171

117172
The specified key `iot_timestamp` must match the expected expression - if it does not or is missing/empty then it will be excluded.
118173

@@ -122,6 +177,9 @@ If you want to set multiple `Regex` or `Exclude`, you can use `Logical_Op` prope
122177

123178
Note: If `Logical_Op` is set, setting both 'Regex' and `Exclude` results in an error.
124179

180+
181+
{% tabs %}
182+
{% tab title="fluent-bit.conf" %}
125183
```python
126184
[INPUT]
127185
Name dummy
@@ -138,6 +196,27 @@ Note: If `Logical_Op` is set, setting both 'Regex' and `Exclude` results in an e
138196
[OUTPUT]
139197
Name stdout
140198
```
199+
{% endtab %}
200+
201+
{% tab title="fluent-bit.yaml" %}
202+
```yaml
203+
pipeline:
204+
inputs:
205+
- name: dummy
206+
dummy: '{"endpoint":"localhost", "value":"something"}'
207+
tag: dummy
208+
filters:
209+
- name: grep
210+
match: '*'
211+
logical_op: or
212+
regex:
213+
- value something
214+
- value error
215+
outputs:
216+
- name: stdout
217+
```
218+
{% endtab %}
219+
{% endtabs %}
141220
142221
Output will be
143222
```

0 commit comments

Comments
 (0)