Skip to content

Commit ac7b50e

Browse files
pipeline: filters: lua: add response code filtering example (#900)
Signed-off-by: Fran Sanjuán <francesc.sanjuan@marfeel.com>
1 parent 7f7abec commit ac7b50e

File tree

1 file changed

+85
-0
lines changed

1 file changed

+85
-0
lines changed

pipeline/filters/lua.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,3 +262,88 @@ end
262262
```
263263

264264
See also [Fluent Bit: PR 811](https://github.com/fluent/fluent-bit/pull/811).
265+
266+
### Response code filtering
267+
268+
In this example, we want to filter istio logs to exclude lines with response codes between 1 and 399.
269+
Istio is configured to write the logs in json format.
270+
271+
#### Lua script
272+
273+
Script `response_code_filter.lua`
274+
275+
```lua
276+
function cb_response_code_filter(tag, timestamp, record)
277+
response_code = record["response_code"]
278+
if (response_code == nil or response_code == '') then
279+
return 0,0,0
280+
elseif (response_code ~= 0 and response_code < 400) then
281+
return -1,0,0
282+
else
283+
return 0,0,0
284+
end
285+
end
286+
```
287+
288+
#### Configuration
289+
290+
Configuration to get istio logs and apply response code filter to them.
291+
292+
```ini
293+
[INPUT]
294+
Name tail
295+
Path /var/log/containers/*_istio-proxy-*.log
296+
multiline.parser docker, cri
297+
Tag istio.*
298+
Mem_Buf_Limit 64MB
299+
Skip_Long_Lines Off
300+
301+
[FILTER]
302+
Name lua
303+
Match istio.*
304+
Script response_code_filter.lua
305+
call cb_response_code_filter
306+
307+
[Output]
308+
Name stdout
309+
Match *
310+
```
311+
312+
#### Input
313+
314+
```json
315+
{
316+
"log": {
317+
"response_code": 200,
318+
"bytes_sent": 111328341,
319+
"authority": "randomservice.randomservice",
320+
"duration": 14493,
321+
"request_id": "2e9d38f8-36a9-40a6-bdb2-47c8eb7d399d",
322+
"upstream_local_address": "10.11.82.178:42738",
323+
"downstream_local_address": "10.10.21.17:80",
324+
"upstream_cluster": "outbound|80||randomservice.svc.cluster.local",
325+
"x_forwarded_for": null,
326+
"route_name": "default",
327+
"upstream_host": "10.11.6.90:80",
328+
"user_agent": "RandomUserAgent",
329+
"response_code_details": "via_upstream",
330+
"downstream_remote_address": "10.11.82.178:51096",
331+
"bytes_received": 1148,
332+
"path": "/?parameter=random",
333+
"response_flags": "-",
334+
"start_time": "2022-07-28T11:16:51.663Z",
335+
"upstream_transport_failure_reason": null,
336+
"method": "POST",
337+
"connection_termination_details": null,
338+
"protocol": "HTTP/1.1",
339+
"requested_server_name": null,
340+
"upstream_service_time": "6161"
341+
},
342+
"stream": "stdout",
343+
"time": "2022-07-28T11:17:06.704109897Z"
344+
}
345+
```
346+
347+
#### Output
348+
349+
In the output only the messages with response code 0 or greater than 399 are shown.

0 commit comments

Comments
 (0)