Skip to content

Commit 43c49f6

Browse files
author
Pat
authored
filter_lua: add example of variable processing (#670)
Taken from Slack example by user: https://fluent-all.slack.com/archives/C0CTQGHKJ/p1641927953057600?thread_ts=1641863724.045000&cid=C0CTQGHKJ Signed-off-by: Patrick Stephens <pat@calyptia.com>
1 parent efeda6a commit 43c49f6

File tree

1 file changed

+46
-0
lines changed

1 file changed

+46
-0
lines changed

pipeline/filters/lua.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,52 @@ For functional examples of this interface, please refer to the code samples prov
9595

9696
[https://github.com/fluent/fluent-bit/tree/master/scripts](https://github.com/fluent/fluent-bit/tree/master/scripts)
9797

98+
#### Environment variable processing
99+
100+
As an example that combines a bit of LUA processing with the [Kubernetes filter](./kubernetes.md) that demonstrates using environment variables with LUA regex and substitutions.
101+
102+
Kubernetes pods generally have various environment variables set by the infrastructure automatically which may contain useful information.
103+
104+
In this example, we want to extract part of the Kubernetes cluster API name.
105+
106+
The environment variable is set like so:
107+
`KUBERNETES_SERVICE_HOST: api.sandboxbsh-a.project.domain.com`
108+
109+
We want to extract the `sandboxbsh` name and add it to our record as a special key.
110+
111+
```
112+
[FILTER]
113+
Name lua
114+
Alias filter-iots-lua
115+
Match iots_thread.*
116+
Script filters.lua
117+
Call set_landscape_deployment
118+
119+
filters.lua: |
120+
-- Use a Lua function to create some additional entries based
121+
-- on substrings from the kubernetes properties.
122+
function set_landscape_deployment(tag, timestamp, record)
123+
local landscape = os.getenv("KUBERNETES_SERVICE_HOST")
124+
if landscape then
125+
-- Strip the landscape name from this field, KUBERNETES_SERVICE_HOST
126+
-- Should be of this format
127+
-- api.sandboxbsh-a.project.domain.com
128+
-- Take off the leading "api."
129+
-- sandboxbsh-a.project.domain.com
130+
--print("landscape1:" .. landscape)
131+
landscape = landscape:gsub("^[^.]+.", "")
132+
--print("landscape2:" .. landscape)
133+
-- Take off everything including and after the - in the cluster name
134+
-- sandboxbsh
135+
landscape = landscape:gsub("-.*$", "")
136+
-- print("landscape3:" .. landscape)
137+
record["iot_landscape"] = landscape
138+
end
139+
-- 2 - replace existing record with this update
140+
return 2, timestamp, record
141+
end
142+
```
143+
98144
### Number Type
99145

100146
+Lua treats number as double. It means an integer field (e.g. IDs, log levels) will be converted double. To avoid type conversion, The `type_int_key` property is available.

0 commit comments

Comments
 (0)