Skip to content

Commit 317db1c

Browse files
development: wasm-filter-plugins: add new example using wasm-filter (fluent#1177)
* development: wasm-filter-plugins: add new example using wasm-filter with rust Signed-off-by: Kenrique Ortega <53656046+kenriortega@users.noreply.github.com> Signed-off-by: kenriortega <kenriortega@gmail.com> * development: wasm-filter-plugins: change the PR suggestions by @patrick-stephens Signed-off-by: kenriortega <kenriortega@gmail.com> * development: wasm-filter-plugins: change the PR adding suggestions by @patrick-stephens Signed-off-by: kenriortega <kenriortega@gmail.com> * Apply suggestions from code review Signed-off-by: Pat <patrick.j.k.stephens@gmail.com> * development: wasm-filter-plugins: add new example using wasm-filter Signed-off-by: kenriortega <kenriortega@gmail.com> --------- Signed-off-by: Kenrique Ortega <53656046+kenriortega@users.noreply.github.com> Signed-off-by: kenriortega <kenriortega@gmail.com> Signed-off-by: Pat <patrick.j.k.stephens@gmail.com> Co-authored-by: Pat <patrick.j.k.stephens@gmail.com>
1 parent 4d309a7 commit 317db1c

File tree

1 file changed

+38
-0
lines changed

1 file changed

+38
-0
lines changed

development/wasm-filter-plugins.md

+38
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,44 @@ For example, one of the examples of [Rust WASM filter](https://github.com/fluent
128128
[0] dummy.local: [1666270589.270348000, {"lang"=>"Rust", "message"=>"dummy", "original"=>"{"message":"dummy"}", "tag"=>"dummy.local", "time"=>"2022-10-20T12:56:29.270348000 +0000"}]
129129
[0] dummy.local: [1666270590.271107000, {"lang"=>"Rust", "message"=>"dummy", "original"=>"{"message":"dummy"}", "tag"=>"dummy.local", "time"=>"2022-10-20T12:56:30.271107000 +0000"}]
130130
```
131+
Another example of a Rust WASM filter is the [flb_filter_iis](https://github.com/kenriortega/flb_filter_iis) filter.
132+
This filter takes the [Internet Information Services (IIS)](https://learn.microsoft.com/en-us/iis/manage/provisioning-and-managing-iis/configure-logging-in-iis) [w3c logs](https://learn.microsoft.com/en-us/iis/manage/provisioning-and-managing-iis/configure-logging-in-iis#select-w3c-fields-to-log) (with some custom modifications) and transforms the raw string into a standard Fluent Bit JSON structured record.
133+
134+
```text
135+
[INPUT]
136+
Name dummy
137+
Dummy {"log": "2023-08-11 19:56:44 W3SVC1 WIN-PC1 ::1 GET / - 80 ::1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/115.0.0.0+Safari/537.36+Edg/115.0.1901.200 - - localhost 304 142 756 1078 -"}
138+
Tag iis.*
139+
140+
[FILTER]
141+
Name wasm
142+
match iis.*
143+
WASM_Path /plugins/flb_filter_iis_wasm.wasm
144+
Function_Name flb_filter_log_iis_w3c_custom
145+
accessible_paths .
146+
147+
[OUTPUT]
148+
name stdout
149+
match iis.*
150+
```
151+
152+
The incoming raw strings from an IIS log are composed of the following fields:
153+
154+
`date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port c-ip cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-bytes cs-bytes time-taken c-authorization-header`
155+
156+
The output after the filter logic will be:
157+
158+
```text
159+
[0] iis.*: [[1692131925.559486675, {}], {"c_authorization_header"=>"-", "c_ip"=>"::1", "cs_bytes"=>756, "cs_cookie"=>"-", "cs_host"=>"localhost", "cs_method"=>"GET", "cs_referer"=>"-", "cs_uri_query"=>"-", "cs_uri_stem"=>"/", "cs_user_agent"=>"Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/115.0.0.0+Safari/537.36+Edg/115.0.1901.200", "date"=>"2023-08-11 19:56:44", "s_computername"=>"WIN-PC1", "s_ip"=>"::1", "s_port"=>"80", "s_sitename"=>"W3SVC1", "sc_bytes"=>142, "sc_status"=>"304", "source"=>"LogEntryIIS", "tag"=>"iis.*", "time"=>"2023-08-15T20:38:45.559486675 +0000", "time_taken"=>1078}]
160+
```
161+
This filter approach provides us with several powerful advantages inherent to programming languages.
162+
For instance, it:
163+
- Can be extended by adding type conversion to fields such as `sc_bytes, cs_bytes, time_taken`. This is particularly useful when we need to validate our data results.
164+
- Allows for the use of conditions to apply more descriptive filters, for example, "get only all logs that contain status codes above 4xx or 5xx".
165+
- Can be used to define a `allow/deny` list using a data structure array or a file to store predefined IP addresses.
166+
- Makes it possible to call an external resource such as an API or database to enhance our data.
167+
- Allows all methods to be thoroughly tested and shared as a binary bundle or library.
168+
These examples can be applied in our demo and can serve as an ideal starting point to create more complex logic, depending on our requirements.
131169

132170
### Optimize execution of WASM programs
133171

0 commit comments

Comments
 (0)