-
Notifications
You must be signed in to change notification settings - Fork 543
Adding UDP output plugin doc. Fixes #2144. #2195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
84265b1
Updated to include new input plugin doc for Windows System Statistics…
eschabell 8b9641d
Updated to include new input plugin doc for Windows System Statistics…
eschabell 2bdfa9a
Lint fix for winstat input plugin doc fix.
eschabell 3144255
Merge branch 'fluent:master' into master
eschabell 6643c35
Merge branch 'fluent:master' into master
eschabell a7bc44a
Merge branch 'fluent:master' into master
eschabell 9f752bb
Adding UDP output plugin doc. Applies to #2144.
eschabell bce57f1
Adding UDP output plugin doc to SUMMARY listing. Applies to #2144.
eschabell 1102bc1
Lint fixes for udp output plugin doc fix.
eschabell File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,205 @@ | ||
| --- | ||
| description: Send logs to a remote UDP server | ||
| --- | ||
|
|
||
| # UDP | ||
|
|
||
| The _UDP_ output plugin lets you send records to a remote UDP server. The payload can be formatted in different ways as required. | ||
|
|
||
| ## Configuration parameters | ||
|
|
||
| This plugin supports the following parameters: | ||
|
|
||
| | Key | Description | Default | | ||
| |:--- |:----------- |:------- | | ||
| | `Host` | Target host where the UDP server is listening. | `127.0.0.1` | | ||
| | `Port` | `UDP` Port of the target service. | `5170` | | ||
| | `Format` | Specify the data format to be printed. Supported formats: `msgpack`, `json`, `json_lines`, `json_stream`. | `json_lines` | | ||
| | `json_date_key` | Specify the name of the time key in the output record. To disable the time key, set the value to `false`. | `date` | | ||
| | `json_date_format` | Specify the format of the date. Supported formats: `double`, `epoch`, `epoch_ms`, `iso8601`, `java_sql_timestamp`. | `double` | | ||
| | `raw_message_key` | Use a raw message key for the message. When set, the plugin sends the value of this key as the raw message instead of formatting it as JSON. | _none_ | | ||
| | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` | | ||
|
|
||
| ## Get started | ||
|
|
||
| To send records to a `UDP` server, you can run the plugin from the command line or through the configuration file. | ||
|
|
||
| ### Command line | ||
|
|
||
| The `UDP` plugin can read the parameters from the command line through the `-p` argument (property) or by setting them directly through the service URI. The URI format is the following: | ||
|
|
||
| ```text | ||
| udp://host:port | ||
| ``` | ||
|
|
||
| Using the format specified, start Fluent Bit through: | ||
|
|
||
| ```shell | ||
| fluent-bit -i cpu -t cpu -o udp://192.168.2.3:5170 -p format=json_lines -v | ||
| ``` | ||
|
|
||
| which is similar to: | ||
|
|
||
| ```shell | ||
| fluent-bit -i cpu -t cpu -o udp -p Host=192.168.2.3 -p Port=5170 \ | ||
| -p Format=json_lines -o stdout -m '*' | ||
| ``` | ||
|
|
||
| ### Configuration file | ||
|
|
||
| In your main configuration file append the following: | ||
|
|
||
| {% tabs %} | ||
| {% tab title="fluent-bit.yaml" %} | ||
|
|
||
| ```yaml | ||
| pipeline: | ||
| inputs: | ||
| - name: cpu | ||
| tag: cpu | ||
|
|
||
| outputs: | ||
| - name: udp | ||
| match: '*' | ||
| host: 192.168.2.3 | ||
| port: 5170 | ||
| format: json_lines | ||
| ``` | ||
|
|
||
| {% endtab %} | ||
| {% tab title="fluent-bit.conf" %} | ||
|
|
||
| ```text | ||
| [INPUT] | ||
| Name cpu | ||
| Tag cpu | ||
|
|
||
| [OUTPUT] | ||
| Name udp | ||
| Match * | ||
| Host 192.168.2.3 | ||
| Port 5170 | ||
| Format json_lines | ||
| ``` | ||
|
|
||
| {% endtab %} | ||
| {% endtabs %} | ||
|
|
||
| ## JSON format examples | ||
|
|
||
| ### JSON lines format | ||
|
|
||
| This example sends CPU metrics in JSON lines format to a `UDP` server: | ||
|
|
||
| {% tabs %} | ||
| {% tab title="fluent-bit.yaml" %} | ||
|
|
||
| ```yaml | ||
| pipeline: | ||
| inputs: | ||
| - name: cpu | ||
| tag: cpu | ||
|
|
||
| outputs: | ||
| - name: udp | ||
| match: '*' | ||
| host: 127.0.0.1 | ||
| port: 5170 | ||
| format: json_lines | ||
| json_date_key: timestamp | ||
| json_date_format: iso8601 | ||
| ``` | ||
|
|
||
| {% endtab %} | ||
| {% tab title="fluent-bit.conf" %} | ||
|
|
||
| ```text | ||
| [INPUT] | ||
| Name cpu | ||
| Tag cpu | ||
|
|
||
| [OUTPUT] | ||
| Name udp | ||
| Match * | ||
| Host 127.0.0.1 | ||
| Port 5170 | ||
| Format json_lines | ||
| Json_date_key timestamp | ||
| Json_date_format iso8601 | ||
| ``` | ||
|
|
||
| {% endtab %} | ||
| {% endtabs %} | ||
|
|
||
| Run the following in a separate terminal, `netcat` will start listening for messages on `UDP` port `5170`: | ||
|
|
||
| ```shell | ||
| nc -u -l 5170 | ||
| ``` | ||
|
|
||
| After Fluent Bit connects, you should see output in JSON format: | ||
|
|
||
| ```text | ||
| {"timestamp":"2024-01-15T10:30:45.123456Z","cpu_p":1.1875,"user_p":0.5625,"system_p":0.625} | ||
| {"timestamp":"2024-01-15T10:30:46.123456Z","cpu_p":2.25,"user_p":2.0,"system_p":0.25} | ||
| ``` | ||
|
|
||
| ### Raw message format | ||
|
|
||
| When `raw_message_key` is set, the plugin sends the value of the specified key as a raw message instead of formatting it as JSON. This is for when you want to send pre-formatted messages: | ||
|
|
||
| {% tabs %} | ||
| {% tab title="fluent-bit.yaml" %} | ||
|
|
||
| ```yaml | ||
| pipeline: | ||
| inputs: | ||
| - name: tail | ||
| path: /var/log/app.log | ||
| tag: app | ||
|
|
||
| outputs: | ||
| - name: udp | ||
| match: '*' | ||
| host: 127.0.0.1 | ||
| port: 5170 | ||
| raw_message_key: message | ||
| ``` | ||
|
|
||
| {% endtab %} | ||
| {% tab title="fluent-bit.conf" %} | ||
|
|
||
| ```text | ||
| [INPUT] | ||
| Name tail | ||
| Path /var/log/app.log | ||
| Tag app | ||
|
|
||
| [OUTPUT] | ||
| Name udp | ||
| Match * | ||
| Host 127.0.0.1 | ||
| Port 5170 | ||
| Raw_message_key message | ||
| ``` | ||
|
|
||
| {% endtab %} | ||
| {% endtabs %} | ||
|
|
||
| ## Message size limitations | ||
|
|
||
| `UDP` has a maximum datagram size of `65535` bytes. If a record exceeds this size, the plugin will send it but log a debug message. For large messages, consider using `TCP` or `HTTP` output plugins instead. | ||
|
|
||
| ## Testing | ||
|
|
||
| To test the `UDP` output plugin, you can use `netcat` to listen for incoming `UDP` messages: | ||
|
|
||
| ```shell | ||
| nc -u -l 5170 | ||
| ``` | ||
|
|
||
| Or use `socat` for more advanced testing: | ||
|
|
||
| ```shell | ||
| socat UDP-RECV:5170 STDOUT | ||
| ``` | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.