diff --git a/pipeline/outputs/elasticsearch.md b/pipeline/outputs/elasticsearch.md index fc3b7857b..90c4e70a8 100644 --- a/pipeline/outputs/elasticsearch.md +++ b/pipeline/outputs/elasticsearch.md @@ -42,8 +42,9 @@ The **es** output plugin, allows to ingest your records into an [Elasticsearch]( | Current\_Time\_Index | Use current time for index generation instead of message record | Off | | Logstash\_Prefix\_Key | When included: the value in the record that belongs to the key will be looked up and over-write the Logstash\_Prefix for index generation. If the key/value is not found in the record then the Logstash\_Prefix option will act as a fallback. Nested keys are not supported \(if desired, you can use the nest filter plugin to remove nesting\) | | | Suppress\_Type\_Name | When enabled, mapping types is removed and `Type` option is ignored. Types are deprecated in APIs in [v7.0](https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html). This options is for v7.0 or later. | Off | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 2 | -> The parameters _index_ and _type_ can be confusing if you are new to Elastic, if you have used a common relational database before, they can be compared to the _database_ and _table_ concepts. Also see [the FAQ below](elasticsearch.md#faq-multiple-types) +> The parameters _index_ and _type_ can be confusing if you are new to Elastic, if you have used a common relational database before, they can be compared to the _database_ and _table_ concepts. Also see [the FAQ below](elasticsearch.md#faq) ### TLS / SSL diff --git a/pipeline/outputs/file.md b/pipeline/outputs/file.md index 4331392bc..2abd0393f 100644 --- a/pipeline/outputs/file.md +++ b/pipeline/outputs/file.md @@ -6,11 +6,13 @@ The **file** output plugin allows to write the data received through the _input_ The plugin supports the following configuration parameters: -| Key | Description | -| :--- | :--- | +| Key | Description | Default | +| :--- | :--- | :--- | | Path | Directory path to store files. If not set, Fluent Bit will write the files on it's own positioned directory. note: this option was added on Fluent Bit v1.4.6 | | File | Set file name to store the records. If not set, the file name will be the _tag_ associated with the records. | | Format | The format of the file content. See also Format section. Default: out\_file. | +| Mkdir | Recursively create output directory if it does not exist. Permissions set to 0755. | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 1 | ## Format diff --git a/pipeline/outputs/forward.md b/pipeline/outputs/forward.md index 5c06400b0..0d0b92b7e 100644 --- a/pipeline/outputs/forward.md +++ b/pipeline/outputs/forward.md @@ -21,6 +21,7 @@ The following parameters are mandatory for either Forward for Secure Forward mod | Send_options | Always send options (with "size"=count of messages) | False | | Require_ack_response | Send "chunk"-option and wait for "ack" response from server. Enables at-least-once and receiving server can control rate of traffic. (Requires Fluentd v0.14.0+ server) | False | | Compress | Set to "gzip" to enable gzip compression. Incompatible with Time_as_Integer=True and tags set dynamically using the [Rewrite Tag](https://app.gitbook.com/s/-LKKSx-3LBTCtaHbg0gl-887967055/pipeline/filters/rewrite-tag.md) filter. (Requires Fluentd v0.14.7+ server) | | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 2 | ## Secure Forward Mode Configuration Parameters diff --git a/pipeline/outputs/http.md b/pipeline/outputs/http.md index c06df7698..51376c8c5 100644 --- a/pipeline/outputs/http.md +++ b/pipeline/outputs/http.md @@ -4,28 +4,29 @@ The **http** output plugin allows to flush your records into a HTTP endpoint. Fo ## Configuration Parameters -| Key | Description | default | -| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | -| host | IP address or hostname of the target HTTP Server | 127.0.0.1 | -| http\_User | Basic Auth Username | | -| http\_Passwd | Basic Auth Password. Requires HTTP\_User to be set | | -| port | TCP port of the target HTTP Server | 80 | -| Proxy | Specify an HTTP Proxy. The expected format of this value is [http://host:port](http://host/:port). Note that _https_ is **not** supported yet. Please consider not setting this and use `HTTP_PROXY` environment variable instead, which supports both http and https. | | -| uri | Specify an optional HTTP URI for the target web server, e.g: /something | / | -| compress | Set payload compression mechanism. Option available is 'gzip' | | -| format | Specify the data format to be used in the HTTP request body, by default it uses _msgpack_. Other supported formats are _json_, _json\_stream_ and _json\_lines_ and _gelf_. | msgpack | -| allow\_duplicated\_headers | Specify if duplicated headers are allowed. If a duplicated header is found, the latest key/value set is preserved. | true | -| log\_response\_payload | Specify if the response paylod should be logged or not. | true | -| header\_tag | Specify an optional HTTP header field for the original message tag. | | -| header | Add a HTTP header key/value pair. Multiple headers can be set. | | -| json\_date\_key | Specify the name of the time key in the output record. To disable the time key just set the value to `false`. | date | -| json\_date\_format | Specify the format of the date. Supported formats are _double_, _epoch_ and _iso8601_ (eg: _2018-05-30T09:39:52.000681Z_) | double | -| gelf\_timestamp\_key | Specify the key to use for `timestamp` in _gelf_ format | | -| gelf\_host\_key | Specify the key to use for the `host` in _gelf_ format | | -| gelf\_short\_message\_key | Specify the key to use as the `short` message in _gelf_ format | | -| gelf\_full\_message\_key | Specify the key to use for the `full` message in _gelf_ format | | -| gelf\_level\_key | Specify the key to use for the `level` in _gelf_ format | | -| successful\_response\_code | Specify what a successful HTTP response code is in case you need to retry for other HTTP codes (E.g. 204 where) | | +| Key | Description | default | +| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | +| host | IP address or hostname of the target HTTP Server | 127.0.0.1 | +| http\_User | Basic Auth Username | | +| http\_Passwd | Basic Auth Password. Requires HTTP\_User to be set | | +| port | TCP port of the target HTTP Server | 80 | +| Proxy | Specify an HTTP Proxy. The expected format of this value is `http://HOST:PORT`. Note that HTTPS is **not** currently supported. It is recommended not to set this and to configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | | +| uri | Specify an optional HTTP URI for the target web server, e.g: /something | / | +| compress | Set payload compression mechanism. Option available is 'gzip' | | +| format | Specify the data format to be used in the HTTP request body, by default it uses _msgpack_. Other supported formats are _json_, _json\_stream_ and _json\_lines_ and _gelf_. | msgpack | +| allow\_duplicated\_headers | Specify if duplicated headers are allowed. If a duplicated header is found, the latest key/value set is preserved. | true | +| log\_response\_payload | Specify if the response paylod should be logged or not. | true | +| header\_tag | Specify an optional HTTP header field for the original message tag. | | +| header | Add a HTTP header key/value pair. Multiple headers can be set. | | +| json\_date\_key | Specify the name of the time key in the output record. To disable the time key just set the value to `false`. | date | +| json\_date\_format | Specify the format of the date. Supported formats are _double_, _epoch_ and _iso8601_ (eg: _2018-05-30T09:39:52.000681Z_) | double | +| gelf\_timestamp\_key | Specify the key to use for `timestamp` in _gelf_ format | | +| gelf\_host\_key | Specify the key to use for the `host` in _gelf_ format | | +| gelf\_short\_message\_key | Specify the key to use as the `short` message in _gelf_ format | | +| gelf\_full\_message\_key | Specify the key to use for the `full` message in _gelf_ format | | +| gelf\_level\_key | Specify the key to use for the `level` in _gelf_ format | | +| successful\_response\_code | Specify what a successful HTTP response code is in case you need to retry for other HTTP codes (E.g. 204 where) | | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 2 | ### TLS / SSL diff --git a/pipeline/outputs/prometheus-remote-write.md b/pipeline/outputs/prometheus-remote-write.md index f978c8172..df45dc009 100644 --- a/pipeline/outputs/prometheus-remote-write.md +++ b/pipeline/outputs/prometheus-remote-write.md @@ -18,7 +18,8 @@ Important Note: The prometheus exporter only works with metric plugins, such as | uri | Specify an optional HTTP URI for the target web server, e.g: /something | / | | header | Add a HTTP header key/value pair. Multiple headers can be set. | | | log_response_payload | Log the response payload within the Fluent Bit log | false | -| add_label | This allows you to add custom labels to all metrics exposed through the prometheus exporter. You may have multiple of these fields | | +| add_label | This allows you to add custom labels to all metrics exposed through the prometheus exporter. You may have multiple of these fields | | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 2 | ## Getting Started diff --git a/pipeline/outputs/splunk.md b/pipeline/outputs/splunk.md index 3eadae290..20ac1057b 100644 --- a/pipeline/outputs/splunk.md +++ b/pipeline/outputs/splunk.md @@ -12,17 +12,17 @@ To get more details about how to setup the HEC in Splunk please refer to the fol Connectivity, transport and authentication configuration properties: -| Key | Description | default | -| ------------------------- | ----------------------------------------------------------------------------------------------------------------------- | --------- | -| host | IP address or hostname of the target Splunk service. | 127.0.0.1 | -| port | TCP port of the target Splunk service. | 8088 | -| splunk\_token | Specify the Authentication Token for the HTTP Event Collector interface. | | -| http\_user | Optional username for Basic Authentication on HEC | | -| http\_passwd | Password for user defined in HTTP\_User | | -| http\_debug\_bad\_request | Added in 1.8.12, if Splunk returns a bad format error allows you to print out the message as part of the log | off | -| http\_buffer\_size | Buffer size used to receive Splunk HTTP responses. In 1.8.12 this size is auto-adjustable depending on the payload size | 2M | -| compress | Set payload compression mechanism. The only available option is `gzip`. | | -| channel | Specify X-Splunk-Request-Channel Header for the HTTP Event Collector interface. | | +| Key | Description | default | +| :--- | :--- | :--- | +| host | IP address or hostname of the target Splunk service. | 127.0.0.1 | +| port | TCP port of the target Splunk service. | 8088 | +| splunk\_token | Specify the Authentication Token for the HTTP Event Collector interface. | | +| http\_user | Optional username for Basic Authentication on HEC | | +| http\_passwd | Password for user defined in HTTP\_User | | +| http\_buffer\_size | Buffer size used to receive Splunk HTTP responses | 2M | +| compress | Set payload compression mechanism. The only available option is `gzip`. | | +| channel | Specify X-Splunk-Request-Channel Header for the HTTP Event Collector interface. | | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 2 | Content and Splunk metadata (fields) handling configuration properties: diff --git a/pipeline/outputs/stackdriver.md b/pipeline/outputs/stackdriver.md index 74a808c86..d1643b5af 100644 --- a/pipeline/outputs/stackdriver.md +++ b/pipeline/outputs/stackdriver.md @@ -29,6 +29,7 @@ Before to get started with the plugin configuration, make sure to obtain the pro | tag\_prefix | Set the tag\_prefix used to validate the tag of logs with k8s resource type. Without this option, the tag of the log must be in format of k8s\_container\(pod/node\).\* in order to use the k8s\_container resource type. Now the tag prefix is configurable by this option \(note the ending dot\). | k8s\_container., k8s\_pod., k8s\_node. | | severity\_key | Specify the name of the key from the original record that contains the severity information. | | | autoformat\_stackdriver\_trace | Rewrite the _trace_ field to include the projectID and format it for use with Cloud Trace. When this flag is enabled, the user can get the correct result by printing only the traceID (usually 32 characters). | false | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 2 | ### Configuration File diff --git a/pipeline/outputs/standard-output.md b/pipeline/outputs/standard-output.md index 3dc4f7bb2..94b1e111d 100644 --- a/pipeline/outputs/standard-output.md +++ b/pipeline/outputs/standard-output.md @@ -9,6 +9,7 @@ The **stdout** output plugin allows to print to the standard output the data rec | Format | Specify the data format to be printed. Supported formats are _msgpack_ _json_, _json\_lines_ and _json\_stream_. | msgpack | | json\_date\_key | Specify the name of the time key in the output record. To disable the time key just set the value to `false`. | date | | json\_date\_format | Specify the format of the date. Supported formats are _double_, _epoch_ and _iso8601_ \(eg: _2018-05-30T09:39:52.000681Z_\) | double | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 1 | ### Command Line diff --git a/pipeline/outputs/tcp-and-tls.md b/pipeline/outputs/tcp-and-tls.md index e8d829fdb..33a644e3c 100644 --- a/pipeline/outputs/tcp-and-tls.md +++ b/pipeline/outputs/tcp-and-tls.md @@ -11,6 +11,7 @@ The **tcp** output plugin allows to send records to a remote TCP server. The pay | Format | Specify the data format to be printed. Supported formats are _msgpack_ _json_, _json\_lines_ and _json\_stream_. | msgpack | | json\_date\_key | Specify the name of the time key in the output record. To disable the time key just set the value to `false`. | date | | json\_date\_format | Specify the format of the date. Supported formats are _double_, _epoch_ and _iso8601_ \(eg: _2018-05-30T09:39:52.000681Z_\) | double | +| Workers | Enables dedicated thread(s) for this output. Default value is set since version 1.8.12. For previous versions is 0. | 2 | ## TLS Configuration Parameters