diff --git a/.gitbook.yaml b/.gitbook.yaml index 703fd8f6e..e20fb6dd3 100644 --- a/.gitbook.yaml +++ b/.gitbook.yaml @@ -76,3 +76,6 @@ redirects: output/stdout: ./pipeline/outputs/standard-output.md output/tcp: ./pipeline/outputs/tcp-and-tls.md output/td: ./pipeline/outputs/treasure-data.md + + # Processors + processor/metrics_selector: ./pipeline/processors/metrics_selector.md diff --git a/SUMMARY.md b/SUMMARY.md index a1addc3a0..057fa2e64 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -194,6 +194,8 @@ * [Treasure Data](pipeline/outputs/treasure-data.md) * [Vivo Exporter](pipeline/outputs/vivo-exporter.md) * [WebSocket](pipeline/outputs/websocket.md) +* [Processors](pipeline/processors/README.md) + * [Metrics Selector](pipeline/processors/metrics-selector.md) ## Stream Processing diff --git a/installation/sources/build-and-install.md b/installation/sources/build-and-install.md index 4453ce96e..568130cb4 100644 --- a/installation/sources/build-and-install.md +++ b/installation/sources/build-and-install.md @@ -212,3 +212,11 @@ The _output plugins_ gives the capacity to flush the information to some externa | FLB\_OUT\_TCP | Enable TCP/TLS output plugin | On | | [FLB\_OUT\_TD](../../pipeline/outputs/treasure-data.md) | Enable [Treasure Data](http://www.treasuredata.com) output plugin | On | +### Processor Plugins + +The _processor plugins_ provide the capability to handle the events within the processor pipelines to allow modifying, enrich or drop events. +The following table describes the processors available on this version: + +| option | description | default | +| :--- | :--- | :--- | +| [FLB\_PROCESSOR\_METRICS\_SELECTOR](../../pipeline/processors/metrics-selector.md) | Enable metrics selector processor | On | diff --git a/pipeline/processors/README.md b/pipeline/processors/README.md new file mode 100644 index 000000000..d2a3057f7 --- /dev/null +++ b/pipeline/processors/README.md @@ -0,0 +1,2 @@ +# Processors + diff --git a/pipeline/processors/metrics-selector.md b/pipeline/processors/metrics-selector.md new file mode 100644 index 000000000..962577ad3 --- /dev/null +++ b/pipeline/processors/metrics-selector.md @@ -0,0 +1,61 @@ +# Metrics Selector + +The **metric_selector** processor allows you to select metrics to include or exclude (similar to the `grep` filter for logs). + +## Configuration Parameters + +The native processor plugin supports the following configuration parameters: + +| Key | Description | Default | +| :---------- | :--- | :--- | +| Metric\_Name | Keep metrics in which the metric of name matches with the actual name or the regular expression. | | +| Context | Specify matching context. Currently, metric_name is only supported. | `Metrics_Name` | +| Action | Specify the action for specified metrics. INCLUDE and EXCLUDE are allowed. | | +| Operation\_Type | Specify the operation type of action for metrics payloads. PREFIX and SUBSTRING are allowed. | | + +## Configuration Examples + +Here is a basic configuration example. + +{% tabs %} +{% tab title="fluent-bit.yaml" %} +```yaml +service: + flush: 5 + daemon: off + log_level: info + +pipeline: + inputs: + - name: fluentbit_metrics + tag: fluentbit.metrics + scrape_interval: 10 + + processors: + metrics: + - name: metrics_selector + metric_name: /storage/ + action: include + - name: metrics_selector + metric_name: /fs/ + action: exclude + + - name: labels + delete: name + + + outputs: + - name: stdout + match: '*' +``` +{% endtab %} +{% endtabs %} + + +All processors are only valid with the YAML configuration format. +Processor configuration should be located under the relevant input or output plugin configuration. + +Metric\_Name parameter will translate the strings which is quoted with backslashes `/.../` as Regular expressions. +Without them, users need to specify Operation\_Type whether prefix matching or substring matching. +The default operation is prefix matching. +For example, `/chunks/` will be translated as a regular expression.