Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

processor_labels: Add a documentation for processor_labels #1371

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@
* [Logfmt](pipeline/parsers/logfmt.md)
* [Decoders](pipeline/parsers/decoders.md)
* [Processors](pipeline/processors/README.md)
* [Labels](pipeline/processors/labels.md)
* [Content Modifier](pipeline/processors/content-modifier.md)
* [Metrics Selector](pipeline/processors/metrics-selector.md)
* [SQL](pipeline/processors/sql.md)
Expand Down
1 change: 1 addition & 0 deletions installation/sources/build-and-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,3 +220,4 @@ The following table describes the processors available on this version:
| option | description | default |
| :--- | :--- | :--- |
| [FLB\_PROCESSOR\_METRICS\_SELECTOR](../../pipeline/processors/metrics-selector.md) | Enable metrics selector processor | On |
| [FLB\_PROCESSOR\_LABELS](../../pipeline/processors/labels.md) | Enable metrics label manipulation processor | On |
104 changes: 104 additions & 0 deletions pipeline/processors/labels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Labels

The **labels** processor allows you to manipulate the labels of Metrics.

Similar to the functionality exposed by filters, this processor presents a enriching/modifying mechanism to perform such operations for labels manipulation.
The most significant difference is that processors perform better than filters, and when chaining them, there are no encoding/decoding performance penalties.

Note that processors and this specific component can only be enabled using the new YAML configuration format. Classic mode configuration format doesn't support processors.
patrick-stephens marked this conversation as resolved.
Show resolved Hide resolved

## Configuration Parameters

| Key | Description |
| :---------- | :--- |
| update | Update an existing key with a value into metrics. The key-value pair is required. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if it does not exist? Failure?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, just dismissed the operation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify that in the docs? Some things seem to error and some ignore so useful to know.

| insert | Insert a new key with a value into metrics. The key-value pair is required. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if it exists?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just ignored in that case on fluent/fluent-bit#8824 patch.

| upsert | Upsert a specific key with a value, the `upsert` operation will try to update the value of the key. If the key does not exist, the key will be created. The key-value pair is required. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like the option to always use :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but it's both of effective whether existing the key or not. :)

| delete | Delete a key from the labels of metrics. The key-value pair is required. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if it does not exist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just ignored and dismissed in that case.

| hash | Replace the key value with a hash generated by the SHA-256 algorithm from the specified label name, the binary value generated is finally set as an hex string representation. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weirdly specific option - can we change the hash algorithm and other stuff in the future?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. But I suppose that we shouldn't do this. Because there is no reason to use weaker hash algorithms such as SHA-1 or MD5.



#### Update example

Change the value of the `name` to `fluentbit`:

```yaml
pipeline:
inputs:
- name: fluentbit_metrics
processors:
metrics:
- name: labels
update: name fluentbit
outputs:
- name : stdout
match: '*'
```

#### Insert example

The following example appends the key `agent` with the value `fluentbit` as the label of metrics.

```yaml
pipeline:
inputs:
- name: fluentbit_metrics
processors:
metrics:
- name: labels
insert: agent fluentbit
outputs:
- name : stdout
match: '*'
```

#### Upsert example

Upsert the value of `name` and insert `fluentbit`:

```yaml
pipeline:
inputs:
- name: fluentbit_metrics
processors:
metrics:
- name: labels
upsert: name fluentbit
outputs:
- name : stdout
match: '*'
```

#### Delete example

Delete containing `name` key from metrics:

```yaml
pipeline:
inputs:
- name: fluentbit_metrics
processors:
metrics:
- name: labels
delete: name
outputs:
- name : stdout
match: '*'
```

#### Hash example

Apply the SHA-1 algorithm for the value of the key `hostname`:

```yaml
pipeline:
inputs:
- name: fluentbit_metrics
processors:
metrics:
- name: labels
hash: hostname
outputs:
- name : stdout
match: '*'
```