Skip to content

out_es: Add documentation for index record accessor #1163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

out_es: Add documentation for index record accessor #1163

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e96f4d1
pipeline: outputs: elasticsearch: Add documentation for target_index
Wiston999 Jul 18, 2023
ecc3f33
outputs: pipeline: elasticsearch: Update Index description to reflect…
Wiston999 Aug 17, 2023
9b95d8a
Merge branch 'master' into out-es-target-index-key
esmerel Oct 28, 2025
cd4e53a
Update elasticsearch.md
esmerel Oct 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions pipeline/outputs/elasticsearch.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
| `HTTP_API_Key` | API key for authenticating with Elasticsearch. Must be `base64` encoded. If `HTTP_User` or `Cloud_Auth` are defined, this parameter is ignored. | _none_ |
| `Index` | Index name | `fluent-bit` |
| `Type` | Type name | `_doc` |
| `Target_index` | When included: destination index will be rendered using this record accessor syntax. If any field in the record accessor expression is not found in the record, the value of `Index` setting is used. | _none_ |

Check warning on line 35 in pipeline/outputs/elasticsearch.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Contractions] Feel free to use 'isn't' instead of 'is not'.

Raw Output:
{"message": "[FluentBit.Contractions] Feel free to use 'isn't' instead of 'is not'.", "location": {"path": "pipeline/outputs/elasticsearch.md", "range": {"start": {"line": 35, "column": 152}}}, "severity": "INFO"}
Copy link
Contributor

@patrick-stephens patrick-stephens Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems different to the equivalent approach taken by the Opensearch output: https://docs.fluentbit.io/manual/data-pipeline/outputs/opensearch

I would expect both to use the same value and that's probably a comment on the implementation side.

| `Logstash_Format` | Enable Logstash format compatibility. This option takes a Boolean value: `True/False`, `On/Off` | `Off` |
| `Logstash_Prefix` | When `Logstash_Format` is enabled, the Index name is composed using a prefix and the date, e.g: If `Logstash_Prefix` is equal to `mydata` your index will become `mydata-YYYY.MM.DD`. The last string appended belongs to the date when the data is being generated. | `logstash` |
| `Logstash_Prefix_Key` | When included: the value of the key in the record will be evaluated as key reference and overrides `Logstash_Prefix` for index generation. If the key/value isn't found in the record then the `Logstash_Prefix` option will act as a fallback. The parameter is expected to be a [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md). | _none_ |
Expand Down Expand Up @@ -434,3 +435,20 @@
{% endtabs %}

For records that don't have the field `kubernetes.namespace_name`, the default prefix `logstash` will be used.

### Target_index

Check warning on line 439 in pipeline/outputs/elasticsearch.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Spelling] Spelling check: 'Target_index'?

Raw Output:
{"message": "[FluentBit.Spelling] Spelling check: 'Target_index'?", "location": {"path": "pipeline/outputs/elasticsearch.md", "range": {"start": {"line": 439, "column": 5}}}, "severity": "INFO"}

The following snippet demonstrates using `destination_index` record value as elasticsearch destination index,

Check warning on line 441 in pipeline/outputs/elasticsearch.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Spelling] Spelling check: 'elasticsearch'?

Raw Output:
{"message": "[FluentBit.Spelling] Spelling check: 'elasticsearch'?", "location": {"path": "pipeline/outputs/elasticsearch.md", "range": {"start": {"line": 441, "column": 78}}}, "severity": "INFO"}
using `fallback` as default index name if `destination_index` value is not present in record.

Check warning on line 442 in pipeline/outputs/elasticsearch.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Contractions] Feel free to use 'isn't' instead of 'is not'.

Raw Output:
{"message": "[FluentBit.Contractions] Feel free to use 'isn't' instead of 'is not'.", "location": {"path": "pipeline/outputs/elasticsearch.md", "range": {"start": {"line": 442, "column": 69}}}, "severity": "INFO"}

```text
[OUTPUT]
Name es
Match *
# ...
Index fallback
Target_index fluent-$destination_index
# ...
```

For records that do not have the field `destination__index`, the value of `Index` (`fallback`) will be used.

Check warning on line 454 in pipeline/outputs/elasticsearch.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Contractions] Feel free to use 'don't' instead of 'do not'.

Raw Output:
{"message": "[FluentBit.Contractions] Feel free to use 'don't' instead of 'do not'.", "location": {"path": "pipeline/outputs/elasticsearch.md", "range": {"start": {"line": 454, "column": 18}}}, "severity": "INFO"}