Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 +130,10 @@
* [Logfmt](pipeline/parsers/logfmt.md)
* [Decoders](pipeline/parsers/decoders.md)
* [Processors](pipeline/processors/README.md)
* [Labels](pipeline/processors/labels.md)
* [Content Modifier](pipeline/processors/content-modifier.md)
* [Labels](pipeline/processors/labels.md)
* [Metrics Selector](pipeline/processors/metrics-selector.md)
* [OpenTelemetry Envelope](pipeline/processors/opentelemetry-envelope.md)
* [SQL](pipeline/processors/sql.md)
* [Filters](pipeline/filters/README.md)
* [AWS Metadata](pipeline/filters/aws-metadata.md)
Expand Down
165 changes: 165 additions & 0 deletions pipeline/processors/opentelemetry-envelope.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# OpenTelemetry Envelope

The _OpenTelemetry Envelope` processor is used to transform your data to be compatible with the OpenTelemetry Log schema. If your data was __not__ generated by [OpenTelemetry input](../inputs/opentelemetry.md) and your backend or destination for your logs expects to be in an OpenTelemetry schema.

![](/imgs/processor_opentelemetry_envelope.png)

## Configuration Parameters

The processor does not provide any extra configuration parameter, it can be used directly in your _processors_ Yaml directive.

## Usage Example

In this example, we will use the Dummy input plugin to generate a sample message per second, right after is created the processor `opentelemetry_envelope` is used to transform the data to be compatible with the OpenTelemetry Log schema. The output is sent to the standard output and also to an OpenTelemetry collector which is receiving data in port 4318.


__fluent-bit.yaml__

```yaml
service:
flush: 1
log_level: info

pipeline:
inputs:
- name: dummy
dummy: '{"message": "Hello World"}'

processors:
logs:
- name: opentelemetry_envelope

outputs:
- name : stdout
match: '*'

- name: opentelemetry
match: '*'
host: 127.0.0.1
port: 4318
```

__otel-collector.yaml__

```yaml
receivers:
otlp:
protocols:
http:
endpoint: 127.0.0.1:4318

exporters:
file:
path: out.json
logging:
loglevel: info

service:
telemetry:
logs:
level: debug
pipelines:
logs:
receivers: [otlp]
exporters: [file, logging]
```

You will notice in the standard output of FLuent Bit will print the raw representation of the schema, however, the OpenTelemetry collector will receive the data in the OpenTelemetry Log schema.

Inspecting the output file `out.json` you will see the data in the OpenTelemetry Log schema:


```json
{
"resourceLogs": [
{
"resource": {},
"scopeLogs": [
{
"scope": {},
"logRecords": [
{
"timeUnixNano": "1722904188085758000",
"body": {
"stringValue": "dummy"
},
"traceId": "",
"spanId": ""
}
]
}
]
}
]
}
```

While OpenTelemetry Envelope enrich your logs with the Schema, you might be interested into take a step further and use the [Content Modifier](../processors/content-modifier.md) processor to modify the content of your logs. Here is a quick example that will allow you to add some resource and scope attributes to your logs:

```yaml
service:
flush: 1
log_level: info

pipeline:
inputs:
- name: dummy
dummy: '{"message": "Hello World"}'

processors:
logs:
- name: opentelemetry_envelope

- name: content_modifier
context: otel_resource_attributes
action: upsert
key: service.name
value: my-service

outputs:
- name : stdout
match: '*'

- name: opentelemetry
match: '*'
host: 127.0.0.1
port: 4318
```

The collector JSON output will look like this:

```json
{
"resourceLogs": [
{
"resource": {
"attributes": [
{
"key": "service.name",
"value": {
"stringValue": "my-service"
}
}
]
},
"scopeLogs": [
{
"scope": {},
"logRecords": [
{
"timeUnixNano": "1722904465173450000",
"body": {
"stringValue": "Hello World"
},
"traceId": "",
"spanId": ""
}
]
}
]
}
]
}
```

For more details about further processing, read the [Content Modifier](../processors/content-modifier.md) processor documentation.