fluent-plugin-opentelemetry

Fluentd input/output plugin to forward OpenTelemetry Protocol data.

Installation

RubyGems

gem install fluent-plugin-opentelemetry

Bundler

gem "fluent-plugin-opentelemetry"

And then execute:

$ bundle

Configuration

Input opentelemetry plugin

To receive data, this plugin requires <http> or <grpc> section, or both.

Root section

parameter	type	description	default
tag	string	The tag of the event	required

tag parameter supports ${type} placeholder and it will be expanded with data type.

• log data
  • ${type} will be replaced to logs
• metric data
  • ${type} will be replaced to metrics
• trace data
  • ${type} will be replaced to traces

This can be used to change the output destination for each message type.

Example:

<source>
  @type opentelemetry

  # Expand to opentelemetry.logs, opentelemetry.metrics, opentelemetry.traces according to received data.
  tag opentelemetry.${type}

  <http>
    bind 127.0.0.1
    port 4318
  </http>
</source>

<http> section

This requires to receive data via HTTP/HTTPS.

parameter	type	description	default
bind	string	The address to bind to	0.0.0.0
port	integer	The port to listen to	4318
body_size_limit	size	The size limit of the POSTed element. This value should be larger than the 'chunk_limit_size' in out_opentelemetry plugin	32M (32MiB)
decompression_size_limit	size	The size limit of the decompressed element	256M (256MiB)

<grpc> section

This requires to receive data via gRPC. It needs to install grpc gem manually to use this feature.

Warning

Now, gRPC feature status is experimental.

parameter	type	description	default
bind	string	The address to bind to	0.0.0.0
port	integer	The port to listen to	4317

• <transport> section

Refer Config: Transport Section

When the <grpc> section is configured, <transport tls> applies to the gRPC listener as well. gRPC uses ca_path, cert_path, private_key_path, private_key_passphrase and client_cert_auth, and the other TLS parameters have no effect on the gRPC listener. client_cert_auth requires ca_path, because gRPC has no default trust store for client certificates.

Warning

This changes the behavior of an existing configuration that has both <transport tls> and <grpc>. The gRPC listener used to accept plaintext regardless of <transport tls>, and it now requires TLS. Plaintext OTLP/gRPC clients have to be switched to TLS.

Example

<source>
  @type opentelemetry
  tag opentelemetry

  <http>
    bind 0.0.0.0
    port 4318
  </http>

  <grpc>
    bind 0.0.0.0
    port 4317
  </grpc>
</source>

Input opentelemetry_metrics plugin

This plugin emits Fluentd's metric data that conforms to the OpenTelemetry Protocol. To output the data, it requires to use output opentelemetry plugin.

Root section

parameter	type	description	default
tag	string	The tag of the event	required
emit_interval	time	Determine the rate to emit internal metrics as events	60
metric_name_prefix	string	The prefix of metric name	fluentd.

Example

# Emit Fluentd metrics
<source>
  @type opentelemetry_metrics
  tag opentelemetry.fluentd.metrics
  emit_interval 300s
</source>

# Send Fluentd metrics to OpenTelemetry Collector
<match opentelemetry.fluentd.metrics>
  @type opentelemetry

  <http>
    endpoint "https://127.0.0.1:4318"
  </http>
</match>

Tip

To visualize Fluentd CPU usage (process_cpu_time), use the rate() function since the CPU metric is reported as cumulative time (Counter).

PromQL Example: rate(fluentd_process_cpu_time_seconds_total[$__rate_interval])

Note

Process Metrics (CPU / Memory)

Fluentd operates on a multi-process architecture consisting of a Supervisor process and one or more Worker processes.

Because this plugin runs inside a specific worker process, the process-level metrics it collects (such as CPU time and memory usage) represent only the footprint of that single worker process, not the total resource consumption of the entire Fluentd instance.

If you are using a multi-worker configuration (<system> workers N </system>), be aware that the actual total CPU and memory usage of Fluentd will be higher than what this plugin reports.

For accurate, instance-wide resource monitoring, we strongly recommend using external OS-level monitoring tools rather than relying on application-internal metrics:

• Container Environments (Kubernetes/Docker): Use tools like cAdvisor or kube-state-metrics, etc.
• VM/Bare-metal Environments: Use node_exporter combined with process-exporter, etc.

Output opentelemetry plugin

To send data, this plugin requires <http> or <grpc> section.

<http> section

This requires to send data via HTTP/HTTPS.

parameter	type	description	default
endpoint	string	The endpoint for HTTP/HTTPS request	http://127.0.0.1:4318
proxy	string	The proxy for HTTP/HTTPS request	nil
error_response_as_unrecoverable	bool	Raise UnrecoverableError when the response code is not SUCCESS	true
retryable_response_codes	array	The list of retryable response codes	[429, 502, 503, 504]
read_timeout	integer	The read timeout in seconds	60
write_timeout	integer	The write timeout in seconds	60
connect_timeout	integer	The connect timeout in seconds	60

parameter	type	description	available values	default
compress	enum	The option to compress HTTP request body	text / gzip	text

<grpc> section

This requires to send data via gRPC. It needs to install grpc gem manually to use this feature.

Warning

Now, gRPC feature status is experimental.

parameter	type	description	default
endpoint	string	The endpoint for gRPC request	127.0.0.1:4317
timeout	integer	The maximum time (seconds) to wait a response from the server	60
keepalive_time	integer	The interval in seconds to send gRPC keepalive pings.	30
keepalive_timeout	integer	The timeout in seconds to wait for a keepalive ping acknowledgement.	15

parameter	type	description	available values	default
compress	enum	The option to compress gRPC request body	text / gzip	text

<transport> section

parameter	type	description	default
ca_path	string	Specifies the path of CA Certificate file used to verify the server	nil
cert_path	string	Specifies the path of Certificate file	nil
private_key_path	string	Specifies the path of Private Key file	nil
private_key_passphrase	string	Specifies the public CA private key passphrase	nil
insecure	bool	Skips the verification of the server certificate	false

parameter	type	description	available values	default
min_version	enum	Specifies the lower bound of the supported SSL/TLS protocol	TLS1_1 / TLS1_2 / TLS1_3	nil
max_version	enum	The endpoint for HTTP/HTTPS request	TLS1_1 / TLS1_2 / TLS1_3	nil

Refer Config: Transport Section

The HTTP connection uses ca_path, cert_path, private_key_path, private_key_passphrase, min_version, max_version and insecure, and the other TLS parameters have no effect on it. ca_path is added to the default trusted CAs of the system instead of replacing them.

Fluentd rejects a <transport tls> section that configures no certificate at all, so pair ca_path with insecure true to verify the server without presenting a client certificate. On the HTTP connection, insecure true skips the verification only when ca_path is omitted.

Warning

This changes the behavior of an existing configuration that has both ca_path and insecure true. The HTTP connection used to skip the verification, and it now verifies the server certificate and its hostname. Drop ca_path to keep the previous behavior.

When the <grpc> section is configured, <transport tls> applies to the gRPC connection as well. gRPC uses ca_path, cert_path, private_key_path and private_key_passphrase, and the other TLS parameters have no effect on the gRPC connection. insecure does not weaken the gRPC connection: the server certificate is verified against ca_path, or against the default root certificates of gRPC when ca_path is omitted.

Warning

This changes the behavior of an existing configuration that has both <transport tls> and <grpc>. The gRPC connection used to be plaintext regardless of <transport tls>, and it now uses TLS, so the collector has to accept it.

<buffer> section

parameter	type	description	default
chunk_keys	array	Overwrites the default chunk_keys value in this plugin.	tag
chunk_limit_size	integer	Overwrites the default chunk_limit_size value in this plugin.	8M (8MB)

Note on chunk_limit_size: Currently, this plugin is reducing I/O load by combining data within a chunk and sending it as a single unit. Depending on the configuration of your OpenTelemetry Collector or downstream backend (e.g., maximum request body size limits), an 8MB chunk might be rejected with a 413 Payload Too Large error. If you encounter this error in your Fluentd logs, please adjust (decrease) the chunk_limit_size in your <buffer> section.

Refer Config: Buffer Section

Example

<match opentelemetry.**>
  @type opentelemetry

  <http>
    endpoint "https://127.0.0.1:4318"
  </http>
</match>

Copyright

• Copyright(c) 2025- Shizuo Fujita
• License
  • Apache License, Version 2.0