The OpenTelemetry output plugin supports OAuth2 client-credentials authentication but has no built-in way to authenticate using cloud instance metadata tokens - the standard mechanism on platforms where a VM has a linked service account.
Use case
VMs with linked service accounts need to send OTLP data to cloud logging/monitoring backends that require IAM bearer tokens. Today the only options are a sidecar proxy (OTel Collector with an auth extension) or a custom Fluent Bit build. Both add operational complexity that a native plugin option would eliminate.
The out_stackdriver plugin already solves this for GCP via gce_metadata.c, but the OTel output plugin - which is becoming the recommended path for logs, metrics, and traces - has no equivalent.
Proposed approach
Add three config options to out_opentelemetry:
| Option |
Description |
Default |
metadata_token_url |
HTTP URL of the metadata token endpoint. Presence enables the feature. |
(none) |
metadata_token_header |
Extra header for the GET request (e.g. Metadata-Flavor: Google). |
(none) |
metadata_token_refresh |
Token refresh interval in seconds (minimum 61). |
3600 |
The plugin fetches a Bearer token via HTTP GET from the configured URL, parses the standard { "access_token": "...", "expires_in": N } JSON response, and populates the existing flb_oauth2 context. Token caching, periodic refresh, and 401 retry reuse the existing oauth2 code paths - no new token injection mechanism is introduced.
The design follows the pattern established by out_stackdriver (gce_metadata.c) but is cloud-neutral: no vendor-specific naming, no compile flags, no new dependencies. It works with any metadata endpoint that returns the standard JSON response format.
I have a working implementation at evgfitil#1. Happy to adapt based on maintainer feedback.
The OpenTelemetry output plugin supports OAuth2 client-credentials authentication but has no built-in way to authenticate using cloud instance metadata tokens - the standard mechanism on platforms where a VM has a linked service account.
Use case
VMs with linked service accounts need to send OTLP data to cloud logging/monitoring backends that require IAM bearer tokens. Today the only options are a sidecar proxy (OTel Collector with an auth extension) or a custom Fluent Bit build. Both add operational complexity that a native plugin option would eliminate.
The
out_stackdriverplugin already solves this for GCP viagce_metadata.c, but the OTel output plugin - which is becoming the recommended path for logs, metrics, and traces - has no equivalent.Proposed approach
Add three config options to
out_opentelemetry:metadata_token_urlmetadata_token_headerMetadata-Flavor: Google).metadata_token_refresh3600The plugin fetches a Bearer token via HTTP GET from the configured URL, parses the standard
{ "access_token": "...", "expires_in": N }JSON response, and populates the existingflb_oauth2context. Token caching, periodic refresh, and 401 retry reuse the existing oauth2 code paths - no new token injection mechanism is introduced.The design follows the pattern established by
out_stackdriver(gce_metadata.c) but is cloud-neutral: no vendor-specific naming, no compile flags, no new dependencies. It works with any metadata endpoint that returns the standard JSON response format.I have a working implementation at evgfitil#1. Happy to adapt based on maintainer feedback.