AWS Kinesis Data Firehose Receiver

Status
Stability	alpha: metrics, logs
Distributions	contrib
Issues
Code Owners	@Aneurysm9, @axw

Receiver for ingesting AWS Kinesis Data Firehose delivery stream messages and parsing the records received based on the configured record type.

Configuration

Example:

receivers:
  awsfirehose:
    endpoint: 0.0.0.0:4433
    record_type: cwmetrics
    access_key: "some_access_key"
    tls:
      cert_file: server.crt
      key_file: server.key

The configuration includes the OpenTelemetry collector's server confighttp, which allows for a variety of settings. Only the most relevant ones will be discussed here, but all are available. The AWS Kinesis Data Firehose Delivery Streams currently only support HTTPS endpoints using port 443. This can be potentially circumvented using a Load Balancer.

endpoint:

The address:port to bind the listener to.

default: localhost:4433

See our security best practices doc to understand how to set the endpoint in different environments.

tls:

See documentation for more details.

A cert_file and key_file are required.

encoding:

The ID of an encoding extension for decoding logs or metrics. This configuration also supports the built-in encodings listed in the Encodings section. If no encoding is specified, then the receiver will default to a signal-specific encoding: cwmetrics for metrics, and cwlogs for logs.

record_type:

Deprecated, use encoding instead. record_type will be removed in a future release; it is an alias for encoding.

access_key (Optional):

The access key to be checked on each request received. This can be set when creating or updating the delivery stream. See documentation for details.

Encodings

cwmetrics

The encoding for the CloudWatch metric stream. Expects the format for the records to be JSON. See documentation for details.

cwlogs

The encoding for the CloudWatch log stream. Expects the format for the records to be JSON. For example:

{
  "messageType": "DATA_MESSAGE",
  "owner": "111122223333",
  "logGroup": "my-log-group",
  "logStream": "my-log-stream",
  "subscriptionFilters": ["my-subscription-filter"],
  "logEvents": [
    {
      "id": "123",
      "timestamp": 1725544035523,
      "message": "My log message."
    }
  ]
}

otlp_v1

The OTLP v1 encoding as produced by CloudWatch metric streams. See documentation for details.