Skip to content

Files

Latest commit

 

History

History

coralogixexporter

README.md

Coralogix Exporter

Status
Stability alpha: profiles
beta: traces, metrics, logs
Distributions contrib
Issues Open issues Closed issues
Code Owners @povilasv, @iblancasa, @douglascamata

The Coralogix exporter sends traces, metrics and logs to Coralogix.

Please review the Collector's security documentation, which contains recommendations on securing sensitive information such as the API key required by this exporter.

Configuration

Example configuration:

exporters:
  coralogix:
    # The Coralogix domain
    domain: "coralogix.com"
    # Your Coralogix private key is sensitive
    private_key: "xxx"

    # (Optional) Ordered list of Resource attributes that are used for Coralogix
    # AppName and SubSystem values. The first non-empty Resource attribute is used.
    # Example: application_name_attributes: ["k8s.namespace.name", "service.namespace"]
    # Example: subsystem_name_attributes: ["k8s.deployment.name", "k8s.daemonset.name", "service.name"]
    application_name_attributes:
    - "service.namespace"
    subsystem_name_attributes:
    - "service.name"

    # Traces, Metrics and Logs emitted by this OpenTelemetry exporter 
    # are tagged in Coralogix with the default application and subsystem constants.
    application_name: "MyBusinessEnvironment"
    subsystem_name: "MyBusinessSystem"

    # (Optional) Timeout is the timeout for every attempt to send data to the backend.
    timeout: 30s

Compression

By default, the Coralogix exporter uses gzip compression. Alternatively, you can use zstd compression, for example:

exporters:
  coralogix:
    domain_settings:
      compression: "zstd"

v0.76.0 Coralogix Domain

Since v0.76.0 you can specify Coralogix domain in the configuration file instead of specifying different endpoints for traces, metrics and logs. For example, the configuration below, can be replaced with domain field:

Old configuration:

exporters:
  coralogix:
    traces:
      endpoint: "ingress.coralogix.com:443"
    metrics:
      endpoint: "ingress.coralogix.com:443"
    logs:
      endpoint: "ingress.coralogix.com:443"

New configuration with domain field:

exporters:
  coralogix:
    domain: "coralogix.com"

Coralogix's Domain

Depending on your region, you might need to use a different domain. Here are the available domains:

Region Domain
USA1 coralogix.us
USA2 cx498.coralogix.com
APAC1 coralogix.in
APAC2 coralogixsg.com
APAC3 ap3.coralogix.com
EUROPE1 coralogix.com
EUROPE2 eu2.coralogix.com

Additionally, Coralogix supports AWS PrivateLink, which provides private connectivity between virtual private clouds (VPCs), supported AWS services, and your on-premises networks without exposing your traffic to the public internet.

Here are available AWS PrivateLink domains:

Region Domain
USA1 private.coralogix.com
USA2 ingress.private.cx498-aws-us-west-2.coralogix.com
APAC1 private.coralogix.in
APAC2 private.coralogixsg.com
EUROPE1 private.coralogix.com
EUROPE2 private.eu2.coralogix.com
Learn more about AWS PrivateLink in the documentation page.

Application and SubSystem attributes

v0.62.0 release of OpenTelemetry Collector allows you to map Application name and Subsystem name to Resource attributes. You need to set application_name_attributes and subsystem_name_attributes fields with a list of potential Resource attributes for the AppName and Subsystem values. The first not-empty Resource attribute is going to be used. If multiple resource attributes are available, the order of the attributes in the list determines their priority.

Kubernetes attributes

When using OpenTelemetry Collector with k8sattribute processor, you can use attributes coming from Kubernetes, such as k8s.namespace.name or k8s.deployment.name. The following example shows recommended list of attributes:

exporters:
  coralogix:
    domain: "coralogix.com"
    application_name_attributes:
      - "k8s.namespace.name" 
      - "service.namespace"
    subsystem_name_attributes:
      - "k8s.deployment.name"
      - "k8s.statefulset.name"
      - "k8s.daemonset.name"
      - "k8s.cronjob.name"
      - "service.name"

Host Attributes

OpenTelemetry Collector resourcedetection processor can discover Host Resource attributes, such as host.name and provide Resource attributes using environment variables, which can be used for setting AppName and SubSystem fields in Coralogix.

Example:

processors:
  resourcedetection/system:
    detectors: ["system", "env"]
    system:
      hostname_sources: ["os"]

And setting environment variable such as:

OTEL_RESOURCE_ATTRIBUTES="env=production"

You can configure Coralogix Exporter:

exporters:
  coralogix:
    domain: "coralogix.com"
    application_name_attributes:
      - "env" 
    subsystem_name_attributes:
      - "host.name"

EC2 Attributes

OpenTelemetry Collector resourcedetection processor can discover EC2 Resource attributes, such as EC2 tags as resource attributes.

Example:

processors:
 resourcedetection/ec2:
    detectors: ["ec2"]
    ec2:
      # A list of regex's to match tag keys to add as resource attributes can be specified
      tags:
        - ^ec2.tag.name$
        - ^ec2.tag.subsystem$

NOTE: In order to fetch EC2 tags, the IAM role assigned to the EC2 instance must have a policy that includes the ec2:DescribeTags permission.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "ec2:DescribeTags",
            "Resource": "*"
        }
    ]
}

You can configure Coralogix Exporter:

exporters:
  coralogix:
    domain: "coralogix.com"
    application_name_attributes:
      - "ec2.tag.name" 
    subsystem_name_attributes:
      - "ec2.tag.subsystem"

Custom Attributes

You can combine and create custom Resource attributes using transform processor. For example:

    processors:
      transform:
        error_mode: ignore
        log_statements:
          - context: resource
            statements:
            - set(attributes["applicationName"], Concat(["development-environment", attributes["k8s.namespace.name"]], "-"))

Then you can use the custom Resource attribute in Coralogix exporter:

exporters:
  coralogix:
    domain: "coralogix.com"
    application_name_attributes:
      - "applicationName" 
    subsystem_name_attributes:
      - "host.name"

Exporting to multiple teams based on attributes

You can export the signals based on your business logic (attributes) to different Coralogix teams. To achieve this, you'll need to use the filter processor and setup one pipeline per team. You can setup your filter processors as following (example with metrics):

processors:  
  filter/teamA:
    metrics:
      datapoint:
          - 'attributes["your_label"] != "teamA"'
  filter/teamB:
    metrics:
      datapoint:
          - 'attributes["your_label"] != "teamB"'

This configuration ensures separate processor per each team. Any data points without an attribute for a particular team will be dropped from exporting.

Secondly, set up an individual exporter per each team:

exporters:
  coralogix/teamA:
    metrics:
      endpoint: "otel-metrics.coralogix.com:443"
    private_key: <private_key_for_teamA>
    application_name: "MyBusinessEnvironment"
    subsystem_name: "MyBusinessSystem"
  coralogix/teamB:
    metrics:
      endpoint: "otel-metrics.coralogix.com:443"
    private_key: <private_key_for_teamB>
    application_name: "MyBusinessEnvironment"
    subsystem_name: "MyBusinessSystem"

Finally, join each processor and exporter (and any other components you wish) in the pipelines. Here is an example with a Prometheus receiver:

service:
  pipelines:
    metrics/1:
      receivers: [prometheus]
      processors: [filter/teamA]
      exporters: [coralogix/teamA]
    metrics/2:
      receivers: [prometheus]
      processors: [filter/teamB]
      exporters: [coralogix/teamB]

Custom application and subsystem name

You can pass custom application and subsystem name via the following resource attributes:

  • cx.subsystem.name
  • cx.application.name For example:
receivers:
  filelog/nginx:
    include:
      - '/tmp/tmp.log'
    include_file_path: true
    include_file_name: false
    start_at: end
    resource: 
      cx.subsystem.name: nginx
  filelog/access-log:
    include:
      - '/tmp/access.log'
    include_file_path: true
    include_file_name: false
    resource: 
      cx.subsystem.name: access-log
exporters:
  coralogix:
    domain: 'coralogix.com'
    private_key: "XXX"
    application_name: 'app_name'
    timeout: 30s
service:
  pipelines:
    logs:
      receivers: [filelog/nginx, filelog/access-log]
      exporters: [coralogix]

Need help?

Our world-class customer success team is available 24/7 to walk you through the setup for this exporter and answer any questions that may come up. Feel free to reach out to us via our in-app chat or by sending us an email to support@coralogix.com.