[processor/resourcedetection] missing host.id for system resourcedetection

[sumo-drosiek]

Component(s)

processor/resourcedetection

Is your feature request related to a problem? Please describe.

host.id which is specified in convention in not set by resourcedetectionprocessor

Describe the solution you'd like

I would like to get host.id from /etc/machine-id

Describe alternatives you've considered

N/A

Additional context

N/A

[github-actions]

Pinging code owners:

• processor/resourcedetection: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dmitryax]

Resolved by #18618

[rayrapetyan]

Trying to redefine host.id with:

processors:
  resource:
    attributes:
      - key: host.id
        from_attribute: host.name
        action: upsert

but it doesn't work (host.id in Prometheus is still equal to /etc/machine-id). Moreover, every time I restart otel-collector-contrib container (this happens quite often), it gets a new host.id assigned and I'm getting tons of duplicates in Prometheus.
What's the best way to avoid this behavior?

[sumo-drosiek]

@rayrapetyan What is a desired behavior you want to achieve? having host.name as host.id?

[rayrapetyan]

@sumo-drosiek, right, so it stops refreshing every time otel-collector-contrib container restarts.

[sumo-drosiek]

@rayrapetyan Could you share your pipeline configuration?

[savec80]

Hello @sumo-drosiek ,
The issue we're are facing is this change overwrites Host-Id from AWS metadata, in case you have 2 resourcedetectors: system and ec2. As a result AWSUniqueId is different and it breaks SignalFX integration with AWS.
Is there a flag to skip host-id field from this processor?
Thanks.

[dmitryax]

Hi @savec80. Thanks for reporting. I've submitted a proposal that will fix this problem #21482 and I'm working on it

[rayrapetyan]

@sumo-drosiek, sure here is my config:

receivers:
  otlp/collector:
    protocols:
      grpc:
        endpoint: "0.0.0.0:{{ otel_collector_port }}"
        max_recv_msg_size_mib: 32
  hostmetrics:
    collection_interval: 15s
    root_path: /hostfs
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      load:
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      network:
        metrics:
          system.network.conntrack.count:
            enabled: true
          system.network.conntrack.max:
            enabled: true
      paging:
        metrics:
          system.paging.utilization:
            enabled: true    
  hostmetrics/storage:
    collection_interval: 1m
    root_path: /hostfs
    scrapers:
      disk:
      filesystem:
        metrics:
          system.filesystem.utilization:
            enabled: true
        include_fs_types:
          fs_types: ["ext3", "ext4"]
          match_type: strict
  hostmetrics/processes:
    collection_interval: 30s
    root_path: /hostfs
    scrapers:
      processes:

processors:
  memory_limiter:
    check_interval: 2s
    limit_percentage: 80
    spike_limit_percentage: 40
  batch:
  resource:
    attributes:
      - key: service.name
        value: "{{ otel_service_name }}"
        action: insert
      - key: host.id
        from_attribute: host.name
        action: upsert
  resourcedetection:
    detectors:
      - system
      - env
    timeout: 10s

exporters:
  otlp/gw:
    endpoint: "{{ otelgw_domain }}:{{ otelgw_port }}"
    tls:
      insecure_skip_verify: true
    auth:
      authenticator: basicauth
    compression: gzip

extensions:
  health_check:
  basicauth:
    client_auth:
      username: "{{ otelgw_username }}"
      password:` "{{ otelgw_password }}"

service:
  extensions: [ health_check, basicauth ]
  pipelines:
    metrics:
      receivers: [ hostmetrics, hostmetrics/storage, hostmetrics/processes, otlp/collector ]
      processors: [ memory_limiter, resource, resourcedetection, batch ]
      exporters: [ otlp/gw ]
    traces:
      receivers: [ otlp/collector ]
      processors: [ memory_limiter, resource, batch ]
      exporters: [ otlp/gw ]
    logs:
      receivers: [ otlp/collector ]
      processors: [ memory_limiter, resource, batch ]
      exporters: [ otlp/gw ]

service.name is being populated properly, but host.id is submitted as a new unique value into otel-gw after each container restart.

[krantishetty]

Is this issue fixed? I'm facing same issue to get the hostname details

[sumo-drosiek]

@krantishetty

I'm facing same issue to get the hostname details

Which one do you mean? In this thread few issues have been mentioned already

[krantishetty]

I'm trying to get the hostname details with updating host_id but no luck. Can you advise how to define key host_id value under processors for hostname output?

[sumo-drosiek]

Can you advise how to define key host_id value under processors for hostname output?

If you are using resourcedetectionprocessor you need to enable host.id in configuration:

resourcedetection:
  system:
    resource_attributes:
      host.id:
        enabled: true

This is due to #21482