Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

otel collector failing to collect/export metrics when datadog exporter is unable to retrieve host metadata using gohai #16092

Closed
michelleN opened this issue Nov 6, 2022 · 3 comments
Labels
bug Something isn't working exporter/datadog Datadog components needs triage New item requiring triage

Comments

@michelleN
Copy link

michelleN commented Nov 6, 2022

Component(s)

exporter/datadog

What happened?

Description

I'm not sure if this is a bug of if I am using datadog

Steps to Reproduce

Run otel collector using datadog exporter

Expected Result

Run without errors

Actual Result

The collector starts up fine and later I see errors related to being unable to fetch host metadata and I'm seeing that the metrics are neither being collected nor are they being exported.

Collector version

0.62.1

Environment information

Environment

OS: Ubuntu 22.04
Docker image: otel/opentelemetry-collector-contrib:0.62.1

OpenTelemetry Collector configuration

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:<port>
exporters:
  file:
    path: ./metrics.json
  datadog:
    api:
      site: datadoghq.com
      key: *******
processors:
  batch:
extensions:
  health_check:
  pprof:
    endpoint: :<port>
  zpages:
    endpoint: :<port>
service:
  extensions: [pprof, zpages, health_check]
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [file, datadog]

Log output

From stderr:

gohai/gohai.go:47	Failed to retrieve filesystem metadata	{"kind": "exporter", "data_type": "metrics", "name": "datadog", "error": "df failed to collect filesystem data: %!s(<nil>)"}


warn	gohai/gohai.go:66	Failed to retrieve platform metadata	{"kind": "exporter", "data_type": "metrics", "name": "datadog", "error": "exec: \"uname\": executable file not found in $PATH"}
2022-10-21T20:11:54.639Z	info	metadata/metadata.go:216	Sent host metadata	{"kind": "exporter", "data_type": "metrics", "name": "datadog"}

From stdout:
[Debug] Error fetching info for pid 1: open /etc/passwd: no such file or directory



### Additional context

I'm trying to understand if this is expected behavior or a bug and what exactly is causing these issues. Is it a docker related config issue? I'm running on Nomad so I'm also wondering if it has anything to do with Nomad. Would appreciate any help here debugging/understanding/fixing. Thanks.
@michelleN michelleN added bug Something isn't working needs triage New item requiring triage labels Nov 6, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2022

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the exporter/datadog Datadog components label Nov 6, 2022
@michelleN michelleN changed the title otel failing when datadog exporter is unable to retrieve host metadata using gohai otel collector failing to collect/export metrics when datadog exporter is unable to retrieve host metadata using gohai Nov 6, 2022
@dineshg13
Copy link
Member

This is duplicate of #14186 , we will look into this and fix asap.

@mx-psi
Copy link
Member

mx-psi commented Nov 7, 2022

Thanks for reporting @michelleN, I am closing as a duplicate of #14186; feel free to add aditional details over there

@mx-psi mx-psi closed this as not planned Won't fix, can't repro, duplicate, stale Nov 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working exporter/datadog Datadog components needs triage New item requiring triage
Projects
None yet
Development

No branches or pull requests

3 participants