Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for OTEL Collector #106

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

roopeshsn
Copy link

@roopeshsn roopeshsn commented Apr 7, 2024

No additional instrumentation changes are required for the OTEL collector. It receives (supports prometheus metrics format) metrics from Prometheus and exports them to the desired remote backend. I also tested OTEL collector with Grafana cloud.

#105

Signed-off-by: Roopesh Saravanan <roopeshsaravanan.dev@gmail.com>
@roopeshsn
Copy link
Author

roopeshsn commented Apr 7, 2024

If the end user makes changes to the endpoint field in the otel-collector-config.yml file, then the OTEL collector will export metrics to that backend. So in parallel, one can export metrics to Prometheus backend and visualize them in Grafana in localhost and remote backend (in my case Grafana cloud). I would like to ensure whether this is a desired behavior.

Copy link

@pmoroney pmoroney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks!

Signed-off-by: Roopesh Saravanan <roopeshsaravanan.dev@gmail.com>
Signed-off-by: Roopesh Saravanan <roopeshsaravanan.dev@gmail.com>
Copy link
Contributor

@sanfern sanfern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add other architure support (i.e. arm).
We are getting this error 024-05-06T12:04:52.044+0530 error exporterhelper/queue_sender.go:101 Exporting failed. Dropping data. {"kind": "exporter", "data_type": "metrics", "name": "prometheusremotewrite", "error": "Permanent error: Permanent error: context deadline exceeded", "dropped_items": 39} go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1

metrics:
receivers: [prometheus]
processors: [batch]
exporters: [prometheusremotewrite]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix this new line

.gitignore Outdated
@@ -0,0 +1,2 @@
/dev_environment/.vagrant
/dev_environment/*.log
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix this new line

@@ -160,6 +169,9 @@ then
sleep 1
/etc/init.d/grafana-server stop || true
/etc/init.d/grafana-server start || true

# Start OTEL collector
./root/otelcol --config=/etc/otelcol/config.yml &
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modify this to absolute path

@@ -170,6 +182,9 @@ else
# Start and enable Grafana
systemctl restart grafana-server
systemctl enable grafana-server.service

# Start OTEL collector
./root/otelcol --config=/etc/otelcol/config.yml &
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modify this to absolute path

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So can I change it into ./otelcol --config=/etc/otelcol/config.yml &?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modify path to /usr/local/bin

@roopeshsn
Copy link
Author

Add other architure support (i.e. arm). We are getting this error 024-05-06T12:04:52.044+0530 error exporterhelper/queue_sender.go:101 Exporting failed. Dropping data. {"kind": "exporter", "data_type": "metrics", "name": "prometheusremotewrite", "error": "Permanent error: Permanent error: context deadline exceeded", "dropped_items": 39} go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1

So the OTEL collector works fine in x64 architecture right?

@roopeshsn
Copy link
Author

Add other architure support (i.e. arm). We are getting this error 024-05-06T12:04:52.044+0530 error exporterhelper/queue_sender.go:101 Exporting failed. Dropping data. {"kind": "exporter", "data_type": "metrics", "name": "prometheusremotewrite", "error": "Permanent error: Permanent error: context deadline exceeded", "dropped_items": 39} go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1

This issue is not yet fixed by OTEL. But there is a workaround: open-telemetry/opentelemetry-collector-contrib#31910 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants