Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New component: AMQP/RabbitMQ receiver and exporter to move OTLP data #10592

Closed
corentinaltepe opened this issue Jun 4, 2022 · 6 comments
Closed
Labels

Comments

@corentinaltepe
Copy link

corentinaltepe commented Jun 4, 2022

The purpose and use-cases of the new component

In case of an infrastructure where telemetry can only be tunneled through an AMQP RabbitMQ exchange, it'd be great if one could use the OPTL Collector on both ends of that tunnel to proxy (export and receive) the telemetry data. I believe this is what the Kafka Exporter and the Kafka Receiver are for, though please correct me if I'm wrong.

The existing RabbitMQ Receiver does not serve that purpose. Instead, it is used to collect the RabbitMQ instance's own telemetry.

Example configuration for the component

The architecture would look like this:

  graph LR;
      ProdApp-->OPTLCollectorRabbitMQExporter-->RabbitMQExchange-->OPTLCollectorRabbitMQReceiver-->Backend

Related Architecture

With such contraints, one can use this architecture with logstash to push logs. It is also possible to design a custom made client/proxy to pull and push metrics, resembling what the Prometheus PushProx does, but over AMQP insteand of HTTP.
For reference, here's how the logstash output (exporter) plugin, and the logstash input plugin (receiver) can be configured.

Telemetry data types supported

All three: traces, metrics and logs.

@corentinaltepe corentinaltepe changed the title New component: AMQP/RabbitMQ receiver and exporter to move OTPL data New component: AMQP/RabbitMQ receiver and exporter to move OPTL data Jun 5, 2022
@djaglowski djaglowski added the Sponsor Needed New component seeking sponsor label Jun 6, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Nov 8, 2022

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

@realbucksavage
Copy link

I'm also looking for an RabbitMQ/AMQP0-9-1 exporter. @djaglowski I can work on both receiver and exported implementation as time permits.

@atoulme atoulme changed the title New component: AMQP/RabbitMQ receiver and exporter to move OPTL data New component: AMQP/RabbitMQ receiver and exporter to move OTLP data Mar 12, 2023
@github-actions github-actions bot removed the Stale label May 26, 2023
@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

@github-actions
Copy link
Contributor

This issue has been closed as inactive because it has been stale for 120 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 24, 2023
@realbucksavage
Copy link

Is this addition still on the roadmap?

@realbucksavage
Copy link

To add some more context to it, the exporter can Protobuf serialized traces, metrics, and logs. The sample configuration can look something like this:

exporters:
  amqp091:
    address: amqp://guest:guest@localhost:5672/
    exchange: otlp
    traces_routing_key: otlp.traces
    metrics_routing_key: otlp.metrics
    logs_routing_key: otlp.logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants