[receiver/awsxrayreceiver] Add option to pull x-ray traces from AWS

[Simon-Boyer]

Component(s)

receiver/awsxray

Is your feature request related to a problem? Please describe.

I am trying to pull traces directly from AWS, so that I do not have to setup layers or other configs for all my lambda functions. From my understanding, the x-ray receiver can only receive and process x-ray traces, but it does not actively pull them from AWS.

Describe the solution you'd like

An option that can enable a "pulling" mode: instead of passing the x-ray traces to the receiver, the receiver pulls data directly from AWS every X interval. Could look something like this:

receivers:
  awsxray:
    pull:
      enabled: true
      interval: 30s
    endpoint: 0.0.0.0:2000
    transport: udp
    ...

Describe alternatives you've considered

Writing a separate binary that pulls data from X-Ray and forwards it to the current receiver. Would work, but I think it makes more sense to have it in the receiver.

Additional context

Basically trying to reproduce this datadog integration: https://docs.datadoghq.com/integrations/amazon_xray/

[github-actions]

Pinging code owners:

• receiver/awsxray: @wangzlei @srprash

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

so that I do not have to setup layers or other configs for all my lambda functions

I'm interested in hearing about the complexity of this, as I'm unfamiliar. What all is involved in setting this up? Are there performance implications here?

Does AWS x-ray expose a way to pull traces from it?

The code owners would have much more context and information here, so they may have better questions and thoughts here.

[Simon-Boyer]

What all is involved in setting this up?

I havent looked in details, but i'm guessing simply querying the API endpoint

Are there performance implications here?

Probably, that's why you would change the pull rate according to your needs

Does AWS x-ray expose a way to pull traces from it?

Looks like yes: https://docs.aws.amazon.com/xray/latest/devguide/xray-api-gettingdata.html#xray-api-traces

[crobert-1]

What all is involved in setting this up?

I havent looked in details, but i'm guessing simply querying the API endpoint

I'm wondering how complicated it is to have AWS x-ray export traces to the collector. It sounds like, from what you're saying, it's some amount of work to accomplish this, so I'm wondering what all is involved in making this happen with the collector's existing functionality? Sorry for the confusion there, I didn't word my question very clearly.

[Simon-Boyer]

I'm not sure if I fully understand the question, but I don't think it is really difficult to have AWS exports to the collector.

If we were to use the collector existing functionality, I think we could write something like a small script/program on a cron job that scrapes the last traces on the AWS X-Ray API endpoint, then export them (without changing the format) to an OTEL collector with the xray receiver configured.

But I believe having that feature available, builtin, into the receiver would make sense instead of having it completely separate. As an example, the cloudwatch receiver (for logs) does offers polling data instead of simply receiving it: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awscloudwatchreceiver

[crobert-1]

That's fair, a cron job is definitely not ideal. I'll defer to code owners for better insights into possibly adding this 👍