Support for receiving profiles

[AndreZiviani]

Request

Would be really useful to be able to receive profiles on the agent, something similar to loki.source.api

Use case

The main reasons for me are:

• Pushing profiles to an agent on our infrastructure is faster (lower latency) than to Grafana Cloud via internet, less likely to impact our application
• Don't require devs to add infrastructure logic on the instrumentation, e.g. Grafana Cloud authentication / Tenant authentication, metadata labels (kubernetes labels, business labels, ....)
• Allows more flexibility on sampling

[ptodev]

Hi @AndreZiviani, thank you for raising an issue! Aer you referring to Pprof profiles? Is there a reason why pyroscope.scrape would not be sufficient for the purposes listed above?

[AndreZiviani]

Aer you referring to Pprof profiles?

I'm not sure, I want to be able to instrument my applications with the official Pyroscope libraries (not sure if it sends the profiles as pprof or something else) and send them to grafana agent. I forgot to mention Pyroscope on the original message 🤦‍♂️

Is there a reason why pyroscope.scrape would not be sufficient for the purposes listed above?

Pyroscope libraries does not expose an endpoint for collection, it is only push-mode. I could look for other libraries but would like to stick with the official ones if possible

[ptodev]

I spoke with the Pyroscope team and they agree that this is a very valid request. The demand for this is likely to grow over time, so it is good to keep the issue open, but we cannot commit to when this will be implemented yet.

[AndreZiviani]

ok, thanks for the followup @ptodev

I guess its worth mentioning the future profiling support on opentelemetry, could be a long time before that's approved/implemented but I think its important to consider when implementing either/both options on the agent

[github-actions]

This issue has not had any activity in the past 30 days, so the needs-attention label has been added to it.
If the opened issue is a bug, check to see if a newer release fixed your issue. If it is no longer relevant, please feel free to close this issue.
The needs-attention label signals to maintainers that something has fallen through the cracks. No action is needed by you; your issue will be kept open and you do not have to respond to this comment. The label will be removed the next time this job runs if there is new activity.
Thank you for your contributions!

[fede843]

I am looking forward this feature too!
We are deploying an instance of Grafana Agent per server to gather, process/filter (as much as possible at that instance) and proxy all observability signals so far, and profiling is not fitting there yet. It is true that already existing scrape config might be enough for many cases, but not all. We are trying to link traces to profiles, using Java, and we need the jar agent, at least for now.
It would be interesting to see how this develops in the future, along the adoption profiling into OTEL. That is the key I think.