Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Be able to set an agent log format with a configuration option #147

Open
fede843 opened this issue Mar 22, 2024 · 1 comment
Open

Be able to set an agent log format with a configuration option #147

fede843 opened this issue Mar 22, 2024 · 1 comment

Comments

@fede843
Copy link

fede843 commented Mar 22, 2024

Our scenario

Hello. We are using docker as the underlying technology to deploy apps and the observability stack. For all the logs we decide to use JSON format that is sent to our Loki backend.
We are trying to adopt the pyroscope Java Otel extension (that implicitly has the Java Agent) to send profiling data to our Pyroscope server.

The issue

During our testing, when we are sending the Pyroscope backend server down, all Java apps using the extension are producing lot of logs that don't align with the rest of echosystem. In our case is JSON.
This is the kind of output we are getting from the agent logger in the docker console:

2024-03-21 11:37:15.292 [ERROR] Error uploading snapshot: 502 <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx</center>
</body>
</html>

Currently the agent only has this log parameter PYROSCOPE_LOG_LEVEL which is not enough for this case.

Possible solutions

  • As discussed in this thread a custom logger interface implementation can be done:

https://github.com/grafana/pyroscope-java/blob/f73ebfaae1fcdfdf61d3418a78807c5b2b0[…]/agent/src/main/java/io/pyroscope/javaagent/PyroscopeAgent.java
https://github.com/grafana/pyroscope-java/blob/82b1daba0dc05b1851e6b96f2d760ab7901[…]0c49/agent/src/main/java/io/pyroscope/javaagent/api/Logger.java

However, that would require initializing the profiler outside the otel extension, and this is not available when it's running as a javagent.

  • Add a flag PYROSCOPE_LOG_FORMAT to set the logger format. it could be JSON or LOGFTM to align with rest of Grafana stack.
@devurandom
Copy link

This might be related to #126.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@devurandom @fede843