Make error logs without an underlying exception visible in Traces in AI

[jakubnabrdalik]

Is your feature request related to a problem? Please describe.

Currently the java agent will split log.error calls (using logback and Spring Boot default) between Exceptions and Traces in AI panel, based on whether there was an "Exception" parameter or not passed to the log.error call.

That is extremely unintuitive and makes developers build dashboards/monitoring that do not show real problems.

We've just found out all microservices had errors in monitoring due to that split.

When developers call log.error, if they have an exception from underlying code they will of course pass it to the log.error, so that there is more context visible (call stack etc.). However when there is no underlying exception, developers with experience in JVM or Functional Programming tend NOT to create an exception (because it's expansive on performance, because the stack doesn't have to be gathered, because for FP the situation is better modeled with a returned type), but just log.error. After all, log.error is for logging errors, no matter the source.

On the AI dashboard, a developer enters a query to see all the errors like so

traces
| where cloud_RoleName == "name of the microservice"
| where severityLevel > 2
| order by timestamp desc

and sees warns and errors. And indeed, some errors are visible there. All the developers we have, expected that "severityLevel > 2" gives them all the errors, and neither expected that errors reported with an underlying stack trace are not visible there.

We can get around that with an union in the query, like so

traces
| union exceptions
| where cloud_RoleName == "name of the microservice"
| where severityLevel > 2
| order by timestamp desc

but fields of traces and exceptions do not overlap, so if you are looking for a message (as in the log.error(message)) it will be either in the "message" field, or the "outerMessage" field, depending on whether it's a trace or an exception logged via log.error(exception).

Describe the solution you would like

1. Everything logged on error severity should be available in traces. In a perfect scenario, the additional fields (stack trace, etc) would be visible in customDimensions.

2. If the call to log was without a message, as in
   log.error(exception)
   the exception message should be available in the "message" field of the trace

Describe alternatives you have considered

Since our devs use functional programming in Kotlin/Java/Groovy, they do not throw exceptions when they detect an error, but model it with returned types and monitor with log.error calls (without an exception).

The only solution we found is to add "| union exceptions" to every dashboard, but it makes the number of columns in the grid too large, and requires complicating the dashboard query even more. Simple things should be simple, and this workaround is unnecessarly complex.

Additional context

Tested on Spring Boot (all versions) with AI agent 3.x, default logback configuration, calls via SLF4J.

[mateuszit]

I faced the issue as well. Would be nice to have it implemented

[JohanSpannare]

+1