Add execution context to log records

[joshdover]

In order to support enhanced observability of Kibana itself, we should add a richer "execution context" to the New Platform's logging service that allows correlation between different log records that occur during an "execution context" such as a request or background task.

This context should automatically be attached to any log records produced during an execution context, ideally by leveraging the RequestHandlerContext pattern.

Fields that should be part of this context:

• A unique identifier for the execution context, such as the X-Opaque-Id header

Plugins should be able to leverage this identifier to find other useful information about these log events, such as the current user.

Related issue: #52125

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[pgayvallet]

Do we have an idea how we would 'bind' the logger's records to the request execution context in a node environment without tools such as Java's threadlocal ?

[joshdover]

@thomheymann is this a blocker for security anymore? As I understand it, this only affects regular log messages and should not affect the audit trail logs at all.

While I agree that adding the request ID to regular logs is very useful, I think we are fine if we delay this past 7.10.

[pgayvallet]

@elastic/kibana-security do you knoe if that is still a requierement on your side?

[legrego]

@elastic/kibana-security do you knoe if that is still a requierement on your side?

@pgayvallet I'm not sure what the original goal was. Our audit logs already include a trace id to understand everything that happens within the context of a specific KibanaRequest.

If we're looking for something that spans multiple requests (such as the loading of a dashboard), then we don't have a mechanism for that, yet. I think this would be a welcome addition to our audit events, but this isn't something we are currently prioritizing.

[pgayvallet]

I'm not sure what the original goal was. Our audit logs already include a trace id to understand everything that happens within the context of a specific KibanaRequest.

The idea is to be able to attach a traceId to all our log entries, so that we can track execution of a specific request more easily (so basically what you did for audit logging, but for all our logging system). This is similar to log4js' MDC.