[APM] Create a Logs tab in the Service view to show Logs per service

[jasonrhodes]

APM users would like to be able to see logs for a given service as a view within their Service page.

AC:

• Service view in APM has a new Logs tab next to existing tabs (Overview, Transactions, Errors, Service Map, etc)
• Logs tab uses the Log Stream component to show Logs for the given service, filtered on ECS field "service.name" for now
• Columns to show in this Log Stream component are TBD

Mockup from design issue:

[elasticmachine]

Pinging @elastic/apm-ui (Team:apm)

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[alex-fedotyev]

Hi @Kerry350!
Few comments important for implementation.

Populating logs would likely need to happen by indirectly loose matching APM and Logs.

• Step 1: Query which infrastructure components the APM service had been running on during selected timeframe (which needs to appreciate Kuery bar so it is possible to filter specific version or availability zone in APM).
• Step 2: Show all the logs entries across that infrastructure.

Particularly, this logic would be different for containerized service vs those which are running on VMs and hosts.

1. If "container.id" is present in transaction documents, query all "container.id" values and pass that as filter into logs.
2. Otherwise use "host.name" field same way otherwise.

Serverless will follow same logic once we add support for it.

[alex-fedotyev]

Oh, totally forgot to add to the design an ask to implement search input within the logs to enable quick filtering when looking into service logs.
Thank you!!

[jasonrhodes]

@alex-fedotyev thanks for the notes! I'd like to clarify a few points.

• When we met with @graphaelli and @ogupte, we settled on the above AC that meant we'd initially be querying for logs based on the current service.name value. From your comments, it sounds like you are picturing something different related to host, container, etc. Can we clarify so we can update the AC?
• You mention a query bar, can we make sure we know exactly what we want that to be and how it's meant to function? I imagine the outer "kuery bar" implementation is applied to the queries directed at APM documents and this new search bar is meant to be applied to the Logs query?

[alex-fedotyev]

My logic is primarily around the fact that service.name filter is going to be a significant limitation for the customers, as it would only work when ECS loggers are used and configured in the monitored applications.

On the other side customers often have logs enabled across Kubernetes environments or configured manually using filebeat on their hosts and VMs already, which means that service.name isn't going to be present but logs are still there.
That is why I suggested to consider a different type of query by doing kind of a "join" on either container.id or host.name.

---

I may take back the idea of search bar for now, just realized that with an approach to "join" APM transaction and logs - it would be unclear what kuery bar would apply to (logs or transactions).
My original idea was to introduce filtering by two type of data:

1. Ability to filter down by service metadata like service.version which is part of APM transactions of host name.
2. Ability to filter/search by specific messages like errors message, etc.

My understanding now is that it won't be really possible to get both of these working, right?

[Kerry350]

Thanks for the input, all. I'd like to try and tease out the specific acceptance criteria. How does the following sound:

• There should be a filter bar for filtering down on APM data, e.g. service.version etc. Example:

• The environments filter should be applied:

• There should be a date range selector for setting a date / time range.

• An APM API endpoint should exist which allows to query for infrastructure (containers, hosts etc) relevant to the service / filters. (GET /services/<serviceName>/infrastructure ?). Does anything exist like this at the moment? (I can look, but just in case I miss anything).

• A log stream component exists which queries based on the Infrastructure response (e.g. the relevant container IDs, host names etc).

Anything I've missed?

---

My understanding now is that it won't be really possible to get both of these working, right?

This is the conclusion @jasonrhodes and I reached yesterday in our sync meeting. The UX would be pretty weird as we'd have the two bars, searching across two types of data. It also adds a layer of complexity as our log stream component wraps all of the ceremony around getting the correct indices from our "Source configuration", but the additional search bar would have to live outside of the log stream component, and yet still have that "Source configuration" awareness.

[alex-fedotyev]

This looks great to me, one minor thing you missed - environments filter which is common across APM UI.

Feel free to drop in to the #apm-ui slack channel to clarify on the APM API endpoint.

Thanks @Kerry350 !

[alex-fedotyev]

Hi @Kerry350 !
Quick ask, could you please remove the "event.dataset" column and only keep time and message?
I suspect that dataset is likely going to be the same across all messages and will simply consume space.
Maybe we add other columns later, but that needs more research.