[APM] Implement a dedicated tab for custom dashboards

[akhileshpok]

A dedicated tab should be available for custom dashboards. This tab should also have the time picker and the search bar. This functionality will leverage the recent work on custom dashboards in #163504

Figma link

Tasks

Beta Give feedback

1. Add a new tab and a page that can be used to link dashboards for each service
2. Add the unified search bar that can be used to filter the panels contained in the dashboards (stretch goal)
3. [APM] Add a KQL filter bar to select the services where the new dashboard will be linked to #167855
   Team:APM enhancement +2
   

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)

[boriskirov]

We've started some explorations of how to improve the interaction and structure of the Dashboards with @sqren

You can find them in the Figma link

[gbamparop]

As discussed, the scope for 8.11 will be an MVP that adds a new tab and a page that can be used to link dashboards for each service.

[teresaalvarezsoler]

@boriskirov @akhileshpok are there any reasons why users need to "Link" dashboards instead of selecting the dashboard from a list? This is a very different UX pattern than the one we use elsewhere and it introduces one extra step for selecting a dashboard.

[sorenlouv]

@teresaalvarezsoler Imagine that there are 100s of dashboards in a cluster. We only want to list the relevant dashboards in APM. I'm not attached to the terminology of "linking" but we need a way to select a subset of the dashboards available, and list them in APM.

Additionally, when a dashboard has been selected (linked) we specify which service(s) it should be available for. Some dashboards should show up for all services, some should only show up for java services, and some should only show up for a single service. The user specifies this in the "linking" step. Again, open to changing the terminology but some kind of step is needed.

Lastly: If the visualisations in the dashboard uses service.name and service.environment we will apply the selected service name and environment as filters. However, if the dashboard doesn't have these fields we shouldn't attempt to filter -
it'll result in a "no data" screen. So it should be possible for the user to opt-out of passing these filters. Again, that is part of the linking step.

LMK if you want to chat more about this and we can schedule a call.

[teresaalvarezsoler]

Thanks for the explanation @sqren, but I'm still not sure why we need the "Link" button, can you just show the list of available dashboards in the dropdown (highlighted below)? You can apply all the filters you mentioned and only show the relevant dashboards list there. You will get a cleaner UI and save users an extra step.

[achyutjhunjhunwala]

@teresaalvarezsoler We will miss the persistence logic here if the dashboard is not linked. As a user, for this particular service, whenever i land on this page, i would want Dashboard A to be present and for another Service Dashboard B. With linking we can persist this view when the user lands here. Else we will have to always default to an empty screen unless the user has selected a dashboard from the dropdown or a default dashboard which may not make much sense in terms of the service.

[teresaalvarezsoler]

Selecting a dashboard could work as "linking" it behind the scenes, the latest selected dashboard will be displayed next time the user enters this view. The only caveat I can think of is the dropdown selector to choose the dashboard would always need to be visible but that's much better than unlinking/linking in my opinion.

[sorenlouv]

but I'm still not sure why we need the "Link" button, can you just show the list of available dashboards in the dropdown (highlighted below)?

Are you suggesting to show all dashboards in the list? Again, this will provide a bad UX for people not familiar with the stack and just want to view the small handful of relevant dashboards for the selected service.
In Observability there is often a "Center Of Excellence" with expert knowledge about Elastic. They will curate the experience for the other users so they don't have to sift through the noise (100s of dashboards if only 1 or 2 are relevant).

[sorenlouv]

Selecting a dashboard could work as "linking" it behind the scenes, the latest selected dashboard will be displayed next time the user enters this view.

How will can we "link" behind the scenes? What will determine which filters should apply? Again, the end user might not know these details so it is important that someone can set this up beforehand.

[teresaalvarezsoler]

view the small handful of relevant dashboards for the selected service

I'm proposing to show only the relevant dashboards in the first dropdown

[sorenlouv]

I'm proposing to show only the relevant dashboards in the first dropdown

How will we determine which dashboards are relevant for the service, if they haven't been linked?

[teresaalvarezsoler]

How will we determine which dashboards are relevant for the service, if they haven't been linked?

In the same way you do it here:

[achyutjhunjhunwala]

@teresaalvarezsoler I think i understand the confusion here.
The reason why we have this as a 2 step process is we don't know which dashboard are relevant dashboards for the service. This is where the customers COE will come into action and will create this curated list for the SREs.

Step 1. - COE or equivalent will go through the list of ALL dashboards and start linking relevant dashboards like here

Step 2. - Now the SRE's already have a Curated list of relevant dashboards to select from the dropdown as they have been linked.

So most of the time, the SREs will only see 1 dropdown with the curated list of relevant dashboards (The step 2). The step 1 is something we don't expect to happen every time. Step 1 will be performed very few times by COE initially when the service is setup or in case new dashboards have been added.

So this is not always a 2 step process.

cc: @sqren

[teresaalvarezsoler]

Thanks for the explanation @achyutjhunjhunwala. It definitely makes more sense when there are two personas involved in the process. I think there are several assumptions in this design:

1. Users have a super long list of dashboards which makes it difficult to find a dashboard for a specific service and/or environment
2. Users need to change dashboards very often (that's why a "curated" dropdown list makes it easier than searching in the whole dashboard list everytime)
3. The list of "curated" dashboards doesn't change very often, i.e. end-users will not need to "link" new dashboards.
4. There are usually 2 personas and the admin will know that she can offer a better "curated" view to end-users.
5. End-users will understand that this dropdown is a "curated" view but they can still search for all dashboards if they need it by clicking on "link" new dashboard.

I'm not very knowledgable about your target users, but I guess you are fine with these assumptions. Maybe testing with users is a good idea (if you haven't done so) since it implies a very different pattern of how users find their dashboards elsewhere.

[sorenlouv]

I think that is a good list of assumption, and I think we are aware of most of them. I agree that it will not be a perfect fit for everyone (clusters where every dashboard is an APM-compatible dashboard for instance) but in the majority of cases I think it makes sense.

There are usually 2 personas and the admin will know that she can offer a better "curated" view to end-users.

There doesn't necessarily need to be two personas involved. Even with just a single user it makes a lot of sense to reduce a long list of 100s of dashboards to just the 1 or 2 relevant ones.

End-users will understand that this dropdown is a "curated" view

I can't say for sure that all users will understand this up front. I think we are releasing this under a beta badge. We'll iterate on it as we get feedback.

[marcomusso]

My 2c: wider dropdown menu of dashboards (especially if they have already a common prefix like the default dashboards from beats) and why don't have the option of selecting (or just filtering) dashboards by tags (either in the dropdown menu or globally in a Settings page)?