[APM - Design] Configuring CCS from the APM UI

[nehaduggal]

This new setting in the APM UI will provide a way to configure cross cluster search (CCS) from within the UI instead of having to change the kibana.yml.

[elasticmachine]

Pinging @elastic/apm-ui

[katrin-freihofner]

@nehaduggal can you please provide more details?

• What do users need to configure for CCS? (which settings? Is it a list of clusters? Is it a pattern?
• Are there other things I can configure? (for example, skip_unavailable,...)
• Why should this be done from the APM UI - do you not consider this a global (Kibana) feature?
• Can this also be configured somewhere (kibana.yml) else? How? And will the setting in the UI overwrite the one in the kibana.yml? And can we show the config from the kibana.yml?
• Is there any additional information that a user will need to perform this successfully?
• Will this setting be applied immediately?
• How can the user test if this works as expected?

[sorenlouv]

What do users need to configure for CCS? (which settings? Is it a list of clusters? Is it a pattern?

It's actually quite simple to take advantage of CCS. All the user has to do is instruct the APM UI to look for data in remote clusters in addition to the local cluster. Normally the transaction index looks like:

apm_oss.transactionIndices: apm-*-transaction*

With CCS it should look like:

apm_oss.transactionIndices: *:apm-*-transaction*,apm-*-transaction*

(so basically we just add *:apm-*-transaction*)

[formgeist]

@sqren Just came across this issue - are we missing design still, or does that ticket exist somewhere else? I believe I saw another issue that described adding a new page to the APM Settings section?

[sorenlouv]

I think this will be a lot easier for users to do after #45247.
However, we could also add dedicated UI controls for enabling CCS, eg. a simple checkbox that indicates if you want to search across clusters or not.

The question is then: should users be able to enable CCS per index - or is it all or nothing?

[formgeist]

The question is then: should users be able to enable CCS per index - or is it all or nothing?

I imagine we can start with all and then make the settings granular later on if that makes it easier to implement?

[sorenlouv]

@formgeist Do you need anything from me wrt design of this feature?

[formgeist]

@sqren I gave it a shot and implemented a design based on your comments above. Still not sure whether we decided to go with an all or per index implementation, but here are some screens and demo that shows the design solution.

I'm wondering if what happens if a user overwrites an index option - will they need to explicitly declare the CCS value. I assume if they've called their indices something else across all clusters they would need to define those in the input too. So in the default case our we're just adding the extra *: to search across clusters, because we assume the default index names have been used. Is that the case?

[formgeist]

Based on some feedback from @sqren I've updated the proposed design. A bigger overhaul of the page and how we display each index block.

The changes include a dedicated option for CCS for each index which will add the CCS pattern lookup behind the scenes.

Figma prototype

---

[sorenlouv]

This looks great @formgeist. I intended for us to start this when #48079 got merged in but after talking to @roncohen I'm not sure if the current direction is feasible.

Some users might want to enable CCS but want to specify a specific cluster:

apm-*, us-east:apm-*

Whereas this solution only supports targeting all remote cluster:

apm-*, *:apm-*

Another theoretical issue (I do not know if this is an issue in practice) is that users cannot specify a different index name for their remote clusters:

apm-*, *:foo-*

Not sure if this should be supported or not though.

[formgeist]

@sqren OK, I understand the problem with not being able to define a specific cluster, or at least may not find that immediately possible in that one input field.

I'm suggesting we add a dedicated option for the cross cluster indices when you're enabling it per index. This means saving it as a separate input, not sure how that'll work for the queries? It would immediately default to wildcard all remote clusters with *:apm-* but perhaps it should copy over the existing input so it's really easy for the user to do cross cluster. So i.e. the user has defined apm-myfavouritetransactions we can put *:apm-myfavouritetransactions in the CCS input.

[formgeist]

Had a Zoom chat with @sqren and we came to the conclusion that a single input field is a lot simpler to manage, so we adapted the design to add an option to add CCS indices based on the existing indices (if any has been input), otherwise it will add a simple wildcard CCS parameter for *:apm-* based on the default parameter for each index.

The option to add indices only exists if there's no CCS parameter in the input field, therefore it goes away once the changes have been saved in the end.

Thoughts on the changes?

[sorenlouv]

@formgeist Visually I think the combobox looks better than just a regular input field. However I'm not sure it's worth it until there is support for copying, pasting and editing items.

I've created an issue in EUI here but there doesn't seem to be any movement around it.

[formgeist]

@sqren Definitely not a dealbreaker for implementing CCS support, so I agree we can do with just an input field for now and perhaps leave the combobox design as a design enhancement for when/if the EUI issue is picked up and fixed. At least we can push for some enhancements on that component, unless one of us wants to help extend it for our own use-case 🙂 I can create examples of the input version for the implementation issues.

[formgeist]

Closing in favor of #49732